Voice Agent observability with LangSmith
Summary
This session details how to implement robust observability for voice agents built using the Google ADK and Gemini Live model by integrating LangSmith tracing. The process involves defining a custom plugin that captures not only the conversation transcript but also the full audio stream (user input and agent output). This visibility allows engineers to debug complex interactions, analyze tool usage, track interruption events, and monitor token-level costs for production readiness.
Key takeaways
-
Gemini Live Model Functionality
Gemini Live is Google's native audio model that operates in a speech-to-speech manner. It takes audio directly as input and produces audio as output without transcribing to text, resulting in low latency and natural, emotive voice quality.
-
LangSmith for Observability
2:05
LangSmith is a platform built by LangChain specifically for AI agent observability and evaluations. It provides visibility into the internal workings of the voice agent, which is crucial for safe production deployment.
-
Capturing Conversation Audio
4:00
To ensure the 'source of truth' for a voice interaction is captured, the tracing setup must include functions to record both user audio and agent audio. Recording agent audio requires careful placement (e.g., using a `set play callback` on the audio out class) to capture only what the user actually heard.
Technical details
-
Voice Agent Architecture
150s
The demonstration uses a simple terminal-based weather assistant built with Google ADK, which has access to two tools: one for getting the time and another for getting the weather in any city.
-
LangSmith Integration Setup
280s
Tracing requires defining a custom LangSmith ADK plugin. This involves setting configurations (like project name, tags, metadata) and registering the plugin on the runner. The plugin handles the core tracing logic.
-
Trace Analysis Capabilities
370s
The resulting LangSmith trace provides comprehensive details, including: full audio playback (user/agent segments), conversation transcripts, detailed tool calls (including arguments and duration), event tracking (e.g., 'turn completed', 'interruption event'), and token-level cost breakdowns for accountability.
Mentioned resources
Channel & topics
Watch on YouTube · Back to latest
This independent, AI-assisted summary is provided for commentary and informational purposes. It may contain errors or omit important context. Please watch the original video for the creator's complete presentation. Video, thumbnail, and related copyrights belong to their respective owners.