Voice Agent observability with LangSmith
This session details how to implement robust observability for voice agents built using the Google ADK and Gemini Live model by integrating LangSmith tracing. The process involves defining a custom plugin that captures not only the conversation transcript but also the full audio stream (user input and agent output). This visibility allows engineers to debug complex interactions, analyze tool usage, track interruption events, and monitor token-level costs for production readiness.
Key takeaways
-
Gemini Live Model Functionality
Gemini Live is Google's native audio model that operates in a speech-to-speech manner. It takes audio directly as input and produces audio as output without transcribing to text, resulting in low latency and natural, emotive voice quality.
-
LangSmith for Observability
2:05
LangSmith is a platform built by LangChain specifically for AI agent observability and evaluations. It provides visibility into the internal workings of the voice agent, which is crucial for safe production deployment.
-
Capturing Conversation Audio
4:00
To ensure the 'source of truth' for a voice interaction is captured, the tracing setup must include functions to record both user audio and agent audio. Recording agent audio requires careful placement (e.g., using a `set play callback` on the audio out class) to capture only what the user actually heard.