Voice Agent observability with LangSmith 🌟
Summary
This session demonstrates how to achieve deep observability for voice agents using LangSmith when integrating Google's Gemini Live model and the Google ADK. Since Gemini Live is a native speech-to-speech model that bypasses text transcription to maintain low latency, robust tracing is critical. The process involves setting up tools (like a weather assistant), recording both user and agent audio, and utilizing LangSmith to view comprehensive traces that include transcripts, tool calls, interruption events, full cost breakdowns, and even audio playback for debugging.
Key takeaways
-
Gemini Live's Low-Latency Advantage
Gemini Live is Google's native speech-to-speech model; it takes audio directly as input and produces audio output without transcribing to text, which keeps latency low and ensures a natural voice experience.
-
Comprehensive Voice Agent Tracing
LangSmith provides visibility into the agent's internal workings, capturing not only standard transcripts and tool calls but also specific events like interruptions and detailed token-level cost breakdowns.
-
Production Readiness Tools
The observability provided by LangSmith allows engineers to perform standard LLM operations—such as running evals, adding traces to data sets, building dashboards, and debugging—on complex voice agent interactions.
Technical details
-
Voice Agent Architecture
0s
The demonstration involves building a terminal-based weather assistant using the Google ADK and Gemini Live. The process includes defining and registering a LangSmith Google ADK plugin on the runner.
-
Trace Data Capture
0s
The system is configured to record user and agent audio, ensuring that the trace captures exactly what the user heard. The resulting LangSmith trace includes transcripts, tool call records, interruption events, and cost breakdowns.
-
LangSmith Functionality
0s
Engineers can leverage LangSmith to debug voice agents by reviewing the full trace, which supports standard text-based features like running evals, managing annotation queues, and sharing detailed performance metrics.
Mentioned resources
Channel & topics
Watch on YouTube · Back to latest
This independent, AI-assisted summary is provided for commentary and informational purposes. It may contain errors or omit important context. Please watch the original video for the creator's complete presentation. Video, thumbnail, and related copyrights belong to their respective owners.