# Voice Agent observability with LangSmith

## Executive summary

This session details how to implement robust observability for voice agents built using the Google ADK and Gemini Live model by integrating LangSmith tracing. The process involves defining a custom plugin that captures not only the conversation transcript but also the full audio stream (user input and agent output). This visibility allows engineers to debug complex interactions, analyze tool usage, track interruption events, and monitor token-level costs for production readiness.

## Key takeaways

- Gemini Live Model Functionality: Gemini Live is Google's native audio model that operates in a speech-to-speech manner. It takes audio directly as input and produces audio as output without transcribing to text, resulting in low latency and natural, emotive voice quality.
- LangSmith for Observability: LangSmith is a platform built by LangChain specifically for AI agent observability and evaluations. It provides visibility into the internal workings of the voice agent, which is crucial for safe production deployment.
- Capturing Conversation Audio: To ensure the 'source of truth' for a voice interaction is captured, the tracing setup must include functions to record both user audio and agent audio. Recording agent audio requires careful placement (e.g., using a `set play callback` on the audio out class) to capture only what the user actually heard.

## Technical details

- Voice Agent Architecture: The demonstration uses a simple terminal-based weather assistant built with Google ADK, which has access to two tools: one for getting the time and another for getting the weather in any city.
- LangSmith Integration Setup: Tracing requires defining a custom LangSmith ADK plugin. This involves setting configurations (like project name, tags, metadata) and registering the plugin on the runner. The plugin handles the core tracing logic.
- Trace Analysis Capabilities: The resulting LangSmith trace provides comprehensive details, including: full audio playback (user/agent segments), conversation transcripts, detailed tool calls (including arguments and duration), event tracking (e.g., 'turn completed', 'interruption event'), and token-level cost breakdowns for accountability.

## Practical implications

- Implementing LangSmith tracing is the critical step for moving a voice agent from development to production by providing deep visibility into its behavior.
- The ability to capture and analyze audio recordings in the trace ensures that the source of truth for customer interactions is preserved, even when interruptions occur.
- Engineers can use these traces to run evaluations (evals), build dashboards, debug complex flows, and manage cost accountability via token usage tracking.

## Topics

Voice Agents, Observability, LangSmith, Google ADK, Gemini Live, AI Agent Development, Gemini Live API docs, Google ADK docs, LangSmith docs

Source: https://www.youtube.com/watch?v=z6l5nyE_6-I