Topic

LiveKit

All digests tagged LiveKit

100x More Reliable Voice Agents thumbnail

· 14:35

100x More Reliable Voice Agents

Unmute is introduced as an open, declarative standard for building voice agents, designed to solve the critical problem of vendor lock-in and architectural fragility in the current voice AI landscape. Current agents are often embedded within specific platforms (e.g., ElevenLabs, VAPI, LiveKit), making them non-portable. Unmute shifts the paradigm by declaring the agent's behavior, tool calls, data flows, and failure handling in a single, compiled package, making the LLM an optional resource rather than the sole decision-making spine. This approach ensures structural guarantees and portability across different cloud runtimes.

Key takeaways

  1. Declarative Execution and Portability 2:00

    Unmute allows the agent's behavior to be defined in a single, portable package that compiles to various runtimes (e.g., PipeChat Cloud, LiveKit Cloud, or Slang's architecture). This means the agent's core logic remains version-controlled IP, regardless of the underlying infrastructure or cloud region.

  2. Addressing LLM Over-Reliance 3:00

    Traditional agents assume the LLM is the decision-making spine, leading to unpredictable, expensive, and difficult-to-govern behavior. Unmute enforces execution through declared steps, making the LLM an optional resource that must 'earn its place' in the call.

  3. Structural Guarantees over Instructions 4:00

    The standard moves beyond the false choice between rigid IVR and complex single-prompt agents. By splitting the work into discrete, context-limited steps, the system achieves determinism—allowing specific rules (like a regulated disclosure) to be locked down while other parts remain generative, all within the same call.

Watch on YouTube Full article

Event Recap: Build Smarter Voice Agents - New York Edition thumbnail

· 29:13

Event Recap: Build Smarter Voice Agents - New York Edition

This recap details the complexities of building and deploying production-grade voice AI agents across two distinct sectors: professional networking (Boardy) and regulated healthcare (Flagler Health). Key challenges discussed include maintaining conversational flow, establishing user trust, managing multi-party video meeting interactions, and ensuring subsecond latency for natural conversation. The discussion highlights the difference between highly structured, goal-oriented flows (healthcare) and highly conversational, relationship-driven interactions (networking).

Key takeaways

  1. Design Flow Differences 10:20

    Healthcare voice agents require highly structured, step-by-step flows with strict guardrails (e.g., collecting insurance info) to prevent medical advice or deviation. Conversely, networking agents are designed to handle highly conversational, open-ended interactions to facilitate connections.

  2. The Importance of Trust and Disclosure 21:20

    Building user trust is critical. Speakers emphasized that being upfront and immediately disclosing that the user is speaking to an AI (e.g., 'I'm Sarah and AI') is essential to prevent user frustration and loss of trust.

  3. Technical Challenge: Multi-Party Meetings 24:10

    Handling voice agents in multi-person video meetings (like Google Meet) is technically difficult. The primary challenge is determining when the agent should speak (turn-taking) to avoid false positives (randomly jumping in) or false negatives (failing to reply).

  4. Achieving Low Latency 25:00

    To feel like a natural conversation, the system must achieve subsecond latency. This requires advanced architecture, such as preemptively generating the entire voice pipeline while the user is speaking.

Watch on YouTube Full article

Agent Context Carryover in Livekit Tutorial thumbnail

· 8:07

Agent Context Carryover in Livekit Tutorial

This tutorial demonstrates 'Agent Context Carryover,' a feature released for LiveKit using AssemblyAI's Universal 3.5 Pro model. This capability automatically feeds conversational context into the speech-to-text model, significantly boosting transcription accuracy—especially for proper nouns and key terms—without requiring manual plumbing or explicit context prompting from the developer.

Key takeaways

  1. Agent Context Carryover Functionality

    The feature automatically provides conversational context to the model, improving transcription accuracy when building voice agents on LiveKit. This is achieved by enabling a single parameter rather than implementing complex key term handling.

  2. Implementation Simplicity

    AssemblyAI's LiveKit plugin handles the necessary context plumbing automatically, allowing developers to gain accuracy benefits without setting up custom logic for key terms or sending agent messages manually.

Watch on YouTube Full article