Event Recap: Build Smarter Voice Agents - New York Edition
This recap details the complexities of building and deploying production-grade voice AI agents across two distinct sectors: professional networking (Boardy) and regulated healthcare (Flagler Health). Key challenges discussed include maintaining conversational flow, establishing user trust, managing multi-party video meeting interactions, and ensuring subsecond latency for natural conversation. The discussion highlights the difference between highly structured, goal-oriented flows (healthcare) and highly conversational, relationship-driven interactions (networking).
Key takeaways
-
Design Flow Differences
10:20
Healthcare voice agents require highly structured, step-by-step flows with strict guardrails (e.g., collecting insurance info) to prevent medical advice or deviation. Conversely, networking agents are designed to handle highly conversational, open-ended interactions to facilitate connections.
-
The Importance of Trust and Disclosure
21:20
Building user trust is critical. Speakers emphasized that being upfront and immediately disclosing that the user is speaking to an AI (e.g., 'I'm Sarah and AI') is essential to prevent user frustration and loss of trust.
-
Technical Challenge: Multi-Party Meetings
24:10
Handling voice agents in multi-person video meetings (like Google Meet) is technically difficult. The primary challenge is determining when the agent should speak (turn-taking) to avoid false positives (randomly jumping in) or false negatives (failing to reply).
-
Achieving Low Latency
25:00
To feel like a natural conversation, the system must achieve subsecond latency. This requires advanced architecture, such as preemptively generating the entire voice pipeline while the user is speaking.