Conversational AI — The Video Signal

Sep 16, 2026 · 20:41

Rebuilding the web for agents — Liad Yosef, MCP Apps

The talk outlines the transition to the 'agentic web,' arguing that traditional websites and browser tabs are becoming obsolete as primary interfaces. The core shift involves breaking down complex UIs into atomic components that can be composed by personal AI assistants. The speaker, a co-creator of the MCP Apps specification, details how MCP Apps allows servers to hand real UI interfaces back into a chat context, solving the 'last mile of interaction.' Furthermore, the talk introduces the need for a standardized discovery layer, proposing the use of `AI catalog.json` to make websites 'agent ready' and accessible to autonomous agents.

Key takeaways

MCP Apps Solves Last-Mile Interaction 1:42
MCP Apps allows providers to send UI chunks into chats, ensuring that services retain their brand identity and functionality rather than being reduced to text or databases. This solves the 'last mile' problem for agents, enabling complex interactions like booking hotels or viewing 3D models within a unified chat context.
The Agentic Web Paradigm Shift 5:40
The web is shifting from being a collection of siloed websites (requiring multiple tabs and different UIs for the same intent) to being composed by personal assistants. Assistants become the primary entry point, making websites themselves less critical for task completion.
Need for Agent-Specific Discovery 19:16
Traditional standards like `llms.txt` are insufficient because agents often ignore them, preferring to navigate to docs or homepages. The solution requires a standardized, machine-readable directory, such as the proposed `AI catalog.json`, to expose a website's MCP and API servers directly to agents.

Watch on YouTube Full article

Sep 15, 2026 · 20:25

Act, Confirm, or Stop? Smarter behavior for AI assistants, wearables & robots — Amit Desai, Roku

The presentation argues that improving voice AI user experience requires focusing on a second, often neglected dimension: system behavior under uncertainty. While increasing accuracy (Knob One) is critical, the system's ability to intelligently decide what to do when it is unsure (Knob Two) can yield greater user satisfaction. This is quantified using the Outcome User Cost Heuristic (OUCH), which minimizes the total user effort by assigning differential costs to various bad outcomes (e.g., playing the wrong song vs. simply stating 'I did not understand').

Key takeaways

The Two Knobs of Voice AI Improvement 0:03
User satisfaction can be improved by increasing technical accuracy (Knob One) or by optimizing the system's decision-making process when confidence is low (Knob Two). The latter is often overlooked.
The Outcome User Cost Heuristic (OUCH) 0:10
Instead of treating all errors equally, OUCH minimizes the total user cost by quantifying the relative pain of different bad outcomes (e.g., the effort required to stop a wrong song vs. the time taken to hear 'Sorry, I did not understand').
Adding Conversational Behavior 0:13
Introducing a third behavior—confirming the guess out loud (e.g., 'Did you mean ABC?')—splits the confidence range into three regions (Stop, Confirm, Act) and further lowers the overall user cost.

Watch on YouTube Full article

Sep 15, 2026 · 16:04

I Monitored Crime Audio. Voice Agents Scare Me More. — Sumanyu Sharma, Hamming AI

The presentation compares the monitoring of decentralized, hyper-local crime data (Hamming's initial work) with the rapidly scaling, centralized risks of conversational voice agents. While voice AI is advancing rapidly, reliability remains the primary blocker for large-scale deployment. The speaker emphasizes that because voice agents are centralized, a single prompt or architectural change can have a massive 'blast radius.' He advocates for a continuous monitoring loop—including deep manual analysis, frequency/severity prioritization, and adversarial red teaming—to mitigate risks like unauthorized actions, incorrect information provision, and the leakage of PHI/PII.

Key takeaways

Voice Agents vs. Crime Monitoring 7:12
Crime incidents are generally hyper-local and decreasing, while voice agent usage is centralized and rapidly increasing, potentially handling a trillion calls annually. This centralization means a single failure point can impact millions of users.
The Scale of Risk 8:43
If a 1% error rate is assumed across annual calls, this equates to 10 billion potential bad interactions. In practice, monitoring 10,000 agents shows an error rate closer to 10%, manifesting as skipping eligibility checks or providing incorrect information.
The Continuous Improvement Loop 11:44
Fixing voice agent reliability requires a structured loop: Identify problems, prioritize by frequency and severity, understand the fix, execute the change, verify it hasn't caused regressions, and continue monitoring in production.

Watch on YouTube Full article

Sep 14, 2026 · 29:13

Event Recap: Build Smarter Voice Agents - New York Edition

This recap details the complexities of building and deploying production-grade voice AI agents across two distinct sectors: professional networking (Boardy) and regulated healthcare (Flagler Health). Key challenges discussed include maintaining conversational flow, establishing user trust, managing multi-party video meeting interactions, and ensuring subsecond latency for natural conversation. The discussion highlights the difference between highly structured, goal-oriented flows (healthcare) and highly conversational, relationship-driven interactions (networking).

Key takeaways

Design Flow Differences 10:20
Healthcare voice agents require highly structured, step-by-step flows with strict guardrails (e.g., collecting insurance info) to prevent medical advice or deviation. Conversely, networking agents are designed to handle highly conversational, open-ended interactions to facilitate connections.
The Importance of Trust and Disclosure 21:20
Building user trust is critical. Speakers emphasized that being upfront and immediately disclosing that the user is speaking to an AI (e.g., 'I'm Sarah and AI') is essential to prevent user frustration and loss of trust.
Technical Challenge: Multi-Party Meetings 24:10
Handling voice agents in multi-person video meetings (like Google Meet) is technically difficult. The primary challenge is determining when the agent should speak (turn-taking) to avoid false positives (randomly jumping in) or false negatives (failing to reply).
Achieving Low Latency 25:00
To feel like a natural conversation, the system must achieve subsecond latency. This requires advanced architecture, such as preemptively generating the entire voice pipeline while the user is speaking.

Watch on YouTube Full article

Aug 25, 2026 · 36:31

⏭️ Forward Deployed: Voice AI on what works in 2026

The discussion explores the advanced engineering challenges of deploying real-time voice AI agents in enterprise settings. Speakers emphasize that achieving human-like performance requires complex, multi-stage cascaded pipelines (Speech-to-Text $\rightarrow$ LLM $\rightarrow$ Text-to-Speech), rather than simple end-to-end Voice-to-Voice models. Key challenges include managing latency trade-offs, ensuring reliability through model fallbacks, solving non-trivial problems like turn-taking, and architecting systems to handle complex context while maintaining low cost.

Key takeaways

Architectural Standard: Cascaded Pipeline 3:45
The current state-of-the-art for reliable voice agents is a cascaded pipeline (Speech-to-Text $\rightarrow$ LLM $\rightarrow$ Text-to-Speech). While Voice-to-Voice models are appealing, they currently lack the necessary reliability and control required for enterprise use cases. [0:03:45]
The Latency vs. Intelligence Trade-off 3:15
There is an inherent trade-off between the intelligence of the response and the latency. Achieving highly intelligent responses often results in slower processing, which must be managed to feel natural to the end user. [0:03:15]
Complexity of Conversation Flow 3:05
Solving conversational nuances like turn-taking (knowing when a pause means thinking versus ending the conversation) is not trivial. Furthermore, systems must account for different use cases: outbound calls are easier because guardrails can be strict, while inbound calls require handling unknown user context. [0:03:25]
Optimizing LLM Context and Cost 3:58
Due to the limitations of LLMs (forgetting information in the middle of a massive prompt), advanced architectures must employ context optimization techniques. This includes using specialized models or implementing 'compaction' methods, which is becoming critical for both performance and unit economics. [0:03:58]

Watch on YouTube Full article

Jul 25, 2026 · 21:17

Evals-Driven Development for a Mental Health AI Coach — Akele Reed & Dave Revere, SonderMind

SonderMind details its approach to building Sonder, a clinically grounded AI coach for mental health support. The core focus is on achieving safety and effectiveness through eval-driven development, moving beyond general-purpose LLMs which are unsuitable for high-stakes care. The architecture employs modular guardrails (Input/Output) within a Supervisor/Executor/Evaluator framework. Safety is maintained by integrating clinical judgment into the CI pipeline via annotated traces, ensuring that every model or prompt change is scored against expert-defined 'evals' before deployment.

Key takeaways

Specialized AI for Mental Health 2:00
General purpose LLMs are insufficient for mental health care due to the high stakes involved. Sonder was designed specifically to act as a front door to human therapy or provide support between sessions, addressing a critical gap in current AI offerings.
Robust Architecture and Guardrails 4:00
The system uses separate Input and Output guardrails—acting as 'sandwich' layers—to monitor user input and the entire conversation flow for clinical safety risks. This modularity allows iteration without compromising safety.
Clinical Feedback Loop in CI 9:40
Safety is enforced by capturing edge-case conversations (traces) and having licensed professionals annotate them. These annotations generate typed 'evals' that gate releases, ensuring every model or prompt change is tested against real-world clinical judgment.
Focus on Correct Triggers 6:10
The guardrails are calibrated not to generate more alerts (false positives), but to correctly identify genuine safety risks. The system must distinguish between a user in an active crisis and one seeking past support.

Watch on YouTube Full article