# I Monitored Crime Audio. Voice Agents Scare Me More. — Sumanyu Sharma, Hamming AI

## Executive summary

The presentation compares the monitoring of decentralized, hyper-local crime data (Hamming's initial work) with the rapidly scaling, centralized risks of conversational voice agents. While voice AI is advancing rapidly, reliability remains the primary blocker for large-scale deployment. The speaker emphasizes that because voice agents are centralized, a single prompt or architectural change can have a massive 'blast radius.' He advocates for a continuous monitoring loop—including deep manual analysis, frequency/severity prioritization, and adversarial red teaming—to mitigate risks like unauthorized actions, incorrect information provision, and the leakage of PHI/PII.

## Key takeaways

- Voice Agents vs. Crime Monitoring: Crime incidents are generally hyper-local and decreasing, while voice agent usage is centralized and rapidly increasing, potentially handling a trillion calls annually. This centralization means a single failure point can impact millions of users.
- The Scale of Risk: If a 1% error rate is assumed across annual calls, this equates to 10 billion potential bad interactions. In practice, monitoring 10,000 agents shows an error rate closer to 10%, manifesting as skipping eligibility checks or providing incorrect information.
- The Continuous Improvement Loop: Fixing voice agent reliability requires a structured loop: Identify problems, prioritize by frequency and severity, understand the fix, execute the change, verify it hasn't caused regressions, and continue monitoring in production.

## Technical details

- Voice AI Architecture & Reliability: Improvements are driven by better underlying infrastructure and orchestration layers. Teams are experimenting with hybrid architectures combining multiple voice-to-voice modalities and cascading stacks to maintain reliability while keeping latency low.
- Testing and Validation: To validate fixes, the speaker recommends moving beyond simple replaying of real calls. Better methods include keeping the intent constant but changing wordings, patterns, accents, or adding new intents. For critical components (like outbound agents), A/B testing is critical for real-life performance validation.
- Adversarial Testing and Data Security: The surface area for attack increases with capability. The risk is that agents can be tricked into revealing sensitive data (PHI/PII). The defense requires deep investment in pre-deployment testing (text-to-text or voice-to-voice) and running 24/7 red teaming.

## Practical implications

- Implement a structured monitoring process that prioritizes failures based on both frequency and severity (P0 targets).
- Do not skip manual listening to individual calls; this provides crucial qualitative insights that quantitative metrics miss.
- Treat voice agent deployment as a continuous process requiring constant adversarial testing and monitoring, especially when handling sensitive data (PHI/PII).
- Use A/B testing and red teaming to validate changes and ensure that fixes do not introduce regressions.

## Topics

AI Ethics, Conversational AI, System Reliability, Natural Language Processing (NLP), Build Engineering, Risk Management, Sumanyu Sharma (Speaker), Hamming AI

Source: https://www.youtube.com/watch?v=qStB9GbppMU