Evals-Driven Development for a Mental Health AI Coach — Akele Reed & Dave Revere, SonderMind
Summary
SonderMind details its approach to building Sonder, a clinically grounded AI coach for mental health support. The core focus is on achieving safety and effectiveness through eval-driven development, moving beyond general-purpose LLMs which are unsuitable for high-stakes care. The architecture employs modular guardrails (Input/Output) within a Supervisor/Executor/Evaluator framework. Safety is maintained by integrating clinical judgment into the CI pipeline via annotated traces, ensuring that every model or prompt change is scored against expert-defined 'evals' before deployment.
Key takeaways
-
Specialized AI for Mental Health
2:00
General purpose LLMs are insufficient for mental health care due to the high stakes involved. Sonder was designed specifically to act as a front door to human therapy or provide support between sessions, addressing a critical gap in current AI offerings.
-
Robust Architecture and Guardrails
4:00
The system uses separate Input and Output guardrails—acting as 'sandwich' layers—to monitor user input and the entire conversation flow for clinical safety risks. This modularity allows iteration without compromising safety.
-
Clinical Feedback Loop in CI
9:40
Safety is enforced by capturing edge-case conversations (traces) and having licensed professionals annotate them. These annotations generate typed 'evals' that gate releases, ensuring every model or prompt change is tested against real-world clinical judgment.
-
Focus on Correct Triggers
6:10
The guardrails are calibrated not to generate more alerts (false positives), but to correctly identify genuine safety risks. The system must distinguish between a user in an active crisis and one seeking past support.
Technical details
-
Modular Guardrail System
240s
The architecture includes separate Input Guardrails (checking user message) and Output Guardrails (checking the AI response/conversation state). Keeping these guardrails as separate LLMs with dedicated 'judge calls' makes them more robust, harder to circumvent via prompt engineering, and improves reliability despite potential trade-offs in latency and cost.
-
Eval-Driven Safety Testing
580s
The process involves capturing conversation traces that are annotated by clinicians. This annotation generates a typed eval, which includes the conversation input, expected result, and expected observation/assertion. These structured evals then become mandatory gates for any prompt or model change.
-
System Calibration Philosophy
690s
The design prioritizes 'correct triggers' over preventing all alerts, acknowledging that overly conservative guardrails can isolate users. The focus is on creating benchmarks that serve real human needs by analyzing real failure modes (false positives and false negatives matter).
-
Open Source Shared Baseline
980s
To accelerate safety across the industry, SonderMind open-sourced its data sets, providing 200 input guardrail scenarios and 100 output guardrail scenarios that have been clinically reviewed and calibrated.
Mentioned resources
- SonderMind
- Headspace, Ethna, Anthem
Channel & topics
Watch on YouTube · Back to latest
This independent, AI-assisted summary is provided for commentary and informational purposes. It may contain errors or omit important context. Please watch the original video for the creator's complete presentation. Video, thumbnail, and related copyrights belong to their respective owners.