The Video Signal

Topic

Guardrails

All digests tagged Guardrails

· 21:17

Evals-Driven Development for a Mental Health AI Coach — Akele Reed & Dave Revere, SonderMind

SonderMind details its approach to building Sonder, a clinically grounded AI coach for mental health support. The core focus is on achieving safety and effectiveness through eval-driven development, moving beyond general-purpose LLMs which are unsuitable for high-stakes care. The architecture employs modular guardrails (Input/Output) within a Supervisor/Executor/Evaluator framework. Safety is maintained by integrating clinical judgment into the CI pipeline via annotated traces, ensuring that every model or prompt change is scored against expert-defined 'evals' before deployment.

Key takeaways

  1. Specialized AI for Mental Health 2:00

    General purpose LLMs are insufficient for mental health care due to the high stakes involved. Sonder was designed specifically to act as a front door to human therapy or provide support between sessions, addressing a critical gap in current AI offerings.

  2. Robust Architecture and Guardrails 4:00

    The system uses separate Input and Output guardrails—acting as 'sandwich' layers—to monitor user input and the entire conversation flow for clinical safety risks. This modularity allows iteration without compromising safety.

  3. Clinical Feedback Loop in CI 9:40

    Safety is enforced by capturing edge-case conversations (traces) and having licensed professionals annotate them. These annotations generate typed 'evals' that gate releases, ensuring every model or prompt change is tested against real-world clinical judgment.

  4. Focus on Correct Triggers 6:10

    The guardrails are calibrated not to generate more alerts (false positives), but to correctly identify genuine safety risks. The system must distinguish between a user in an active crisis and one seeking past support.

Watch on YouTube Full article

· 1:06:04

Why This Company Won't Let AI Agents Touch Bash

The presentation details the evolution of an internal AI agent platform at Cyera, transforming a personal assistant project into an enterprise-wide tool for data security and operational efficiency. The core focus is on building robust guardrails to prevent agents from becoming uncontrolled 'black boxes.' Key architectural innovations include whitelisting tools over blacklisting them, implementing a structured citation system for verifiable claims, and replacing traditional RAG with a Knowledge Graph (KG) that allows agents to navigate interconnected data like an LLM wiki. The platform emphasizes controlled deployment, allowing multiple developers to build and own specialized agents while maintaining centralized security and governance.

Key takeaways

  1. Controlled Agent Architecture

    The system prioritizes control by whitelisting specific tools rather than blacklisting forbidden actions. A critical guardrail is the use of structured, validated output, ensuring that an agent cannot execute arbitrary code (like unrestricted Bash) or leak sensitive data outside its designated context.

  2. Citation and Hallucination Mitigation 20:55

    To ensure reliability, every claim generated by the agent must be backed by a citation. This is achieved by forcing the model to output structured blocks containing both the claim and the source reference. A second model then performs clean-context verification against the raw data to drastically reduce hallucinations.

  3. Knowledge Graph over RAG 32:33

    The platform utilizes a Knowledge Graph (KG) instead of standard RAG for context retrieval. This allows agents to 'walk' connections between entities (e.g., an exception, a service, and the related pull request), providing more structured and reliable data exploration than simply dumping retrieved documents.

  4. Adoption through Platform Engineering 50:27

    To drive adoption across engineering teams, the platform was designed to be easily customizable. By allowing users to name and modify their own agents (e.g., 'It's my agent'), the barrier to entry is lowered, promoting organic growth with a strategy of 'carrots, not sticks.'

Watch on YouTube Full article