Evals for High-Stakes AI Agents — Doneyli De Jesus & Annabell Schäfer, Langfuse
Summary
This workshop provides a deep dive into evaluating high-stakes AI agents, particularly for regulated industries like finance. The core message is that traditional observability is insufficient for non-deterministic LLM systems. The speakers outline the systematic 'AI Engineering Loop,' which mandates continuous improvement through structured practices: detailed tracing, dataset curation, running experiments, and implementing robust evaluation (Evals) that can be integrated into CI/CD pipelines to prevent regressions before deployment.
Key takeaways
-
The Problem with Non-Deterministic AI
AI agents can fail silently; system metrics may look 'green' (e.g., latency is fine), but the agent may fail to execute critical steps (like issuing a refund) or hallucinate actions. This necessitates specialized evaluation methods beyond traditional monitoring.
-
The AI Engineering Loop
20:47
Continuous improvement requires a systematic loop: 1) Tracing (logging all steps, inputs, and outputs); 2) Monitoring (finding aggregate signals and failure modes); 3) Building Datasets (creating representative test cases); 4) Running Experiments (varying prompts, models, or tools); and 5) Evaluation (measuring quality).
-
Implementing Quality Gates in CI/CD
The process can be automated by integrating evaluations into the CI/CD pipeline. By triggering a CI job on any prompt change, the system can run experiments against a defined dataset and fail the build if quality scores drop or guardrails are violated.
-
Hybrid Evaluation Approach
50:07
For maximum reliability, a mix of evaluators is recommended: LLM-as-judge (using one LLM to score another's output), deterministic code evaluators (for exact matches), and custom guardrail checks (e.g., blocking investment advice).
Technical details
-
Tracing and Data Masking
1247s
Tracing captures the entire execution path, including root traces, individual steps, tool calls, and metadata. For regulated industries, PII identification and masking are critical, ensuring sensitive data (like credit card numbers) is redacted before persistence.
-
Dataset Curation and Annotation
2035s
Datasets serve as the 'test cases' for the agent. Ground truth data can be built via expert curation, shadow runs (running the agent parallel to human processes), or by annotating production traces. Domain experts provide feedback by scoring outputs against defined failure modes (e.g., missing citations).
-
Evaluation Methods
3007s
Evaluation approaches include LLM-as-judge (using an LLM to assess quality) and deterministic code evaluators (for 100% accurate checks). The goal is to establish measurable 'scores' for quality dimensions like source recall, language match, and compliance.
-
System Architecture and Deployment
3212s
The platform supports self-hosted deployments and utilizes ClickHouse for data storage due to the massive scale of observed events. Data can be consumed via API, CLI, or MCP.
Mentioned resources
- Langfuse
- AIE CODE summit
- Langfuse Academy
Channel & topics
Watch on YouTube · Back to latest
This independent, AI-assisted summary is provided for commentary and informational purposes. It may contain errors or omit important context. Please watch the original video for the creator's complete presentation. Video, thumbnail, and related copyrights belong to their respective owners.