Designing Agents (The Floor Is the Frontier) — Ben Hylak, Raindrop
Summary
The talk argues that traditional evaluation methods designed for simple chatbots are inadequate for modern AI agents. The focus must shift from merely finding issues to 'raising the floor'—mitigating catastrophic worst-case failures that break user trust (e.g., recommending a competitor or deleting data). Practically, this requires treating agent evaluations as robust code tests rather than relying on prompt playgrounds, and focusing analysis on issue start dates and affected user percentages.
Key takeaways
-
Focus on Raising the Floor (Safety)
17:33
The most critical concern for agents is not their maximum capability (the ceiling), but their worst-case behavior (the floor). The floor represents failures that break user trust, such as accidental data deletion or sending inappropriate communications.
-
Evals Must Be Code, Not Prompts
12:00
Evaluation suites should be implemented as code (like unit or end-to-end tests) rather than relying on managed prompt playgrounds. This approach is necessary because the underlying structure of agents and their tools has fundamentally changed.
-
Prioritize Issue Context
14:00
When tracking issues, knowing only that an issue exists is insufficient. It is critical to track two metrics: when the issue started (to identify recent changes) and what percentage of users are affected.
-
Tactical Analysis Tips
17:05
1. **Clusters are not issues:** Clustering traces is useful for one-off analysis but does not scale well or reliably track changes over time. 2. **Code Mode Scaling:** Apply code mode concepts to traces by writing classifiers and running them in a sandbox at production volume. 3. **Anomaly Investigation:** Agents are poor at detecting anomalies, so instead, surface deterministic signals (like keyword spikes) for the agent to investigate.
Technical details
-
Evaluation Methodology
720s
Evals should be treated as code, resembling unit or end-to-end tests. Tools like Vitest evals and OpenAI's macro evals exemplify this shift.
-
Agent Failure Modes
480s
Agents can be powerful due to their creativity, but this also leads to catastrophic failures. The floor is defined by the worst possible outcome (e.g., deleting data or sending 'AI slop').
-
Issue Tracking Metrics
840s
For effective issue tracking, measure both the start date of an issue and the percentage of users affected. This helps determine if a problem is worth investigating.
Mentioned resources
- Raindrop
- Workshop
- howtoeval.com
Channel & topics
Watch on YouTube · Back to latest
This independent, AI-assisted summary is provided for commentary and informational purposes. It may contain errors or omit important context. Please watch the original video for the creator's complete presentation. Video, thumbnail, and related copyrights belong to their respective owners.