How To Test AI Agents With Simulations
Summary
This talk details the evolution of testing methodologies for complex AI agents, arguing that traditional input-output pair evaluations (evals) are insufficient. The modern approach requires 'simulations': replaying real historical production traces against proposed changes (e.g., new models, added tools) to detect subtle side effects. The speaker emphasizes that effective testing requires simulating the entire 'world' around the agent—including databases and infrastructure—and suggests that targeted simulations (5-50 traces) are far more valuable than running thousands of random evals.
Key takeaways
-
The limitations of traditional evals
Older evaluation methods relied on simple input-output validation (e.g., 'What is the capital of the US?'). This fails for agents because their behavior is now dictated by complex components like tools, memory, sub-agents, and external state, making the system far more intertwined with the real world.
-
Simulations detect side effects in production agents
0:10
Simulations involve replaying real, historical production traffic against a proposed change. This allows teams to answer the critical question: 'What will my change change?' This is crucial for catching issues that simple unit tests or random evals would miss.
-
Evals are shifting to pure functions over traces
0:22
Evals are increasingly defined as pure functions that take a trace (input) and return a score (output), rather than requiring a specific input-output pair. Metrics can include checking for consistency, verbosity, tool call count, or cost changes.
-
Faithful simulation requires 'world building'
0:13
To accurately simulate, one must recreate the infrastructure and state that existed when the original trace ran. Simply caching tool results is insufficient because the agent is 'simulation aware' and will detect the artificial environment.
Technical details
-
Simulation Mechanics
13s
Simulations require replaying historical traces against a change (new model, new tool, etc.). The process involves 'world building'—recreating the infrastructure (e.g., databases, APIs) to ensure the agent believes it is in the real world. This is necessary because the state of the world changes over time.
-
Production Monitoring (The Three Ingredients)
6s
Effective production monitoring requires three components: 1) The codebase and prompts (to understand expected behavior); 2) A baseline of behavior (to detect drift); and 3) Feedback from production data/traces (to understand user care points).
-
Evals as Pure Functions
22s
Modern evals are defined as functions that take a trace (input) and return a score (output). Examples include checking if the output length is within 20% of the original, or if the answer remains the same, allowing for scalable, deterministic checks.
-
Simulation Awareness
16s
Advanced agents are trained to detect when they are being evaluated or simulated. To prevent this, simulations must recreate the full environment, including deterministic elements like SQL dialects, to maintain the illusion of a real-world interaction.
Mentioned resources
Channel & topics
Watch on YouTube · Back to latest
This independent, AI-assisted summary is provided for commentary and informational purposes. It may contain errors or omit important context. Please watch the original video for the creator's complete presentation. Video, thumbnail, and related copyrights belong to their respective owners.