How To Test AI Agents With Simulations
This talk details the evolution of testing methodologies for complex AI agents, arguing that traditional input-output pair evaluations (evals) are insufficient. The modern approach requires 'simulations': replaying real historical production traces against proposed changes (e.g., new models, added tools) to detect subtle side effects. The speaker emphasizes that effective testing requires simulating the entire 'world' around the agent—including databases and infrastructure—and suggests that targeted simulations (5-50 traces) are far more valuable than running thousands of random evals.
Key takeaways
-
The limitations of traditional evals
Older evaluation methods relied on simple input-output validation (e.g., 'What is the capital of the US?'). This fails for agents because their behavior is now dictated by complex components like tools, memory, sub-agents, and external state, making the system far more intertwined with the real world.
-
Simulations detect side effects in production agents
0:10
Simulations involve replaying real, historical production traffic against a proposed change. This allows teams to answer the critical question: 'What will my change change?' This is crucial for catching issues that simple unit tests or random evals would miss.
-
Evals are shifting to pure functions over traces
0:22
Evals are increasingly defined as pure functions that take a trace (input) and return a score (output), rather than requiring a specific input-output pair. Metrics can include checking for consistency, verbosity, tool call count, or cost changes.
-
Faithful simulation requires 'world building'
0:13
To accurately simulate, one must recreate the infrastructure and state that existed when the original trace ran. Simply caching tool results is insufficient because the agent is 'simulation aware' and will detect the artificial environment.