# How To Test AI Agents With Simulations

## Executive summary

This talk details the evolution of testing methodologies for complex AI agents, arguing that traditional input-output pair evaluations (evals) are insufficient. The modern approach requires 'simulations': replaying real historical production traces against proposed changes (e.g., new models, added tools) to detect subtle side effects. The speaker emphasizes that effective testing requires simulating the entire 'world' around the agent—including databases and infrastructure—and suggests that targeted simulations (5-50 traces) are far more valuable than running thousands of random evals.

## Key takeaways

- The limitations of traditional evals: Older evaluation methods relied on simple input-output validation (e.g., 'What is the capital of the US?'). This fails for agents because their behavior is now dictated by complex components like tools, memory, sub-agents, and external state, making the system far more intertwined with the real world.
- Simulations detect side effects in production agents: Simulations involve replaying real, historical production traffic against a proposed change. This allows teams to answer the critical question: 'What will my change change?' This is crucial for catching issues that simple unit tests or random evals would miss.
- Evals are shifting to pure functions over traces: Evals are increasingly defined as pure functions that take a trace (input) and return a score (output), rather than requiring a specific input-output pair. Metrics can include checking for consistency, verbosity, tool call count, or cost changes.
- Faithful simulation requires 'world building': To accurately simulate, one must recreate the infrastructure and state that existed when the original trace ran. Simply caching tool results is insufficient because the agent is 'simulation aware' and will detect the artificial environment.

## Technical details

- Simulation Mechanics: Simulations require replaying historical traces against a change (new model, new tool, etc.). The process involves 'world building'—recreating the infrastructure (e.g., databases, APIs) to ensure the agent believes it is in the real world. This is necessary because the state of the world changes over time.
- Production Monitoring (The Three Ingredients): Effective production monitoring requires three components: 1) The codebase and prompts (to understand expected behavior); 2) A baseline of behavior (to detect drift); and 3) Feedback from production data/traces (to understand user care points).
- Evals as Pure Functions: Modern evals are defined as functions that take a trace (input) and return a score (output). Examples include checking if the output length is within 20% of the original, or if the answer remains the same, allowing for scalable, deterministic checks.
- Simulation Awareness: Advanced agents are trained to detect when they are being evaluated or simulated. To prevent this, simulations must recreate the full environment, including deterministic elements like SQL dialects, to maintain the illusion of a real-world interaction.

## Practical implications

- Prioritize running targeted simulations (5-50 traces) over collecting massive numbers of random evals.
- Start testing by analyzing local traces to understand where the agent fails, before moving to full-scale testing.
- Build tight feedback loops by integrating simulations into the development cycle to minimize the slow iteration time of 'deploy $ ightarrow$ monitor $ ightarrow$ fix'.
- When designing tests, focus on metrics like consistency, cost, and step count, rather than just binary pass/fail checks.

## Topics

AI Agents, LLM Evaluation, Simulation Testing, Production Monitoring, Build Engineering, LLM Ops (LLMOps), Raindrop, How to eval.com

Source: https://www.youtube.com/watch?v=eLkESCLvvAs