How We Built an Agent That Improves Itself — Zubin Aysola, Weights & Biases
The presentation details the architecture of ARIA, a self-improving AI agent developed by Weights & Biases. The core innovation is a robust, closed-loop evaluation framework that allows the agent to perform 'auto-research' by taking production traces, converting them into offline evaluation tasks, identifying bugs (e.g., missing SDK calls), writing fixes, and benchmarking the improved variant against the production version. This system ensures that both production and research environments are byte-for-byte identical, creating a continuous 'eval flywheel' for agent improvement.
Key takeaways
-
The Covariance Challenge in Agent Evaluation
2:11
A major challenge is that benchmarks, evaluations, and agent configurations are all covariant. To maintain airtight measurement, the team keeps the production and research agents byte-for-byte identical (5:50).
-
The Eval Flywheel
The system creates a continuous improvement loop: every production miss or win is captured as a new task, which is then used to train and improve the agent, minimizing manual benchmark creation (12:03).
-
Offline Evaluation Framework
The framework uses a dedicated offline evaluation sandbox to simulate environments and run agent variants. This process involves converting production traces into structured evaluation tasks (3:40).
-
Dual Scoring Mechanism
16:54
ARIA scores itself in two ways: pass/fail (normative) and relative comparison, allowing comparison between different agent variants (10:14).