Long-Horizon Agents Need Experiments, Not Just Prompts — Erina Karati
Summary
This talk outlines the challenges of building multi-agent AI systems that maintain state and social consistency over long time horizons. The speaker introduces Project Paradox, a modular, stateful framework for game agents. The core problem—the degradation of social consistency (e.g., losing the source of a rumor)—is addressed by implementing an 'Auto Research' layer. This layer functions as an experimental loop, running controlled scenarios, collecting structured traces, and proposing constrained policy changes only if a balanced scorecard improves, thereby moving agent improvement from simple prompt tuning to systematic, verifiable engineering.
Key takeaways
-
Long-Horizon Agents Need Experiments, Not Just Prompts
20:33
Improving multi-agent behavior requires running controlled scenarios and collecting structured traces, rather than relying solely on prompt engineering or single-run demos. The goal is to evaluate entire runs, not just single answers.
-
Provenance and Source Attribution are Critical
Simple Retrieval-Augmented Generation (RAG) memory is insufficient. Agents must track where information came from (firsthand, secondhand, verified, uncertain) to maintain social consistency.
-
Optimization Requires a Balanced Scorecard
Instead of optimizing for a single metric (e.g., 'agent quality'), a balanced scorecard must measure multiple dimensions like reach, source retention, uncertainty preservation, action consistency, and privacy containment.
-
The Auto Research Loop
13:33
The proposed solution is a meta-system that defines a controlled scenario, runs the agents, scores the behavior, and proposes small, constrained changes to the agent protocol (the 'editable surface').
Technical details
-
Project Paradox Architecture
253s
The framework is intentionally stateful, featuring per-agent memory (backed by RAG), emotion tracking (small vector), a trust matrix (belief scores), and an importance score for memories to prioritize critical context.
-
Auto Research Layer
923s
This layer operates outside the agent village, reading full traces of a run. It compares outcomes against scenario ground truth, scores the behavior, and proposes constrained changes to the agent protocol or cognitive policy. It ensures changes only improve the score and maintain guardrails.
-
Controlled Scenarios
1143s
To evaluate social behavior, scenarios must be controlled (e.g., testing public fact diffusion, rumor uncertainty, or replanning due to blocked routes). This allows for objective measurement of system improvement.
-
Policy Surface Control
The auto research layer must only edit a small, frozen policy surface (e.g., memory writing policy, communication prompt, belief rules) to prevent 'gaming' the evaluation and ensure controlled improvement.
Mentioned resources
- Project Paradox
- Supercell AI Innovation Lab
- Microsoft
Channel & topics
Watch on YouTube · Back to latest
This independent, AI-assisted summary is provided for commentary and informational purposes. It may contain errors or omit important context. Please watch the original video for the creator's complete presentation. Video, thumbnail, and related copyrights belong to their respective owners.