Memory Harnesses for Long-Running Research Agents — Stefania Druga, Sakana.ai
The presentation details the design and efficacy of 'memory harnesses' for managing state in long-running research agents. The core finding is that while memory harnesses offer no benefit when task context fits within the model's window (adding only cost), they are critical for solving long-horizon tasks where relevant information sits far outside the current context. The speaker emphasizes treating memory as a write-manage-read control loop, not merely an attached database.
Key takeaways
-
Memory is a Control Loop, Not a Database
5:44
The memory harness must be viewed as a 'write-manage-read' control loop wrapped around the model, rather than simply a retrieval database. This architectural focus allows for precise state management.
-
Local Models Enable Sovereignty and Cost Control
2:36
Running evaluations on local hardware (e.g., M3 Ultra) enables better control over the entire pipeline, which is crucial for maintaining 'sovereign AI' capabilities and reducing operational costs.
-
Ranked Recall Outperforms Other Policies
7:16
Across the X-Bench benchmark (over 68 questions), the 'ranked recall' policy consistently achieved the best performance, outperforming even approaches that simply gate memory usage or use an ideal 'oracle' ground truth.