From Context to Memory: Your Agents Need a Real Memory Layer — Anders Swanson, Oracle
Summary
Anders Swanson argues that treating agent memory as a systems problem is necessary because current methods (like context windows or scratch files) fail to provide persistent, scalable recall. He outlines a modern memory architecture that involves four stages: formation, recall, evolution, and evaluation. This system requires a multi-model database to store typed memories (episodic, semantic, procedural, working state) and utilizes hybrid search, governance controls (like redaction), and continuous feedback loops to ensure data integrity and utility.
Key takeaways
-
Context Windows are not Memory
0:57
Context windows are merely input for the current run, not a persistent experience. Useful work synthesized by an agent is lost at the end of a session without persistence, forcing subsequent agents to repeat work and increasing the risk of hallucinations and failure. (0:57)
-
The Need for Structured Memory
2:07
Effective memory requires transforming useful agent experiences into managed, structured state. Simple workarounds like markdown files or tools like grep do not scale in large, multi-tenant environments, necessitating a database solution. (2:07)
-
The Memory Lifecycle Pattern
5:07
Modern memory systems converge on a pattern involving memory formation, recall, evolution, and evaluation. This process requires governance controls, such as redacting PII before writing, and continuous feedback to prevent memories from becoming stale, poisoned, or conflicting. (3:07, 4:37)
Technical details
-
Typed Memories
382s
Memory should be categorized into four types: Episodic (specific past events/transcripts), Semantic (facts about workflows/preferences), Procedural (reusable rules, like a build step), and Working State (short-term, temporary data). (6:22)
-
Storage Schema Requirements
437s
A robust memory object must include: a primary ID, text content, JSON tags, a vector embedding (for similarity search), a text index (for keyword search), graph links (for related memories), temporal facts (validity window, last used), and audit events (creation/usage log). (7:17)
-
Memory Formation and Governance
497s
Memory formation must be intentional and starts with **redaction** to strip out PII and secrets. This is followed by enrichment (metadata, scope) and embedding, ensuring the memory is properly scoped to the agent, team, or workflow. (8:17)
-
Hybrid Search for Recall
602s
Recall is achieved through hybrid search, which fuses multiple retrieval methods: vector similarity search, full-text keyword search, relational entity search, graph traversals, and temporal/feedback signals. Results are fused, re-ranked, and scored to provide a comprehensive context card. (10:02)
Mentioned resources
Channel & topics
Watch on YouTube · Back to latest
This independent, AI-assisted summary is provided for commentary and informational purposes. It may contain errors or omit important context. Please watch the original video for the creator's complete presentation. Video, thumbnail, and related copyrights belong to their respective owners.