Total Recall: Agent Memory and Harness Engineering — Ignacio Martinez, Oracle
The presentation details the architecture of an AI agent, arguing that while Large Language Models (LLMs) provide the non-deterministic 'reasoning core,' the true engineering effort lies in the 'agent harness.' This harness is a multi-layered system designed to transform the LLM's unpredictable output into reliable, repeatable, and autonomous workflows. Key architectural components include advanced memory management (short-term, long-term, shared), semantic layers for institutional knowledge capture, and robust data storage solutions that combine the flexibility of files with the transactional consistency of databases.
Key takeaways
-
The Agent Architecture Shift
17:30
An AI agent is defined as a Large Language Model (the reasoning core, which is rented and uncontrolled) augmented by a harness. The goal of harness engineering is to ensure reliable and predictable output, compensating for the model's inherent non-determinism.
-
Data Layer Complexity and Solutions
27:06
The data layer is critical, requiring a hybrid approach to storage. While files are simple and model-friendly, they lack transactional consistency. Databases provide ACID properties, high availability, and vector search. The optimal solution is a hybrid system, such as Oracle DBFS, which combines the benefits of both.
-
Memory and Context Management
33:20
Agent memory is categorized into short-term (ephemeral), long-term (episodic), and shared memory. To prevent 'context rot' and maintain relevance, the harness must implement sophisticated context engineering techniques, such as context compaction and summarization.
-
The Agent Workflow Loop
50:00
The core operational structure is the agent loop (Observe $ ightarrow$ Reason $ ightarrow$ Act). This loop must be failure-resistant and can be enhanced through techniques like the Toolbox pattern and model routing (using small experts for specific tasks) to improve efficiency and reliability.
