Lessons from Studying Every Memory System — Shlok Khemani, Independent
The talk provides a deep dive into the evolution and architectural differences of memory systems in consumer AI applications (ChatGPT, Claude, Gemini). The core thesis is that 'memory' is not a standardized technology but rather a function of compute, requiring careful trade-offs between profile size, update frequency, and context window cost. Speakers highlight that while general architectures are converging toward running profiles, the specific implementation details remain unique to each product, meaning memory cannot be outsourced.
Key takeaways
-
Memory is not standardized
There is no single way to implement AI memory; products evolve independently (e.g., ChatGPT uses dense keywords/running profile; Claude uses full sentences/tools).
-
The Compute Trade-off
A running profile requires balancing two costs: the cost to maintain (update frequency/compute) and the serving cost (profile length in context window). This trade-off dictates product design.
-
Memory is a Product Problem
The biggest limitation of current AI memory systems is not technology, but product design. They often fail to reason over rich external sources like emails or calendars, leading to conflicts (e.g., conflicting travel dates).