# Lessons from Studying Every Memory System — Shlok Khemani, Independent

## Executive summary

The talk provides a deep dive into the evolution and architectural differences of memory systems in consumer AI applications (ChatGPT, Claude, Gemini). The core thesis is that 'memory' is not a standardized technology but rather a function of compute, requiring careful trade-offs between profile size, update frequency, and context window cost. Speakers highlight that while general architectures are converging toward running profiles, the specific implementation details remain unique to each product, meaning memory cannot be outsourced.

## Key takeaways

- Memory is not standardized: There is no single way to implement AI memory; products evolve independently (e.g., ChatGPT uses dense keywords/running profile; Claude uses full sentences/tools).
- The Compute Trade-off: A running profile requires balancing two costs: the cost to maintain (update frequency/compute) and the serving cost (profile length in context window). This trade-off dictates product design.
- Memory is a Product Problem: The biggest limitation of current AI memory systems is not technology, but product design. They often fail to reason over rich external sources like emails or calendars, leading to conflicts (e.g., conflicting travel dates).

## Technical details

- ChatGPT Memory Evolution: V1 (Feb 2024) was a user-managed list of facts, suffering from the burden of manual memory management and staleness. V2 (Apr 2025) introduced 'user knowledge memories'—a running profile updated asynchronously every few days, containing dense keyword clues (up to ~4,000 tokens).
- Claude Memory Architecture: V1 lacked a user profile, relying instead on two tools: searching previous conversations by keyword or time period. V2 added a running profile that is visible to the user, smaller (~1,000 tokens), uses complete sentences, and updates every 24 hours.
- The Running Profile Concept: A running profile synthesizes new conversation data into a persistent summary added to the context window. This process constitutes a continual learning loop that is currently happening outside of model weights.
- Memory System Flaws (Staleness/Conflict): Even advanced systems can fail; ChatGPT's V2 profile showed overlapping dates for travel (Thailand and Turkey), demonstrating that the system cannot resolve conflicts or reason over external evidence (like flight bookings).

## Practical implications

- Memory systems must be built in-house and evolve alongside the product; they cannot be outsourced.
- Engineers must treat memory as a foundational component, not an afterthought, integrating it with other data sources (email, calendar).
- The design of any memory system requires explicit consideration of compute costs: balancing update frequency against context window serving size.

## Topics

AI Architecture, LLM Memory Systems, Product Design, Continual Learning, Context Management, Guardian Angels

Source: https://www.youtube.com/watch?v=5ZGyKWjQDr0