# Context Engineering in 2026 — Louis-François Bouchard, Omar Solano & Samridhi Vaid, Towards AI

## Executive summary

This talk provides a deep dive into context engineering for large language model (LLM) agents, focusing on optimizing performance and cost in real-world applications like an AI tutor. The core finding is that compaction techniques (like summarization) are often detrimental because they invalidate the prompt cache, forcing the system to pay full price for every token. For optimal recall and cost efficiency, retaining the full history—especially when leveraging advanced caching mechanisms—is superior to aggressive context reduction.

## Key takeaways

- Compaction is a potential trap due to Prompt Caching: When using prompt caching (which can make cached tokens up to 50 times cheaper), any transformation or summarization of the context invalidates the cache, forcing full-price token usage. Compaction must shrink the context by more than 50 times just to pay for itself.
- Full History Retention Wins on Recall: Experiments showed that leaving the full history untouched (the 'full history' preset) provided the best memory recall and overall performance, even though it was the most expensive option. Aggressive compaction techniques significantly degraded factual retrieval.
- Hybrid Search is Superior for Retrieval: For knowledge base browsing, a hybrid search combining semantic similarity (embedding model) with keyword search (BM25) is necessary to achieve high recall. Pure dense retrieval failed when facts were buried at large context sizes (e.g., 400k tokens), while BM25 maintained 100% accuracy.

## Technical details

- Context Management Components: An agent's context includes the system prompt, course content chunks, tool definitions, chat history, old tool outputs (tool calls/results pairs), and user questions. Optimizing this entire input stream is key.
- Context Window Limitations: The model's context window is finite. The primary challenges are managing context within a single session and maintaining memory across multiple sessions (statelessness).
- Compaction Techniques: Methods include selective retention (LLMs deciding what to keep), continuous summarization, sliding window trimming (using the last N turns), and progressive disclosure (loading only necessary skills/context).
- Retrieval Augmented Generation (RAG): The system uses a hybrid search pipeline: Semantic Search (e.g., using Coherent model) combined with BM25 for keyword indexing. The knowledge base is structured into 'raw' files, 'generated' titles, and a 'wiki' index.

## Practical implications

- Do not compact context by default; instead, name the specific constraint (e.g., token limit) you are facing.
- Prioritize optimizing for cache hits over aggressive summarization to minimize cost and maximize speed.
- Implement logging of all metrics (input/output tokens, cached tokens, time-to-first-token) to understand performance bottlenecks.
- For large data sets with interconnected topics, consider a Wiki or graph structure approach for memory offloading.

## Topics

Context Engineering, LLM Agents, Prompt Caching, Retrieval Augmented Generation (RAG), AI Architecture, Hugging Face Space, AI Tutor App Repository

Source: https://www.youtube.com/watch?v=WP3hjUXd918