Context Engineering in 2026 — Louis-François Bouchard, Omar Solano & Samridhi Vaid, Towards AI
This talk provides a deep dive into context engineering for large language model (LLM) agents, focusing on optimizing performance and cost in real-world applications like an AI tutor. The core finding is that compaction techniques (like summarization) are often detrimental because they invalidate the prompt cache, forcing the system to pay full price for every token. For optimal recall and cost efficiency, retaining the full history—especially when leveraging advanced caching mechanisms—is superior to aggressive context reduction.
Key takeaways
-
Compaction is a potential trap due to Prompt Caching
22:06
When using prompt caching (which can make cached tokens up to 50 times cheaper), any transformation or summarization of the context invalidates the cache, forcing full-price token usage. Compaction must shrink the context by more than 50 times just to pay for itself.
-
Full History Retention Wins on Recall
33:33
Experiments showed that leaving the full history untouched (the 'full history' preset) provided the best memory recall and overall performance, even though it was the most expensive option. Aggressive compaction techniques significantly degraded factual retrieval.
-
Hybrid Search is Superior for Retrieval
22:48
For knowledge base browsing, a hybrid search combining semantic similarity (embedding model) with keyword search (BM25) is necessary to achieve high recall. Pure dense retrieval failed when facts were buried at large context sizes (e.g., 400k tokens), while BM25 maintained 100% accuracy.