Beyond RAG: A Relational Context Engine That Cuts Token Burn — Unblocked
Summary
The talk introduces the concept of a Context Engine, arguing that modern AI agents require more than simple Retrieval-Augmented Generation (RAG) or vector search. A Context Engine gathers and structures 'institutional knowledge'—including PRs, Slack threads, and incident history—to provide 'grounded context.' This approach allows agents to answer complex, structural, and temporal questions (e.g., 'Which PRs did I merge last week?') and significantly reduces operational costs by providing highly focused context bundles.
Key takeaways
-
Context Engines vs. Vector Search
11:51
Vector search excels at semantic content retrieval (finding related documents), but it fails at structural questions that require querying relationships, temporal data, or specific system metadata. A Context Engine addresses this by synthesizing data from disparate sources (PRs, Notion, Slack).
-
Cost and Accuracy Improvement
6:00
Demonstrations showed that attaching a context engine resulted in a 50% reduction in API costs and enabled complex debugging, such as tracing a regression to a specific model switch (Opus 48 vs Opus 46).
-
The Structured Query Pipeline
17:11
The core process involves turning natural language queries into safe, validated database queries. This requires five key steps: Automatic Schema Discovery, Identity Resolution, Query Validation, Retries, and Full-Text Search.
Technical details
-
Automatic Schema Discovery
1031s
The engine dynamically discovers the database schema at runtime by sampling documents, allowing the use of LLMs or traditional procedural methods to infer structures like enums.
-
Identity Resolution
This process maps user identities across different systems (e.g., GitHub usernames) to ensure that queries correctly attribute actions to the intended individual, even if the identifiers differ.
-
Query Validation and Safety
Validation is critical because LLM-generated queries can be dangerous (e.g., containing raw JavaScript function operators). The engine validates queries and can constrain the LLM from interacting with sensitive metadata fields (like tenant IDs).
-
Error Handling and Retries
The system implements a self-correction loop: if a query fails against the database, the error is fed back to the LLM, allowing it to attempt to self-correct the query structure.
-
Full-Text Search Integration
Full-text search is added as a final layer to ensure comprehensive querying, complementing the structured, relational capabilities of the engine.
Mentioned resources
- Unblocked
- Document Query Engine (GitHub)
- Context Engine Simulator
Channel & topics
Watch on YouTube · Back to latest
This independent, AI-assisted summary is provided for commentary and informational purposes. It may contain errors or omit important context. Please watch the original video for the creator's complete presentation. Video, thumbnail, and related copyrights belong to their respective owners.