The Data Context Layer for Agents — Yoni Michael & Brandon Callender, Typedef
Summary
This technical workshop details the necessity and implementation of a 'Context Layer' for empowering AI agents to operate reliably within complex data and code systems. The core argument is that model capability is not the bottleneck; rather, the lack of structured, queryable context leads to failure. For data systems, the layer computes lineage, grain, and join safety to ensure agents understand the true meaning of data, even when SQL compiles. For codebases, it builds a traversible graph of symbols and relationships, moving beyond ambiguous text search (`grep`) to provide deterministic, reliable context for code agents.
Key takeaways
-
The Context Gap: Beyond Syntax
10:29
Agents fail not because they can't write SQL or code, but because they lack semantic understanding. For data, valid SQL is insufficient if the agent doesn't know the data's 'grain' (what one row represents) or if an upstream change alters the metric's meaning. For code, text search is ambiguous; a graph is required to resolve symbol identities and dependencies.
-
The Four Facts of Data Context
15:02
To enable reliable data agents, the context layer must compute and expose four key facts: 1) The data grain (what a row represents); 2) Join safety (preventing unintended row multiplication); 3) Metric dependencies (understanding what relates to what); and 4) Model relationships (clustering assets into business domains).
-
Deterministic Context Engineering
20:00
The most reliable context is derived deterministically. The process involves parsing transformation logic (e.g., `dbt` project) down to the AST, verifying join safety by profiling tables, and using LLMs only for summarizing or resolving ambiguity, keeping the core facts tied to the implementation itself.
-
Evaluation is Paramount (Evals)
28:20
Building reliable agents requires rigorous evaluation. Benchmarks must measure determinism and consistency (Pass@K) across multiple runs, not just single-shot accuracy. The goal is to prove that the new context layer significantly improves the agent's ability to maintain correctness and consistency over time.
Technical details
-
Data Context Layer
1000s
This infrastructure is a graph that computes and refreshes atomically upon every data commit. It integrates information from the data warehouse, transformation code, and BI tools. It allows agents to perform 'pre-flight' impact analysis, predicting if a change (e.g., modifying a model's grain) will break downstream metrics.
-
Code Context Layer
2000s
The goal is to build a traversible graph of the codebase. The process involves: 1) Parsing code using tools like `Tree-sitter` (syntactic level); 2) Resolving symbols using `rustdoc` (identity level); and 3) Storing relationships in a graph database like `Neo4j`. This allows the agent to answer questions like 'What depends on this symbol?' in a single graph traversal, avoiding the ambiguity of text search.
-
Semantic Layering (Rungs)
2500s
Context is built in layers: Level 1 (raw strings/`grep`); Level 2 (semantic search/RAG); and Level 3 (traversable graph). The final layer adds semantic meaning (e.g., classifying a symbol as a 'measure' or 'dimension') using tools like `Fenic`, which combines data frame operations with LLMs, ensuring the semantic analysis remains auditable.
-
Agent Tooling and Prompting
3500s
The agent must be engineered to use the graph tools. The most critical step is designing the tool interface to mimic the model's existing 'muscle memory' (e.g., naming the tool 'search' and returning results in the format of `grep`) to overcome the model's natural tendency to default to file system probing.
Mentioned resources
- Neo4j
- Tree-sitter
- rustdoc
- Fenic
- Cypher
Channel & topics
Watch on YouTube · Back to latest
This independent, AI-assisted summary is provided for commentary and informational purposes. It may contain errors or omit important context. Please watch the original video for the creator's complete presentation. Video, thumbnail, and related copyrights belong to their respective owners.