# Why Your Company Needs a Context Graph (and How to Build It) — Gil Feig, Merge

## Executive summary

The talk outlines the necessity of a 'context graph'—a comprehensive 'company brain'—for building production-grade AI agents. The core argument is that simple live API lookups (like those from MCP servers) fail when attempting semantic queries across vast, historical datasets (e.g., 'Which customers were upset last year?'). The solution requires a multi-layered architecture that combines live data lookups with synced, cached data stored in a vector database. This data flow is managed by a Router and a Summarizer, and crucially, must incorporate company-specific 'skills' and 'memories' to ensure the agent follows defined business processes. Traceability and provenance are mandatory for all data used in business-critical decisions.

## Key takeaways

- Context is more than tools; it's data and process.: A context graph must integrate third-party data (e.g., NetSuite, Jira), structured context, memories, and company-defined skills/processes, not just API connections.
- Live lookups are insufficient for semantic queries.: Live API lookups are effective for simple, single-action lookups (e.g., checking a specific customer's status in Stripe), but fail when the query requires searching across large datasets or complex semantic analysis.
- The architecture requires a Router and a Summarizer.: A Router directs the incoming prompt to the correct handler, while the Summarizer aggregates and synthesizes data retrieved from both synced and live context sources before sending the final answer to the agent.
- Traceability and Provenance are non-negotiable.: Every answer must track its data source, fetch time, associated ID, and what conditions (e.g., age, invalidation) might render the data outdated, especially for business-critical decisions.

## Technical details

- Context Graph Components: The graph integrates third-party data, structured context, memories (agent-formed knowledge), and skills (company processes).
- Data Tiers and Retrieval Strategy: Four tiers of context are used: 1) Live API Calls (simple, cheap, preferred for single lookups); 2) Cached Data (essential for semantic, complex lookups, requires a vector database); 3) Derived Context (summarizing pulled data for future use); and 4) Prompt/Skills/Memory (the guiding logic).
- Context Selection Flowchart: The process flow dictates checking for a Skill or Memory match first. If external data is needed, the system checks the cache for freshness/SLA compliance. If the cache fails, it falls back to a live API call. The final context is then summarized and returned.

## Practical implications

- When designing AI agents, do not rely solely on live API calls for historical or aggregate data; implement a robust data synchronization layer (caching) to support semantic querying.
- Model business processes (e.g., 'how to determine customer happiness') as explicit 'skills' within the context layer, ensuring the agent follows company-mandated logic.
- Build data pipelines that automatically generate 'derived context' (summaries of pulled data) to prevent redundant lookups and improve efficiency.
- Mandate metadata storage for all data sources to ensure full traceability, including the source, fetch time, and invalidation criteria.

## Topics

Context Graph, AI Agents, Vector Databases, Semantic Search, Data Provenance, LLM Architecture, Merge, Unified, Agent Handler, Gateway

Source: https://www.youtube.com/watch?v=cSz7aL2nl2U