Demand-Driven Context: What 300 Incidents Reveal About Tribal Knowledge - Raj Navakoti
Summary
This talk introduces Demand-Driven Context (DDC), a methodology for curating enterprise knowledge by leveraging AI agents. Instead of attempting to document all institutional knowledge upfront, DDC treats real production incidents as knowledge tests. By letting an AI agent fail repeatedly on historical incidents, the system identifies precise knowledge gaps (vocabulary, business processes, system requirements) that are essential for resolution. The process is iterative, culminating in a 'living context' that continuously updates knowledge via a 'Propose Knowledge' skill, which requires a human gate (PR review) to prevent knowledge decay.
Key takeaways
-
Incident Management is a Knowledge Test
12:00
Incidents are highly valuable artifacts for knowledge discovery, as they surface 'tribal knowledge'—information that is critical but undocumented. This makes them ideal for testing and building a knowledge base.
-
DDC Methodology: Failure as a Signal
8:20
DDC is inspired by Test-Driven Development (TDD). The agent is given an incident and must identify exactly what knowledge is missing to resolve it, rather than just attempting to solve it with existing documentation.
-
Knowledge Curation is Finite and Targeted
18:00
The goal is not to dump all organizational knowledge (which is infinite in scope), but to identify the finite, minimum context required for a specific agent (e.g., an Incident Resolution agent) to succeed.
-
Maintaining Knowledge Requires a Human Gate
18:20
To prevent knowledge decay (knowledge rot), a 'Propose Knowledge' skill is implemented. When the agent learns something new or finds a conflict, it generates a Pull Request (PR) on the knowledge base, which must be reviewed and merged by a human.
Technical details
-
Demand-Driven Context (DDC)
500s
A methodology that uses AI agents to fail on real-world problems (incidents) to systematically identify and curate the minimum necessary knowledge context, mimicking the principle of TDD.
-
AI Agent Capabilities and Limitations
300s
Initial attempts using frontier models (GPT-5, Opus 4.5) with standard RAG pipelines failed because the incident scope was too broad (a whole domain, not just one team). The agent needs to be prompted to ask for specific missing information (e.g., 'What is a nightly batch retry policy?').
-
Knowledge Gap Identification
600s
The agent is prompted to identify knowledge gaps across specific entity types: vocabulary, business processes, system requirements, or API schemas. This moves the agent from merely summarizing to actively diagnosing knowledge deficits.
-
Knowledge Base Maintenance Loop
1100s
The 'Propose Knowledge' skill allows the agent to act as a continuous learning mechanism. When it learns a new fact or detects a conflict, it generates a PR, ensuring that knowledge updates are subject to human review and merge.
Mentioned resources
- NDC Conferences
- Raj Nakoti's Website/LinkedIn
- GitHub Toolkit/Paper
Channel & topics
Watch on YouTube · Back to latest
This independent, AI-assisted summary is provided for commentary and informational purposes. It may contain errors or omit important context. Please watch the original video for the creator's complete presentation. Video, thumbnail, and related copyrights belong to their respective owners.