If we want them to do Knowledge Work, design them as Knowledge Agents — Benjamin Clavié, Mixedbread
Summary
The talk distinguishes between 'Coding Agents' and 'Knowledge Agents,' arguing that most real-world tasks fall under the latter. While code provides durable cues (identifiers, file paths), knowledge work is inherently ambiguous, diffuse, and context-dependent (e.g., legal or medical research). The speaker posits that AI agents must be designed to mimic human knowledge work patterns—specifically, through advanced orchestration. This involves breaking down complex, open-ended problems, utilizing multiple specialized tools (primitives like BM25 and semantic search), and employing sub-agents (searchers) to synthesize findings into memos, thereby reducing the 'oracle gap' between perfect knowledge and the agent's output.
Key takeaways
-
Knowledge Work vs. Coding Work
Coding is a special, easy case because code has durable cues (identifiers, method definitions). Knowledge work, however, is defined by ambiguous information input and requires reconstructing intent and judgment, making it significantly harder for agents.
-
The Knowledge Loop
9:15
Human progress in knowledge work is driven by a single self-optimizing loop: better tools create new roles, new roles generate knowledge, and new knowledge demands better tools. This pattern should guide agent design.
-
The Necessity of Orchestration
Effective agent performance requires more than just a single tool. The most significant gains come from architectural improvements, such as having a main agent delegate tasks to specialized 'searcher agents' that return structured memos, which reduces the 'oracle gap' (the difference between perfect information and the system's output) by up to 40%.
-
Tooling is not Neutral
Tools are not merely incremental improvements; they are critical for overcoming performance ceilings. The ability to use a tool (e.g., a library catalog vs. physically searching archives) determines if a task is scalable and cheap enough to be practical.
Technical details
-
Knowledge Work Definition
250s
Knowledge work is defined as processing ambiguous information (input) to produce an actionable judgment or decision (output). It is also defined topologically: if a problem requires search, it is a knowledge problem.
-
Benchmark Performance (Browse Comp Plus)
On deep research tasks (200,000 documents), poorly optimized tools (e.g., unoptimized BM25) achieve low accuracy (~60%). Optimized hybrid harnesses can reach 98% accuracy while requiring 20% fewer tool calls, demonstrating the value of optimization and efficiency.
-
Multimodal Search and MQA
For enterprise tasks using PDFs (MQA benchmark), the ability to process structured data (tables) and not just rely on OCR text significantly boosts accuracy. The combination of advanced tools and organizational structure is necessary to reach peak performance.
-
Agent Architecture Improvement
The 'Mixbr agent' architecture improves performance by having a main agent break down the problem and delegate research to sub-agents. This structured approach significantly improves the system's ability to handle open-ended, complex queries.
Mentioned resources
- Browse Comp Plus
- MQA (Multi-Question Answering)
- Mixbread
- Gemini 3
Channel & topics
Watch on YouTube · Back to latest
This independent, AI-assisted summary is provided for commentary and informational purposes. It may contain errors or omit important context. Please watch the original video for the creator's complete presentation. Video, thumbnail, and related copyrights belong to their respective owners.