Your Coding Agent Is 6 Months Out of Date — Jakub Hojsan, Exa
Summary
Jakub Hojsan of Exa details how they built specialized search capabilities for coding and code-review agents, addressing the critical issue of Large Language Model (LLM) knowledge cutoffs. Exa's solution provides context-rich, token-efficient highlights by distilling massive documents (e.g., 100,000 characters) down to only the necessary snippets (e.g., 500 characters) for the LLM. The talk emphasizes that simply adding a web search tool is insufficient; agents require explicit rules for when and how to search. Exa's platform offers transparency (full search trace), cost efficiency, and model-provider independence, allowing integration across various LLMs and tools via the Exa MCP.
Key takeaways
-
The Knowledge-Cutoff Gap in Code Review
LLMs suffer from a knowledge cutoff, meaning they cannot review changes (like PRs) made after their training date. This gap is critical in code review, where reviewing recent changes requires access to current repository information.
-
Context-Rich, Token-Efficient Highlights
0:02
Instead of returning entire web pages, Exa's search engine uses an interpretation step to distill large documents into small, highly relevant snippets (e.g., 500 characters) for the LLM, significantly improving context quality and efficiency.
-
Agents Need Explicit Search Rules
0:04
Bolting on a web search tool is not enough. Agents must be provided with explicit rules (e.g., 'When a diff bumps a dependency, look at the upstream source') to guide when and how to use the search tool.
-
Exa's Technical Advantages
0:04
Exa provides transparency by giving the full search trace (exact queries, sources, and highlights), offering cost-effective pricing compared to native web search APIs, and ensuring model-provider independence.
Technical details
-
Search Architecture
0s
Exa uses a semantic search engine that provides context-rich highlights rather than traditional 'blue links.' The search process is described as computational, pulling specific lines at runtime without relying on an LLM for the core retrieval, thus adding zero extra latency.
-
Exa MCP
4s
The Exa Model Control Plane (MCP) allows for standardization, enabling a fixed API and set of parameters that can be used across various model providers (OpenAI, Anthropic, GLM), ensuring model-provider independence.
-
Query-Dependent Highlights
6s
This feature distills an entire page down to only the information needed to answer a specific query, allowing the model to focus on highly targeted context.
-
Reranking and Indexing
10s
The search process is a multi-stage process involving query embedding, keyword filtering, and semantic search, culminating in a reranking step to ensure the highest quality, most relevant information is passed to the model.
Mentioned resources
Channel & topics
Watch on YouTube · Back to latest
This independent, AI-assisted summary is provided for commentary and informational purposes. It may contain errors or omit important context. Please watch the original video for the creator's complete presentation. Video, thumbnail, and related copyrights belong to their respective owners.