The Search Engine for the Agentic Web — Will Bryk, Exa
Summary
The talk introduces Exa, a search engine designed specifically for AI agents, arguing that traditional search engines (like Google) are fundamentally flawed because they are built for human recommendation rather than precise information retrieval. The core premise is that as AI systems become ubiquitous, the volume of searches issued by machines will surpass human searches by a thousandfold by 2026. Exa addresses this by providing a high-quality, customizable search API that enables agents to perform complex, database-like queries over the world's information, moving beyond simple keyword matching.
Key takeaways
-
AI Search Volume Prediction
The speaker predicts that in 2026, the number of searches issued by AI systems will exceed those issued by humans. This volume is expected to increase by a thousandfold in the following years, necessitating a specialized search infrastructure.
-
The Flaw of Recommendation Engines
5:10
Mainstream search engines are designed as recommendation engines, meaning they prioritize suggesting related content (e.g., 'shirts with stripes' when 'shirts without stripes' is queried) rather than providing exact, database-level answers. AI agents require the opposite: perfect retrieval.
-
The Perfect Search Thought Experiment
10:05
The ideal search involves running a Language Model (LLM) over a complex query and a document pair to determine a match. While this is highly accurate, scaling it to a trillion documents costs millions per query, making cost-optimization the primary engineering challenge.
-
The Business Catalyst
13:45
The company's business model was catalyzed when external users requested programmatic API access to their search engine, proving the need for an AI-focused search API, rather than just a consumer product.
Technical details
-
Search Architecture for Agents
140s
Exa's search is designed for AI agents, serving diverse use cases like coding agents (e.g., Cursor) and go-to-market agents (e.g., HubSpot). It is built to handle complex, multi-faceted queries that require deep information synthesis.
-
Search Optimization Techniques
380s
The system moves beyond simple keyword search by utilizing neural networks and embeddings. To manage computational cost, the service offers super efficient token extraction, allowing LLMs to receive only the most critical tokens (e.g., 100 tokens) from large documents, significantly reducing downstream LLM costs.
-
Structured Output and Flexibility
470s
The API supports structured output, enabling agents to retrieve specific, formatted data points (e.g., 'Give me the college they went to and the year they graduated') rather than just raw text snippets. The platform is highly customizable, allowing clients to define search scope (e.g., specific domains, time windows).
-
Data Integration and Market Creation
680s
Exa is building a marketplace for data providers, allowing them to partner with the platform. This enables developers to access valuable, non-public data sources, combining public web information with proprietary data for comprehensive queries.
Mentioned resources
- Exa
- Cursor
- HubSpot
- LLM (Large Language Model)
Channel & topics
Watch on YouTube · Back to latest
This independent, AI-assisted summary is provided for commentary and informational purposes. It may contain errors or omit important context. Please watch the original video for the creator's complete presentation. Video, thumbnail, and related copyrights belong to their respective owners.