# The Search Engine for the Agentic Web — Will Bryk, Exa

## Executive summary

The talk introduces Exa, a search engine designed specifically for AI agents, arguing that traditional search engines (like Google) are fundamentally flawed because they are built for human recommendation rather than precise information retrieval. The core premise is that as AI systems become ubiquitous, the volume of searches issued by machines will surpass human searches by a thousandfold by 2026. Exa addresses this by providing a high-quality, customizable search API that enables agents to perform complex, database-like queries over the world's information, moving beyond simple keyword matching.

## Key takeaways

- AI Search Volume Prediction: The speaker predicts that in 2026, the number of searches issued by AI systems will exceed those issued by humans. This volume is expected to increase by a thousandfold in the following years, necessitating a specialized search infrastructure.
- The Flaw of Recommendation Engines: Mainstream search engines are designed as recommendation engines, meaning they prioritize suggesting related content (e.g., 'shirts with stripes' when 'shirts without stripes' is queried) rather than providing exact, database-level answers. AI agents require the opposite: perfect retrieval.
- The Perfect Search Thought Experiment: The ideal search involves running a Language Model (LLM) over a complex query and a document pair to determine a match. While this is highly accurate, scaling it to a trillion documents costs millions per query, making cost-optimization the primary engineering challenge.
- The Business Catalyst: The company's business model was catalyzed when external users requested programmatic API access to their search engine, proving the need for an AI-focused search API, rather than just a consumer product.

## Technical details

- Search Architecture for Agents: Exa's search is designed for AI agents, serving diverse use cases like coding agents (e.g., Cursor) and go-to-market agents (e.g., HubSpot). It is built to handle complex, multi-faceted queries that require deep information synthesis.
- Search Optimization Techniques: The system moves beyond simple keyword search by utilizing neural networks and embeddings. To manage computational cost, the service offers super efficient token extraction, allowing LLMs to receive only the most critical tokens (e.g., 100 tokens) from large documents, significantly reducing downstream LLM costs.
- Structured Output and Flexibility: The API supports structured output, enabling agents to retrieve specific, formatted data points (e.g., 'Give me the college they went to and the year they graduated') rather than just raw text snippets. The platform is highly customizable, allowing clients to define search scope (e.g., specific domains, time windows).
- Data Integration and Market Creation: Exa is building a marketplace for data providers, allowing them to partner with the platform. This enables developers to access valuable, non-public data sources, combining public web information with proprietary data for comprehensive queries.

## Practical implications

- Build-Engineers developing AI agents must integrate specialized search APIs (like Exa) to ensure agents can perform deep, accurate information retrieval, moving beyond simple keyword searches.
- The need for optimized data pipelines that handle structured output and token extraction is critical for reducing operational costs in LLM-powered applications.
- Architecting agentic systems requires considering data sourcing beyond the public web, integrating proprietary data via marketplaces for comprehensive results.

## Topics

AI Agents, Information Retrieval, Search Engine Architecture, LLM Integration, API Development, Exa, Cursor, HubSpot, LLM (Large Language Model), Google

Source: https://www.youtube.com/watch?v=59AA5kIoqjA