# Vector Isn't Enough: Hybrid Search & Retrieval — Jeff Vestal & James Williams, Elastic

## Executive summary

This workshop details the architecture of advanced retrieval systems for AI agents, arguing that robust retrieval is paramount for accurate LLM outputs. It guides attendees through combining semantic (vector) search and traditional lexical (BM25) search into a 'hybrid search' approach. Key focus areas include optimizing the retrieval layer using techniques like reciprocal rank fusion (RF), linear combination, and advanced reranking, all within the Elasticsearch platform.

## Key takeaways

- Hybrid Search is the Core Thesis: The most effective retrieval systems combine the strengths of dense vector semantic search (good for paraphrasing/meaning) and lexical search (BM25, good for exact identifiers/keywords). Combining these methods is crucial for comprehensive search results.
- The Importance of Source Citation: For trustworthy AI agents, the ability to cite sources is critical. The system must not only provide an answer but also show the underlying documents, mitigating the risk of LLM hallucination.
- Reranking for Precision: Reranking is a second, optional phase that improves precision by combining the query and document during scoring. Listwise rerankers (like the Gina V3 model) are particularly useful when candidate documents are highly similar.
- Agentic Search Architecture: Modern agentic search systems should be designed to allow the LLM to perform multiple steps (e.g., diagnose a problem, then suggest a fix) by utilizing tools and skills, which can be programmed via APIs.

## Technical details

- Vector Search & Embeddings: Semantic search uses embedding models (e.g., GINA) to convert text into vectors. The search mechanism typically uses Hierarchical Navigable Small Worlds Graph (HNSW) for efficient nearest neighbor search. It is vital to use the same embedding model for both document ingestion and query time.
- Hybrid Search Combination Methods: Combining search scores can be done using Reciprocal Rank Fusion (RF) (out-of-the-box) or Linear Combination, which normalizes scores and allows developers to tune weights for different search components.
- Elasticsearch Query Language (ESQL): ESQL is a new, SQL-like query language within Elasticsearch designed for agentic search use cases. It allows piping commands together and is highly parameterizable, making it ideal for complex, multi-step queries.
- Advanced Search Capabilities: The platform supports advanced features like filtering (narrowing the candidate pool by keyword, geo-location, or date), and search templates, which allow routing different query types to specific search strategies.

## Practical implications

- Design the search layer to be hybrid, combining BM25 and vector search, rather than relying on a single method.
- Implement a multi-step agentic workflow that uses tools and skills, allowing the LLM to perform iterative searches and data analysis.
- Prioritize the ability to source-cite answers, ensuring the LLM's output is traceable back to the retrieved documents.
- Use ESQL for building complex, parameterized search queries that can handle multiple data sources and logical steps.

## Topics

Hybrid Search, Vector Search, BM25, Retrieval Augmented Generation (RAG), Agentic Systems, Elasticsearch, ESQL, Reranking, Workshop Environment (ELA), Source Code Repository, Elasticsearch Documentation

Source: https://www.youtube.com/watch?v=ksjwtQop4os