← All artifacts · All patterns · Retrieval & RAG

Hybrid Search

aka BM25 + Dense, Lexical + Semantic Retrieval

category: retrieval · status: mature

Combine sparse lexical retrieval (BM25) with dense vector retrieval and fuse the results. A coding-assistant searches its codebase for 'how do we authenticate with Stripe?' Pure semantic search misses files that mention 'stripe-api-key' verbatim; pure keyword search misses files that talk about 'payment processor authentication'. Hybrid search runs both at once: a keyword scorer catches the exact tokens, an embedding scorer catches the conceptual matches, and a fusion step blends the two ranked lists.

2. Problem

A team is running a retrieval pipeline over a corpus where the user queries fall into two very different shapes. Some queries are short and exact, hinging on matching specific identifiers, product codes, person names, or technical terms verbatim. Other queries are longer and rely on semantic similarity between paraphrased ideas, where the surface vocabulary may differ between query and source. A single retrieval method serves only one of these well.

Dense vector retrieval handles paraphrase and semantic similarity but misses queries that hinge on an exact identifier the embedding has flattened away. Sparse keyword retrieval — BM25 and similar lexical methods — handles exact terms but misses paraphrased queries whose vocabulary does not overlap with the source text. Picking either method alone means leaving recall on the table for whichever query shape was not chosen, and no downstream re-ranking stage can rescue a chunk that was never retrieved in the first place.

Competing forces:

• Score fusion (RRF, weighted sum, learned) is a design choice.
• Two indexes mean two pipelines to maintain.
• Tuning fusion weights is empirical and corpus-specific.

3. When to Use

• Queries mix semantic intent with rare tokens (codes, IDs, proper nouns) that embeddings miss.
• The corpus is heterogeneous enough that one retriever loses recall on part of it.
• Latency budget tolerates two retrievers plus a fusion step.

4. When Not to Use

• The corpus is uniformly conceptual; dense alone is enough.
• The corpus is uniformly keyword-driven; BM25 alone is enough.
• Sub-50ms p95 retrieval is required and the second retriever blows the budget.

5. Architecture Diagram

flowchart TD
  Q[Query] --> S[Sparse retrieval BM25]
  Q --> D[Dense retrieval embeddings]
  S --> F[Fusion / Reciprocal Rank Fusion]
  D --> F
  F --> R[Top-k results]

Hybrid Search runs sparse and dense retrievers in parallel and fuses their rankings before downstream stages.

6. Components

• Sparse retriever — BM25 or similar lexical index returns a ranked list on exact-term matches
• Dense retriever — bi-encoder plus vector index returns a ranked list on semantic similarity
• Fusion stage — Reciprocal Rank Fusion or score-normalised aggregation combines the two rankings
• Top-N selector — emits the fused top-N for downstream reranking or generation

7. Tools

• BM25 engine — Elasticsearch, OpenSearch, or Tantivy-class lexical index
• Vector index — Pinecone, Weaviate, or pgvector-class dense retrieval backend
• Embedding model — produces query and document vectors for the dense leg

8. Guardrails

Therefore: index the corpus both lexically and semantically and fuse the rankings, so that exact-match recall and semantic recall both contribute to the final top-N.

9. Failure Modes

• Raw score mixing — BM25 and cosine scores are added directly, one scale swamps the other, and the fusion acts like a single retriever.
• Index skew — a document is updated in the dense index but not the BM25 index, and the two lists rank different versions of the same content.
• Mis-tuned fusion — weights tuned on one corpus favour the wrong side on another, and exact identifier queries lose their match.
• Shallow per-side k — each retriever returns too few items, so a chunk ranked moderately by both never reaches the fused list.
• Two indexes to keep in sync.
• Fusion tuning is empirical.

10. Evaluation Metrics

• Recall@k against dense-only and sparse-only baselines — quantifies the fusion lift
• Exact-token query recall — does fusion recover the queries dense alone misses (codes, IDs, proper nouns)
• Paraphrase-query recall — does fusion preserve the queries BM25 alone misses
• Fusion-weight sensitivity — how much quality depends on the tuned weights or the RRF constant k
• Index-sync drift — how often the two indexes fall out of sync on writes

11. Code Examples

function hybrid_search(query, k=10):
    bm25_hits  = bm25_index.search(query, k=k)
    dense_hits = vector_index.search(embed(query), k=k)

    # Reciprocal Rank Fusion — rank-based, scale-free
    scores = {}
    C = 60                                                # standard RRF constant
    for rank, doc in enumerate(bm25_hits):
        scores[doc.id] = scores.get(doc.id, 0) + 1 / (C + rank)
    for rank, doc in enumerate(dense_hits):
        scores[doc.id] = scores.get(doc.id, 0) + 1 / (C + rank)

    fused = sort_desc(scores.items(), by=value)
    return [docs[doc_id] for doc_id, _ in fused[:k]]

from llama_index.core import VectorStoreIndex
from llama_index.core.retrievers import QueryFusionRetriever
from llama_index.retrievers.bm25 import BM25Retriever

index = VectorStoreIndex.from_documents(documents)

vector_retriever = index.as_retriever(similarity_top_k=5)
bm25_retriever = BM25Retriever.from_defaults(
    docstore=index.docstore,
    similarity_top_k=10,
)

retriever = QueryFusionRetriever(
    [vector_retriever, bm25_retriever],
    retriever_weights=[0.6, 0.4],
    similarity_top_k=10,
    mode="relative_score",
    use_async=True,
)

nodes = retriever.retrieve("What is task decomposition?")

from haystack_integrations.document_stores.weaviate.document_store import (
    WeaviateDocumentStore,
)
from haystack_integrations.components.retrievers.weaviate import (
    WeaviateHybridRetriever,
)

document_store = WeaviateDocumentStore(url="http://localhost:8080")
retriever = WeaviateHybridRetriever(
    document_store=document_store,
    top_k=10,
    alpha=0.5,                                  # 0 = pure BM25, 1 = pure vector
)

result = retriever.run(
    query="What is task decomposition?",
    query_embedding=[0.0] * 768,                # supply your real embedding here
)
for d in result["documents"]:
    print(d.content[:200])