Advanced RAG: repair retrieval one failure at a time

Use hybrid search when wording is not the whole story

Lexical retrieval is useful for exact terms, identifiers, and rare names. Dense retrieval helps with paraphrases. A hybrid system retrieves from both and combines candidates. Because their raw scores have different meanings, adding them directly can produce arbitrary rankings.

Rank fusion is one way to combine ordered lists without pretending their score scales match. Keep the original source IDs, remove duplicates, and measure whether the merged set actually contains more relevant evidence. Northstar's product-code queries are a useful test case: semantic similarity alone may favor the general product family and miss the exact code.

Rerank a candidate set before spending context

A reranker examines the query and candidate passage together to estimate relevance. Retrieve a broader set, then rerank and select a smaller evidence set. This adds work, so compare the final answer improvement as well as retrieval metrics and response time.

Do not hide a poor first-stage retriever behind a reranker. A passage absent from the candidate pool cannot be promoted. Inspect candidate recall first, then ranking precision. Preserve authorization filters across every retrieval path; a second search service should not quietly broaden the user's access.

Rewrite or decompose without changing the question

A short follow-up such as “What about damaged ones?” may need the previous subject to become a useful search query. Decomposition can split a compound question into damage reporting and delivery timing, retrieve each, and combine the supported findings. Preserve the original question for answer generation and review.

HyDE generates a hypothetical document and embeds it as a retrieval aid. That generated text is a search instrument, not evidence. It must not become the source for a factual answer. Query expansion has a similar risk: a helpful-looking rewrite may introduce a condition the user never supplied.

Handle tables, images, and context compression carefully

Multimodal retrieval may require representations suited to images, text, or both. A chart needs labels and units; a table needs headers; a scanned page may need OCR with quality checks. Store a traceable source region so a reader can inspect the original evidence.

Compression can reduce context cost, but it may remove the exact exception that makes the answer correct. Compare compressed and original passages on exception questions. Agentic retrieval adds iterative search and tools; give it limits and test whether the extra steps improve outcomes. More retrieval stages are justified by measured failure reduction, not by a more impressive architecture diagram.

References and further reading

Use these primary references for deeper study and current API details. Examples in this lesson use fictional Northstar data.

• Research paper — 2212.10496
• pgvector documentation — pgvector