Why RAG Fails at Retrieval, Not at the Model
Most disappointing RAG demos are blamed on the LLM. In practice the model is rarely the problem — the passages it was given are. Here is where retrieval breaks down and how to fix it.
Why RAG Fails at Retrieval, Not at the Model
When a retrieval-augmented generation (RAG) system gives a wrong answer, the instinct is to swap in a bigger model. But a language model can only work with the context it is handed. If the right passage never reaches the prompt, no model can recover it.
Where retrieval usually breaks
- Documents are read badly. Tables flattened into text, scans skipped, headings lost.
- Chunks cut meaning in half. A clause separated from its sub-clauses, a table row separated from its header.
- Search is meaning-only. Dense vectors miss exact identifiers, part numbers and reference codes that keyword search finds instantly.
- Nothing checks permissions. The best passage may belong to someone the user is not allowed to see.
- No one measures. Without test questions, regressions go unnoticed.
What good retrieval looks like
Strong pipelines read documents with structure intact, chunk on natural boundaries, run keyword and semantic search together, rerank the results, enforce access control inside the query, and return sources with every passage.
This is the layer we focus on at Intellara, and the reason we built troveGEN as a retrieval-first product.
