Language models are optimized to produce fluent continuations, not verified ones. Fluency and correctness are correlated often enough to be dangerous — close enough that a wrong answer can read exactly like a right one.
Retrieval-augmented generation is usually framed as a way to give a model “more information.” That framing undersells what it needs to do. The harder job is making the connection between a retrieved passage and the generated answer visible and checkable — so a wrong answer fails loudly, as a broken citation, rather than quietly, as confident prose.
That means ranking for relevance is only half the system. The other half is generation that’s constrained to what was actually retrieved, and an evaluation loop that can tell the difference between “answered from the source” and “answered from the model’s memory wearing the source as a citation.” Get that distinction wrong and retrieval adds latency without adding trust.