A RAG answer cites papers and still invents a disagreement

A research assistant retrieves relevant papers and attaches citations, yet tells a practitioner the literature is split when the cited studies agree. A RAG builder’s account describes that failure in their own system. The lesson is that source presence and synthesis accuracy are separate properties.

Test retrieval before prose

Collect questions with known relevant passages, including questions that need contrary evidence. Inspect extraction, filtering, keyword and vector retrieval, and reranking outputs separately. The post describes missing extracted pages and an early filtering step that skewed which papers were considered. Those are plausible pipeline risks to test; its reported measurements are not a benchmark for other deployments.

Test whether the answer reflects agreement

Create examples where sources agree, genuinely conflict or do not answer. Score whether the response invents a split, erases a real split or overstates certainty. LangSmith’s evaluation guidance distinguishes offline datasets for regression checks from online monitoring for production drift. A citation check should confirm that each claim is supported by the cited passage, not merely that a link exists.

Compare versions with identical evidence

When changing prompts, rerankers or filters, run old and new versions against the same questions and the same source snapshot. Record retrieval recall, answer faithfulness and the action recommendation separately. Human review remains necessary for consequential professional advice.

Adovo’s view

Trustworthy AI does not win by sounding more certain. It wins by showing where evidence ends, where interpretation begins and when the right answer is that the sources do not decide.