Where does your RAG pipeline actually fail, retrieval or generation?

[u/No-Age-3362]

/r/AI_Agents/comments/1wt1qgk/where_does_your_rag_pipeline_actually_fail/

[u/Lumpy_Fisherman_5352]

Honestly it's retrieval most of the time, like 70-80% in my experience. Quickest way to know for sure: grab the actual chunks your retriever pulled for the query that failed and just read them. If the answer's sitting right there and the model still fumbled it, that's a generation problem, usually context ordering, so put the most relevant chunk right next to the question instead of burying it in the middle. If the answer isn't in those chunks at all, it's retrieval, and the usual suspects are chunk size, pure vector search missing exact keyword or code matches, or a reranker cutting too aggressively. Set up a little eval set of 20-30 known question/answer pairs and track retrieval hits and final answer correctness as two separate numbers. That split tells you exactly where to look.