Why your AI assistant gives wrong answers (it's probably not the AI)
A better prompt won't fix an assistant that's reading the wrong passage of your document.
When an AI assistant that answers from your own documents (this approach is called RAG, short for retrieval-augmented generation) gives a wrong or unsupported answer, the instinct is to blame the AI itself: try a better prompt, a bigger model, stricter instructions. Almost always, the actual fault happened a step earlier: the system pulled up the wrong passage, or no passage at all, and the AI is doing its best with bad source material.
Retrieval quality is determined mostly by decisions made before any model is involved. How documents are chunked (by fixed token count, or respecting headings and tables) determines whether a chunk contains a complete, answerable unit of information. How chunks are ranked (pure semantic similarity, or hybrid with lexical search) determines whether an exact term match beats a vaguely-related passage. Whether tables and structured data are extracted separately or flattened into prose determines whether numeric answers are even present in retrievable form.
The only reliable way to know if any of this is working is an evaluation set: a list of real questions with known-correct answers, checked against retrieval output directly, before the model ever generates a response. Without one, every reported improvement is a guess.
We build the eval set before we tune anything. It's the only way to tell whether the second version is actually better than the first, rather than just different.