An internal policy assistant that cites its sources
Built a retrieval-augmented assistant over policy and procedure documents, with source citations on every answer.
Staff were spending hours a week searching a shared drive of policy PDFs and asking colleagues instead, because full-text search returned the right document maybe half the time and never the right paragraph.
The documents were inconsistently formatted — scanned PDFs, tables, nested headings — and any wrong answer on a compliance question carried real cost, so citations were non-negotiable, not a nice-to-have.
We built a chunking pipeline that respects document structure rather than splitting on a fixed token count, added hybrid lexical and semantic retrieval, and built a 60-question evaluation set from real staff queries with known-correct answers to measure retrieval quality before writing a single line of assistant UI.
An internal assistant with inline citations back to the source paragraph, and a re-runnable evaluation set the client's team can use to check quality after every document update.
Our first retrieval pass under-performed on tables embedded in the policy PDFs — a large share of the compliance-critical numbers lived there. We rebuilt the chunking step to extract tables separately before re-running the eval set, which cost roughly a week against the original timeline. We flagged the slip to the client as soon as we saw the eval numbers, rather than after the fact.