The chatbot they asked for, and the rulebase it needed first
Built a bitemporal eligibility rulebase, then an internal assistant on top of it that answers only from those rules — citing the rule, its version, and the dates it applied, including as of any past date.
The practice wanted a chatbot for eligibility criteria. But those criteria lived across spreadsheets, saved prompt text, and consultants' memory, with no single current version and no record of what the rules had been in the past. A conversational interface over that would have been a confident, fluent, unauditable liar — and the fluency is what makes it dangerous, because it removes the friction that would prompt someone to check.
Criteria change frequently and sometimes retroactively, and the people who edit them are not engineers. Under Canadian licensing rules, telling an applicant whether they qualify is restricted to licensed representatives — so the assistant had to be internal-only and structurally incapable of producing an eligibility determination, not merely instructed against it.
We inverted the ask: rulebase first, assistant second. Every rule is modelled bitemporally — the period it applies to, and separately when the system learned of it — because retroactive criteria announcements are routine here. The assistant has no free-form knowledge path; queries resolve to structured lookups against that store, and where a rule doesn't exist it says so rather than falling back on the model.
A bitemporal ruleset as the practice's single source of truth, a review-gated editing workflow their own team runs, a fixture-based test suite, a replay harness that reproduces any past decision against the rules in force on its date, and an internal assistant that answers in plain language with the rule, version, and effective dates attached — including as-of-any-past-date queries.
We designed with effective dates only. Then the criteria changed mid-build — retroactively — and the model couldn't represent the difference between what applied and what we knew when. That broke the assistant's spec too, since as-of querying is only meaningful over a bitemporal store. We moved to full bitemporal modelling mid-engagement, at about a week's cost. The change that broke the design was also the proof it was needed: in this domain, a criteria change mid-project is the base rate, not the exception.