Tier 2 · fixed-price engagement
Hallucination and grounding audit
How often your system is wrong, measured against a golden set built from your own corpus, and handed over so you can run the same measurement after every change.
Who this is for
A team that has shipped a retrieval or question-answering system and is being asked how accurate it is, with no number to give.
Fixed price
$5,900
- Duration
- 2 weeks
- Deposit
- $1,000
- Other currencies
- €5,400 · £4,490 · S$7,900
- Catalog
- T2-03
Thirty minutes with the engineer who would run the work, before any paperwork.
The work, stage by stage
In order, with what happens at each stage and what it produces. The timings are the ones the catalog prices, not an estimate written for this page.
Build the golden set
Days 1 to 4
Questions drawn from your real corpus and your real traffic, each with the passage that answers it identified. The set is the asset: everything after this is a measurement against it, so a weak set makes every later number meaningless.
Define what counts as wrong
Day 5
Groundedness, whether the answer is supported by the passage it cites, is kept separate from correctness, whether it is true. A system can be grounded and wrong, and conflating the two hides which half is broken.
Run the measurement
Week 2, first half
Every question through the system as deployed, scored on groundedness, citation accuracy, and behavior on questions the corpus cannot answer, which is where confident fabrication lives.
Attribute each failure to a stage
Week 2, second half
A wrong answer is a retrieval failure, a ranking failure, a context-assembly failure or a generation failure, and the fix differs for each. Attribution is what turns a score into work.
Hand over the harness
On delivery
The golden set and the scoring harness, in a form your continuous integration can run, so the number is reproducible without us.
Everything that arrives, listed
The contents of the document, and the artifacts that come with it. If something here would not be useful to you, say so before we start and the scope changes.
- The golden set, questions with their supporting passages, which is yours as an evaluation dataset built from your data
- A measured groundedness and citation-accuracy figure for the system as deployed, with the method stated so the figure can be reproduced
- Each failure attributed to retrieval, ranking, context assembly or generation
- The scoring harness, runnable in your CI under the perpetual license our terms grant for our own tooling, so a release can fail when accuracy degrades
- A behavior note on what the system does with questions the corpus cannot answer
What this covers, and who takes the rest
Stated up front, because scope is what derails an engagement in week two rather than at the proposal.
Inside this engagement
- Golden-set construction from your corpus and your traffic
- The deployed system end to end, retrieval through generation
- Refusal and out-of-corpus behavior, measured rather than assumed
Handled separately
Named so you can plan and budget for it, whether we do it or somebody else does.
- Fixing what the audit finds, which is a build engagement once attribution says which stage to work on
- Ongoing monitoring after handover, which the Maintain retainer covers
Start with a call, not a proposal.
Thirty minutes, straight to the engineer who would run the work. We will tell you if a smaller engagement answers your question, or if nothing here does.
Typical reply within one business day · Engagements start at $2,500
Fixed price · 2 weeks