A wrong citation is not a defect
In most software a wrong answer is a defect. In legal software a confidently wrong citation is professional negligence, handed to a user in exactly the form they are inclined to trust. It arrives formatted like a real authority, in the register a correct answer would use, at the point in the workflow where the reader has already decided to stop checking.
That sentence is the design brief for CourtNetra, our legal research platform over 18,863,754 Indian judgments. It is also why the interesting number on that system is not how many documents it searches. It is how often it asserts something the corpus does not support, and how anyone other than us could verify that.
The duty does not sit with the vendor. A practitioner who files a brief containing a fabricated citation is answerable for it under their own professional rules, and no clause in a software contract moves that. A system that asks to be trusted is therefore asking for something it cannot give back. So the design constraint is not trust, it is checkability: every claim carries a pointer to the passage that supports it, or the claim does not get made.
Fluency is the hazard, not incoherence
The failure everyone prepares for is gibberish, and gibberish is close to harmless because any reader catches it. The failure that causes harm is a well-formed citation to a case that says something else, or to a case that does not exist. Both come out of the same mechanism that produces the correct answers: the model is optimized to continue text plausibly, and a plausible continuation of "the leading authority is" is a case name, whether or not the right one is in context.
Two properties of a legal corpus make this sharper. An overruled authority is textually indistinguishable from a live one, so similarity search cannot separate them; on the text they are the same kind of object. And the reader's own expertise works against them, because a practitioner recognizes citation format instantly, and format recognition is precisely the check a fabricated citation passes.
The hazard is not that the system sounds uncertain when it is wrong. It is that it sounds identical when it is wrong.
Which means the product question was never how to make answers sound more authoritative. It was the opposite: how to make the system decline at the boundary of what the corpus supports, and how to demonstrate week after week that it still does.
Grounded has to mean something a second person can check
Ask any vendor for a hallucination rate and you will get a percentage. The percentage is not the claim. The claim is the definition, the set it was measured against, and who adjudicated each case. Without those three the number is not falsifiable, and a number that cannot be falsified is not evidence about the system; it is evidence about the marketing department.
Our operational definition has three conditions, and a response is grounded only when all three hold for every factual assertion in it. Each is checkable by someone who was not involved in generating the answer, which is the property that makes the definition worth having.
- Supported
- A retrieved passage says the thing, not something adjacent to it. A passage answering a neighboring question is not support, and treating it as support is how an evaluation quietly inflates itself.
- Retrieved
- The support was in the context for that specific response. A claim that happens to be true but was never retrieved is a correct guess, and we count it as ungrounded. The mechanism that produced a lucky right answer will produce an unlucky wrong one on the next query, and scoring it as a pass hides exactly the behavior we are trying to detect.
- Attributed
- The citation resolves to the passage actually relied on. A citation to a real case that does not contain the proposition is the single most dangerous output the system can produce, because every component of it survives inspection: the case exists, the reporter exists, the format is correct. It counts as a failure.
The second condition is the one most evaluations omit, and it is the reason we measure groundedness rather than accuracy. An answer can be correct and ungrounded at the same time, and a system that produces correct ungrounded answers is a system that is guessing well this month.
A refusal is not a failure. Where the corpus cannot support a claim, declining is the right output and it scores as correct. That creates an obvious perverse incentive, and the rubric below is built to close it.
The evaluation: a golden set, a rubric, and a run every week
The golden set
A golden set is a fixed collection of queries with known-good answers and, for our purposes, the specific passages that support each one. Ours is built from the corpus we actually serve rather than from a public benchmark, because a public benchmark measures how well a system does on a public benchmark. Four rules govern it, and each exists because breaking it produces a number that looks fine and means nothing.
- Shaped like real traffic. Citation lookups, proposition searches, questions spanning several authorities, and questions about whether an authority is still good law. Those are the shapes users send, so those are the shapes the set contains.
- Deliberately unanswerable items. Queries the corpus genuinely cannot support are included on purpose, so that refusal is measured rather than assumed. An evaluation made only of answerable questions never learns whether the system knows how to stop.
- Adversarial items. Near-miss authorities, superseded ones, and questions whose obvious-looking answer is the wrong one. These are the items that fail first when retrieval quality drifts, which makes them the early warning.
- Frozen targets. Once an item is in the set, its expected answer is not edited to match what the system now does. Editing the target to match the output is the fastest way to make an evaluation useless, and it almost always happens by accident, one reasonable-looking correction at a time.
The rubric
Every scored response gets exactly one label. Two labels pass, two fail, and the fifth exists to stop the gate being gamed.
| Label | Condition | Effect on the rate |
|---|---|---|
| Grounded | Every assertion is supported by a retrieved passage and every citation resolves to it. | Passes |
| Correct refusal | The corpus cannot support an answer and the system declines instead of producing one. | Passes |
| Unsupported assertion | A claim with no retrieved passage behind it, whether or not the claim is true. | Counts against the rate |
| Misattributed citation | The citation resolves to a real authority that does not contain the proposition. | Counts against the rate |
| Incorrect refusal | Support existed in the corpus and was retrieved, and the system declined anyway. | Tracked separately, never counted as a pass |
Who decides
Scoring is mechanical wherever mechanical is possible and human where it is not. Citation resolution is code: the cited passage either contains the proposition or it does not, and that check does not need an opinion. Whether a paraphrased proposition is supported by a passage is a judgment, and judgments are made by a person against the written rubric, with a sample re-scored later to check the rubric is still being applied the same way. Model-assisted scoring is useful for triage and for finding candidates worth a human look. It is not what the reported number rests on, because a model judging a model shares failure modes with it.
The run, and the gate
The evaluation runs weekly in CI, automatically, against the pipeline as it is actually configured: the live retrieval settings, the pinned model version, the real corpus. Not a stripped-down harness that skips retrieval, because retrieval is where most of the failures come from, and an evaluation that bypasses it measures a system nobody uses.
ungrounded = (unsupported + misattributed) / scored_items
if ungrounded > 0.02:
fail() # the release stops hereThe threshold is 2%. Above it, the build fails. Failing means failing: there is no green-with-warnings state, because a warning state is a threshold nobody enforces, and a threshold that whoever is shipping that day can wave through is a suggestion with a number attached.
Why it is a build gate and not a dashboard
This is the part worth taking away, and it is not really about legal software.
Ordinary software regressions announce themselves. A null dereference throws. A schema change breaks a query. A latency regression trips an alert, because latency is a number a machine is already watching. Every one of those has a mechanism that interrupts a human being who did not ask to be interrupted.
Nothing throws when a model simply becomes less right. The service returns 200. Latency is unchanged. The response is the same length, in the same register, with the same steady tone, and it is wrong. There is no exception, no stack trace and no page. The only thing that changed is the fraction of answers a careful reader would reject, and nobody is measuring that at three in the morning.
Quality that fails silently has to be enforced where regressions are blocked, not where they are observed.
A dashboard is consulted when somebody remembers to look, and attention is highest exactly when the system is newest and least likely to have drifted. Six months later the chart is still being drawn and nobody has opened it. A gate is consulted on every release by a machine that does not get bored of the question, and its output is binary, which is the only form in which a quality signal survives contact with a delivery deadline.
What makes the number move is rarely a change we made on purpose. A provider ships a new model version. A chunk size gets tuned to shave latency. The corpus grows and shifts the neighborhood structure of the index, so a query that used to retrieve the right passage now retrieves its neighbor. A prompt gets edited to fix a formatting complaint and loses a constraint on the way. Each change is individually reasonable, none arrives with a failing unit test attached, and the gate is the only thing in the system with an opinion about their combined effect.
What the number does not tell you
Five things, and the first one applies to every accuracy figure any vendor has ever quoted, including ours.
- It is a property of the set, not of the corpus. Under 2% means under 2% on our golden set. A user whose questions look nothing like that set is not covered by the figure. That is the first question to ask about anyone else's number, and it should be the first question asked about this one.
- A rate is not a distribution of harm. One misattributed citation in a high-stakes matter is worse than twenty harmless imprecisions, and a single percentage flattens the difference. The rubric labels are tracked separately for this reason, but the gate itself is a scalar, and a scalar loses information by construction.
- It says nothing about usefulness. A system that answers narrowly and refuses often scores well and serves people badly. Incorrect refusals are tracked precisely because the groundedness rate cannot see them, and no automated check will tell you whether the answers were worth reading.
- Human judgment carries human variance. Whether a paraphrase is supported by a passage is a judgment call. A written rubric and periodic re-scoring reduce drift between adjudicators. They do not eliminate it, and we do not claim they do.
- It ages. The set has to be extended as the corpus grows and as query patterns change. An evaluation set that stops growing quietly stops measuring the current system, and the number it reports keeps looking healthy while it does.
What it costs to maintain
This is the part that gets left out of the conference talk. The harness is not the expensive component; the harness is a weekend. The expensive component is the golden set, which has to be built by somebody who knows the domain well enough to say whether a passage genuinely supports a proposition, and which has to be extended and re-adjudicated as the corpus moves. That is a recurring commitment of expert attention, not a one-time build.
Then there is the cost of the gate doing its job. Every failing run buys an engineer a real investigation, because the useful output of a failed gate is a diagnosis and not a red square. And the gate blocks releases, which means it will eventually block a release somebody wanted badly on a Friday afternoon. That is the entire point of having it, and it is also the exact moment the pressure to add an override arrives.
Where this applies outside law
Everything above generalizes to any domain where a confident wrong answer transfers liability to the reader: a coverage determination, a clinical summary, a KYC finding, a tax position. The corpus changes and the rubric changes with it. Two structural facts do not. You need a definition of grounded that a second person can check, and you need the threshold to sit somewhere that blocks a release rather than somewhere that colors a chart.
If you already have a number, the useful exercise is to write down its method and see whether the number survives. If you do not have one yet, the definition comes first, then the set, then the threshold. Building the harness before the definition produces a green build and no information.
Written by Aniruddh Atrey, founder and CTO of MetaMinds. Published 2 September 2026 and verified 2 September 2026. Every figure quoted here is from CourtNetra, a production system MetaMinds operates. No client is named on this site, and MetaMinds holds neither SOC 2 nor ISO/IEC 42001.