What does Marily Nika's framework say about a 3% hallucination rate?

That the number alone tells you almost nothing. Evaluate on consequence, detectability, and recovery, break the 3% into failure modes (her example: 1.4% outdated facts, 0.8% unsupported claims, 0.5% fabricated citations, 0.2% entity confusion, 0.1% high severity), ask what users do with the answer downstream, and set a launch bar per failure mode rather than one accuracy threshold.

Why split by traffic slice rather than only by failure mode?

Because failures cluster by who is asking. On the launch where I found the broken slice, the failure-mode view showed a modest share of unsupported claims, and the slice view showed one in five customers getting the bad version every time. We fixed the slice and shipped a week later on a number. A failure-mode view alone would have shipped it as it was.

How fast did the accuracy move after launch?

Six weeks. The classifier's held-out set was English-heavy, so non-English customers were scored as angry regardless of content, and 89% became 61% in production. The fix was language-segment evals every week and a standing rule never to trust an aggregate without the per-segment breakdown.

What has to exist before the feature leaves staging?

The instrumentation contract: success metric, leading indicators, a cost meter per successful action, a named eval set with a pass threshold, trace points, a dashboard URL that exists before launch, and a kill condition with the metric, the threshold, and the review period. Missing any of the seven means the feature waits.

Should You Ship an AI Feature That Hallucinates 3% of the Time?

Should you ship an AI feature that hallucinates 3% of the time?

THE SHORT ANSWER

Not on the aggregate. First split the eval by traffic slice, because the 3% concentrates: a feature of mine scored well on average across 200 real cases and failed badly on one slice that was about a fifth of actual traffic, which a failure-mode breakdown would have called a small share of unsupported claims. Then write down who measures it in week six, because a sentiment classifier I shipped went from 89% on held-out data to 61% in production within six weeks. Marily Nika's consequence, detectability, and recovery framework holds; the change is that detectability is a product decision, a named eval set with a threshold, a dashboard before launch, and a kill condition, not a fact about whether users can spot the error.

Marily Nika's interview question is the right one, and her framework survives a real launch. Which Slice Holds the 3%? adds the two numbers that change what a candidate should answer: the slice where the 3% concentrates, from The Eval That Caught What the Demo Did Not, and the week-six drift from 10 AI Agents I Built That Failed. The Honest Retrospective., and moves detectability from the user's side of the table to the product's, using the instrumentation contract in Ship With Observability or Don't Ship.

It belongs to the AI Product Management argument: the parts of the job nobody trained for are the ones that decide whether a number is safe to ship on.

Last reviewed 2026-09-29 · 1 min read

Should you ship an AI feature that hallucinates 3% of the time?

SOURCES

THE LONG VERSION

RELATED ANSWERS