What can an eval catch that a demo cannot?

THE SHORT ANSWER

A demo is a performance on inputs someone chose; an eval is a measurement across inputs that represent reality. We demoed an AI feature on three hand-picked cases and it was flawless, then ran the eval against 200 real cases split into named slices. On average it scored fine. On one slice, about a fifth of real traffic, it produced confidently wrong output, the kind of failure worse than an error because it looks right. The demo was blind to it because a demo never samples the case that breaks the thing. The eval caught it because named slices expose what an average hides. A demo can start a shipping conversation; only an eval can end one.

A demo shows a chosen best case. An eval shows the distribution. That difference is the whole story, because a shipping decision should only ever be made on the distribution.

The failure that almost shipped

We built an AI feature, demoed it on three hand-picked cases, and it was flawless. The room wanted to ship. I ran the eval I had written before building: 200 real inputs, split into named slices. On average it scored well. The long-input slice, about a fifth of real traffic, produced confidently wrong output, the failure that is worse than an error because it looks right and the user trusts it. None of the three demo cases were long, so the demo was structurally blind to it, and the average hid it because four-fifths of traffic dragged the mean up over the failing fifth.

Why the eval saw it

A demo's job is to communicate the idea at its strongest, which means it excludes the cases that undercut the idea. It cannot tell you the product is not ready because it never sampled for readiness. An eval grades against inputs that represent what will actually arrive, and the named slice is what turns "it works" into "it works here and fails there." Shipping the demo would have returned wrong results to one in five users, a trust event that produces exactly the month-two silence a landing failure looks like. The eval cost a day. The rule that came out of it: a demo can start a conversation but cannot end one, and the eval has to exist before the thing does.

SOURCES

THE LONG VERSION

RELATED ANSWERS

Last reviewed 2026-08-12 · 2 min read