The PRD was a coping mechanism. It existed because the person who knew what good looked like could not test the thing themselves, so they wrote it down at length and hoped someone would build the right thing. In 2026 I can test the thing myself. So I stopped writing PRDs and started writing eval sets. The eval is the spec.
What an eval set is
For any AI surface I am building, the eval set is 30 to 200 real input/output pairs that define good. Five moves build one.
- Gather 30 real inputs. Actual customer messages, support tickets, sales call transcripts. Not hypothetical ones. If you do not have real inputs, the feature is not ready.
- Label the desired output for each. This is the actual PM work. If you cannot write it, you do not understand the problem yet.
- Include the failure modes. Add 10 to 20 adversarial cases the system should refuse or handle carefully. These are often more informative than the happy path.
- Score with a rubric. Exact match for structured output, semantic similarity for generative, LLM-as-judge for judgment calls, human review for taste. Different rubric per slice, same file.
- Run the set every day. Every prompt change, every model swap, every deploy. Stored in the repo, running in CI, alerting on regressions.
That is the spec. Engineering builds against it. I do not have an opinion about whether the thing is ready. I have a number.
Why this ends the debate
A typical feature review before the shift was a 40-minute meeting, five slides of opinions, and a decision everyone reversed by Friday. Now the review is one screen: eval scores by surface, the diff since last week, the top three regressions with the inputs that caused them. Twenty minutes, and the decision holds. The eval is the only thing in the room with standing to end the argument. Either the score went up or it did not.
The example that made it click
At Smartcat I ran a feature that had to decide whether a translation request should route to a human linguist or stay with AI. The old way meant writing a routing PRD and watching engineering build something different from what I meant. The new way: I spent an afternoon pulling 150 real translation requests, labeled each one as AI, human, or escalation, and wrote the rubric. Engineering had a working router in three days. No PRD. No spec review. The eval was the spec, and the review was the score.
You are probably about to write a PRD. Do not. Pick that same feature, open a file called evals/feature-name.md, and write 30 real inputs with their correct outputs plus 10 adversarial cases. Hand it to your engineer and say: this is the spec, build against it.