Stop Picking Winners. Design the Tournament.

Product judgment did not die when prototyping got cheap. It moved up a level.

The short version

Cheap prototyping did not kill product judgment, it moved the judgment up a level. When testing an idea cost a quarter of engineering time, the first decision picked the winner and we wrapped it in a twelve-page PRD. When testing an idea costs an hour, the first decision picks the bracket: which eight of fifty backlog ideas get prototyped at all. Same judgment, radically cheaper to be wrong. The job moved from picking winners to designing the tournament, a portfolio exercise built on three questions: which ideas are cheap to test and expensive to skip, which ideas share one prototype, and which cannot be settled by a prototype at all. Marton Gaspar named the failure mode as cosplay: PMs who drop the betting to become part-time engineers running an unseeded tournament where whatever got built first wins.

Last week Marton Gaspar pushed back on my claim that prototypes replaced the upfront bet. His point: you cannot prototype all fifty ideas on the backlog. Someone still has to decide, before any evidence exists, which ideas deserve an hour. Different decisions, not different sequences. He is right, and I conceded it in the thread.

But conceding the point revealed something better than the point itself.

The bet survived. It shrank.

When testing an idea cost a quarter of engineering time, the first decision picked the winner. You got one shot. Judgment had to carry the entire weight of the outcome, which is why we wrapped it in twelve pages of justification and called it a PRD. That whole apparatus is what I argued had collapsed into a brief, a prototype, and an eval.

When testing an idea costs an hour, the first decision picks the bracket. Which eight of the fifty get prototyped. Same judgment, radically cheaper to be wrong, which changes what the judgment has to carry. Building the prototype itself is now the easy part, the hour-long instant prototype any PM can run.

That is the shift. The job before data did not disappear. It moved from picking winners to designing the tournament.

What tournament design actually looks like

Picking a winner is a conviction exercise. You stack evidence, you build a narrative, you defend the choice in a room.

Designing a tournament is a portfolio exercise. Three different questions:

Which ideas are cheap to test and expensive to skip? Those enter the bracket automatically. The cost of being wrong about excluding them exceeds the hour it takes to check. This is the cost of being wrong run in reverse: not what a bad bet costs, but what a skipped test costs.

Which ideas share a prototype? Five backlog items that all hinge on the same workflow assumption are one test, not five. Bracket design is mostly deduplication.

Which ideas cannot be settled by a prototype at all? Pricing, positioning, and platform bets do not resolve in an hour of building. Those stay judgment calls, and pretending a prototype settled them is how teams fool themselves.

The failure mode has a name

Marton called it cosplay, and his definition is the cleanest I have seen: not PMs writing code, but PMs dropping the betting to become part time engineers.

Building the prototypes yourself feels like progress. It is the middle of the job. If nobody is designing the bracket, you are just a slower engineer running an unseeded tournament where whatever got built first wins.

The scarce input was never the prototype. It was deciding which fifty-to-eight cut happens before anyone opens an editor. That call got cheaper to get wrong. It did not get less necessary. Deciding well under that kind of repeated, low-cost pressure is exactly the muscle I mean by judgment reps(coming Jul 13).

Design the tournament. Then let the prototypes play it out.

This frame came out of a comment exchange with Marton Gaspar. The cosplay line is his. Concede the strong point, keep the better frame, a method I unpack in commenting with a frame.