I made thousands of product decisions across twenty-some years and, for the first stretch, learned from almost none of them. Not because I did not care. Because nothing was written down. By the time an outcome arrived, my memory had quietly rewritten what I knew at decision time. Hindsight is a forger, and it forges in your favor. Judgment is trainable, but it trains like a sport: reps plus a scorecard.
Score the decision, not the outcome
Annie Duke's central idea in Thinking in Bets transfers cleanly to product: a decision can be good and the outcome bad, because luck sits between them. Judging the decision by the outcome is what she calls resulting, and it is the default mode of every product org I have seen. The PM who shipped a well-reasoned bet that got run over by a market shift gets cautious. The PM who skipped discovery and got lucky gets bold and promoted. Run that loop a few years and the org has trained itself on noise.
The decision log: eight fields
The scorecard is a decision log. Eight fields, two minutes per entry:
- Decision, in one sentence.
- Date.
- Evidence at the time, three to five bullets of what you actually knew. This field is the anti-forgery device.
- Confidence, as a percentage you can be wrong about.
- What would change my mind. If you cannot fill this, you have a belief, not a decision.
- Review date, on the calendar.
- Outcome, filled in later.
- Lesson, filled in later, about the process not the result.
Log five to ten significant decisions a month: bets, kills, sequencing calls, hires, pricing moves. The unexpected benefit is that the log improves decisions on the way in, not just the review on the way out.
The reps: calibration, premortems, doors
Every quarter, pull the decisions whose review dates have passed and bucket them by stated confidence. Across your 70 percent calls, roughly 70 percent should have gone your way. The first scoring humbles almost everyone. Common pattern: overconfident about timelines and adoption, underconfident about whether customers will pay.
Before a big bet, run Gary Klein's premortem. Announce it is one year from now and the project failed completely, and have everyone write the story of why, independently, for ten minutes. It trains the exact muscle resulting atrophies: generating the failure space before reality picks one.
Then sort decisions into one-way doors, hard to undo, and two-way doors, cheap to undo. Two-way doors get decided fast by whoever is closest. One-way doors get the full kit. Most orgs invert this, agonizing over reversible UI while waving an irreversible pricing change through on a single slide.
Open a blank page tonight and backfill one decision: the biggest call you made last month, with the evidence and confidence you had then. Set a review date. That is the first rep. The rest is just not stopping.