The Decision Log Template: Train Judgment Like a Muscle

Ask a product org for its decision log: the last ten significant decisions, with the evidence that existed at the time and what happened after. Almost none can. The decisions live in Slack threads, meeting memories, and the heads of people who have since changed teams. Which means the org cannot learn from its own calls. It learns by anecdote, and anecdote is dominated by whoever tells the story loudest.

I have watched this at every scale. Startups relitigate the same decision every six months because nobody wrote down why it was made the first time. Large orgs promote people for outcomes that were mostly luck and sideline people whose good calls got unlucky. Both failures have the same root: no record, no learning loop.

The fix costs ten minutes a week. It is a decision log, and the template is below.

The short version

The decision log is one row per significant decision: what you decided, whether it was a one-way or two-way door, the evidence you had at the time, your confidence as a percentage, what would change your mind, a review date, and later the outcome and a quality verdict. The discipline comes from Annie Duke's separation of decision quality from outcome quality and Gary Klein's premortem for big bets. Logging takes ten minutes a week. The quarterly calibration review, where you score your confidence numbers against what actually happened, is where judgment gets trained. This is the instrument behind judgment reps(coming Jul 13), and decision quality sits in the judgment layer of the skill stack, the layer agents cannot reach.

What's in the template

The log schema

One row, eleven fields. ID, date, the decision in one line, the door type (one-way or two-way), the evidence present at the time, a confidence percentage, what would change your mind, a review date, the outcome, a decision-quality verdict, and the lesson.

Two fields do most of the work. Confidence as a number forces you out of "pretty sure" into a falsifiable claim. You cannot calibrate "pretty sure." You can calibrate 75%. What would change my mind is the pre-committed kill switch. Written before the outcome arrives, it protects you from moving the goalposts later, which everyone does and nobody notices themselves doing.

The door-type field comes from the one-way versus two-way door framing Bryar and Carr describe from Amazon. Two-way doors get a row and a fast call. One-way doors get a row plus a premortem.

When to use it

Log a decision when it commits more than a week of team time, when it is expensive to reverse, when the room disagreed, or when you declined something someone senior wanted. That last category is the most valuable and the least recorded. Skipped bets are decisions too, and they are the ones orgs forget fastest.

Do not log everything. A log with two hundred rows of routine prioritization is a log nobody rereads. One to three rows a week is the right volume for most PMs. For a CPO it might be one a week, but each row is heavier.

How to fill it

In the moment, fill the first eight fields. Five minutes. The hard rule: write the evidence field with only what you knew then. No retroactive polishing. If the evidence was thin, write "evidence was thin." Future-you needs the honest version.

The outcome, verdict, and lesson fields stay empty until the review date. That gap is deliberate. Filling the verdict at decision time is just confidence restated. Filling it at review time, with the outcome known but scored separately, is where the resulting trap gets dismantled.

A worked example fragment

Here is the shape of one row from the markdown artifact, sanitized:

Decision: Sunset the legacy reporting module rather than rebuild it. Door: One-way. Evidence at the time: Usage down for three straight quarters, two support engineers carrying it, no new customer had activated it in six months. Confidence: 80%. Would change my mind: Any top-ten account naming it as a renewal condition before the sunset date. Outcome (at review): Two mid-tier accounts complained, none churned, support load dropped as projected. Verdict: Good decision, good outcome. Lesson: The "renewal condition" tripwire was the right kill switch and it never fired. Reuse it.

The verdict line is the muscle. "Good decision, bad outcome" and "bad decision, good outcome" are both legal verdicts, and the log only works if you are willing to use all four quadrants.

The calibration review

Once a quarter, 45 minutes. Reread every row past its review date. Fill in outcomes. Score verdicts. Then bucket your confidence numbers: of everything you marked around 80%, what fraction actually worked out? Around 60%? The gap between your stated confidence and your hit rate is your calibration error, and it is almost never uniform. Most people are well calibrated in one domain and badly overconfident in another, usually the one they enjoy most. Knowing which is which changes how you weigh your own gut, which is the entire point.

The premortem mini-template

For one-way doors, the artifact includes a short premortem block based on Gary Klein's prospective hindsight method. Assume the decision failed twelve months out. Write the failure story. List the three most plausible causes. Decide whether any of them changes the decision or just adds a tripwire to the "would change my mind" field. Ten minutes, and it reliably surfaces risks that forward-looking planning is structurally blind to.

How to use it this week

Download the log. Add the last three significant decisions you remember making, reconstructed as accurately as you can. Imperfect rows beat no rows.

Then log the next real decision in the moment, all eight live fields, with a review date. Put a recurring ten-minute block on Friday afternoons titled "log." Put a 45-minute block on the last Friday of the quarter titled "calibration."

That is the whole system. One row per decision, one ritual per week, one review per quarter. In a year you will have the thing almost no PM and almost no org has: a scored record of your own judgment, and the calibration data to know exactly where to trust it.

Sources: Annie Duke, Thinking in Bets (decision vs. outcome quality, resulting), Gary Klein on the premortem, HBR, Bryar & Carr, Working Backwards (one-way and two-way doors).