The Design Rubric Template

Your taste currently lives in a Thursday meeting. This moves it into something written, scored, and versioned, so it executes when you are not in the room. The rubric is not something you write from principles. It is something you extract from your own judgment on real work. Three blocks, one afternoon for the first pass.

---

Why extraction and not invention

A rubric written from first principles describes a product nobody shipped. It will contain "clarity," "consistency," and "delight," and it will score every output a 4 out of 5 forever.

A rubric extracted from thirty real outputs describes the specific ways your product is actually good and bad. Those are the dimensions worth scoring, and you already know them. They are sitting in the sentences you say in review.

So the process is: grade first, then find the pattern in your own grading.

---

Block 1: Grade thirty outputs on gut feel

No criteria yet. That is deliberate. Being inconsistent here is fine, the inconsistency is data.

Pull thirty real outputs from the surface, or from a prototype of it. Real ones. Not examples you wrote to make a point.

#	Output or link	Gut call (good / mixed / bad)	Why, one sentence
1
2
...
30

Two things to watch as you go, because they are the whole point:

Words that repeat in your "why" column. "Invented a number." "Buried the answer." "Hedged." "Sounded like a robot." Those repeated phrases are your candidate dimensions.

Places you graded two similar outputs differently. Those disagreements with yourself are exactly where the rubric has to get precise, and they are the most valuable rows on the sheet.

---

Block 2: Extract the dimensions

For each repeated phrase, write a candidate dimension, then apply the cost test. A dimension stays only if failing it maps to a real cost.

Candidate dimension	What failing it costs	Keep?
Invents a figure	User acts on a false number, trust gone	KEEP
Buries the answer	Extra read time, mild annoyance	maybe
Tone feels off	Cannot name a cost	CUT

The rule: if you cannot name the cost, cut the dimension. Aim for three or four dimensions that each hurt, not eight that measure vibes. Every dimension you keep is one somebody has to score on every review forever, so the bar is high.

---

Block 3: Write the rubric

Each dimension gets a binary test, not a five-point scale.

Binary because scales lie. Given 1 to 5, two reviewers will both write 3 for different reasons and believe they agreed. Given pass or fail, they have to say which, and the disagreement surfaces where you need it.

Dimension	Passes when	Fails when	Severity
			blocker / major / minor

Severity does the work a numeric scale pretends to do. A blocker means do not ship, one instance is enough. A major means fix before the next release. A minor means log it and watch the rate.

Write the "fails when" column first. It is more specific and it forces the precision the "passes when" column can fake.

---

The calibration round

Before the rubric is real, calibrate it. This takes thirty minutes and it is not optional.

1. Two people score the same five outputs independently, using the rubric.
2. Compare, dimension by dimension.
3. For each disagreement, ask which of these it is:

The rubric is vague. They read the same test differently. Rewrite the test. This is the most common outcome on a first pass and it is a win, not a failure.

The rubric is measuring the wrong thing. They agreed on the score and both feel the score is wrong. The dimension needs replacing.

They disagree about quality, for real. Rare, and important. Escalate it, make the call, and record it. That call is the actual taste you are trying to encode.

Repeat until two people land within one disagreement on five outputs. Then it is usable.

---

Versioning

The rubric is a spec, so it gets versioned like one.

rubric-<surface>-v<n>.md
Changed: <what moved>
Why: <the output or incident that caused the change>
Date:
Owner:

Two rules that keep it honest:

The rubric changes when the product changes, not when a score is inconvenient. Lowering a bar to make a release pass is the failure mode this whole system exists to prevent, and it happens quietly, on a Friday, with good intentions.

Old scores stay comparable or get marked. When a dimension changes, scores from before the change are labelled with the version. Otherwise your quality trend is measuring your rubric edits.

---

Turning it into an eval

The point of the rubric is that it runs without you. Once it is calibrated:

• Every dimension with an objective test becomes an automated check. Fabricated figures, missing citations, format violations, forbidden content. These are cheap and they should run on every build.
• Every dimension needing judgment becomes a scored sample. A fixed set of outputs, scored by a model against your written test, spot-checked by a human weekly.
• Every dimension that needs a person stays with a person, in a named step with a named owner and a slot on the calendar. Do not pretend this category is empty. Pretending it is empty is how products become technically correct and joyless.

Dimension	Automated	Model-scored	Human

---

What the rubric is not

It is not a replacement for looking at the work. It is what makes looking at the work fast, because most of the argument already happened when the rubric was written.

It is not a performance review instrument. The moment scores are attached to people rather than outputs, the scores become political and the rubric dies.

It is not permanent. A rubric that has not changed in a year is describing a product from a year ago.

---

The one thing to do this week

Do Block 1 only. Thirty outputs, gut call, one sentence each. It takes an afternoon and you will find the dimensions in your own notes, which is a different experience from being handed somebody else's framework. Then do Block 2 next week.

---

From "The Rubric Is the Spec", chapter 12 of The Design Operating Model. falkster.com/design/the-rubric-is-the-spec