The Failure Inventory Worksheet
The test for each entry: could this happen on a normal Tuesday, with a normal user, with the system working as designed? If it requires a bug, it belongs in a test plan. If it requires a determined...
The template
The Failure Inventory Worksheet
List the ten worst plausible outputs of a feature before you design the good one. That list is a design doc, and it is usually a better one than the happy path spec, because it is the only artifact that tells you where to spend your constraints. Ninety minutes with three people. Do it before anything gets made.
The rule that makes it work
Plausible, not possible. Anything is possible. The inventory is worthless if it fills up with meteors and adversarial nation states.
The test for each entry: could this happen on a normal Tuesday, with a normal user, with the system working as designed? If it requires a bug, it belongs in a test plan. If it requires a determined attacker, it belongs in a threat model. Everything else belongs here, and "the system working as designed" is the part that catches teams off guard.
Who is in the room
Three people, ninety minutes:
- The designer who will own the surface.
- One engineer who knows what the system will actually do.
- One person who talks to users. Support, sales, research, whoever hears complaints.
That third seat matters most. Designers and engineers imagine failures from the inside. Support knows what failure looks like from the outside, and those are different lists.
The worksheet
Ten rows. Force yourself to ten. The first four are obvious, the last six are where the value is.
| # | The bad output | How it happens | Who gets hurt | Cost | Recoverable? | Design response |
|---|---|---|---|---|---|---|
| 1 | ||||||
| 2 | ||||||
| 3 | ||||||
| 4 | ||||||
| 5 | ||||||
| 6 | ||||||
| 7 | ||||||
| 8 | ||||||
| 9 | ||||||
| 10 |
The bad output. Write the actual thing. Not "inaccurate summary." The sentence the product would produce, as close to verbatim as you can get.
Who gets hurt. Name the person, not the system. "The user" is too general. "The support agent who sends it without reading" is specific enough to design against.
Cost. Money, time, trust, harm, or rework. If you cannot name one, this is not a failure, it is an imperfection.
Recoverable? Yes, with effort, or no. This column drives everything downstream.
Design response. One of the five below.
The five responses
Every entry gets exactly one. Choosing is the design work.
1. Prevent. Make it impossible. A constraint enforced before generation. Expensive, and correct for anything irreversible.
2. Catch. Let it happen, detect it before the user sees it. An eval, a check, a filter. Correct when detection is reliable and the check is cheap.
3. Disclose. Let it through, tell the user what to check. Correct only when the user can actually verify. Disclosure on something unverifiable is responsibility transfer, not design.
4. Recover. Let it through, make undoing it cheap and obvious. Correct for reversible failures with a low frequency.
5. Accept. Do nothing. A legitimate choice for low-cost, low-frequency, easily-spotted failures. Write it down anyway, so that when it shows up in a review nobody spends an afternoon rediscovering that it was a decision.
The distribution tells you something. An inventory where everything is "prevent" means the feature is too dangerous for its value. An inventory where everything is "accept" means nobody in the room was being honest.
The ten prompts, when the room goes quiet
Around entry five, the room stalls. These unstick it.
- What does it output when it has almost no information to work with?
- What does it output for the largest, messiest account you have?
- What happens when two sources disagree?
- What is the most confident wrong thing it could say?
- What would embarrass us if it were screenshotted?
- What could it say that is technically correct and useless?
- What happens when the user is not the person the output is about?
- What does it do at the boundary of what it should refuse?
- What happens the fifth time in a row, when the user has stopped reading?
- What is the worst thing it could do quietly, that nobody notices for a month?
Number ten produces the best entries. Silent failures are the ones that damage trust most, because when they surface, every previous output becomes suspect.
What to do with the finished inventory
The prevents become constraints. Word for word, into the constraint doc, with the failure condition already written for you. That is why this exercise pays for itself immediately.
The catches become eval cases. Each one is a test with an expected outcome. This is the fastest way to build a real eval set, and it is built from your product's actual risks rather than from generic benchmarks.
The recovers become interface work. Undo, correction, and repair surfaces, scoped and owned.
The whole list goes in the ticket. Not a separate document that gets read once. In the ticket, where the work happens.
Re-run it
Once at design time, once after the first month of real usage. The second run is better, because now you have the failures you did not imagine, and comparing the two lists tells you how good your team's imagination is. That calibration is worth having.
The one thing to do this week
Ninety minutes, three people, the feature you are about to start. Ten rows. If you get to ten and the last three feel forced, you probably have a low-risk feature and you can move fast. If you get to ten easily and could keep going, you just learned where the quarter's design work actually is.
From "The Failure Inventory", chapter 14 of The Design Operating Model. falkster.com/design/the-failure-inventory