Recovery and Trust Repair

The short version

Trust repair after an AI is confidently wrong is a designed sequence, and almost nobody has written one. Six steps, in order: detect it, admit it in the same place the wrong answer appeared, contain it by saying what was and was not affected, correct it visibly, name the specific check now in place, and adjust the product's confidence posture on that class of output. Order matters more than eloquence, because good copy in the wrong order still fails. The reason this moment is different from an outage is that the product asserted something and the user acted on it, so they now doubt every earlier answer they did not check. Write the four-sentence script per surface before you need it.

This chapter is not about outages.

An outage is honest. The product is down, everybody can see it, nothing was claimed, and trust takes almost no damage because there was nothing to believe in the first place. Teams already have a process for that, and if yours does not, Incident Response is the version to copy.

This is the other failure. The product asserted something specific, in a confident voice, and it was wrong, and somebody acted on it.

That damages trust differently and the difference is the whole design problem. The user does not just doubt the answer in front of them. They start doubting every previous answer they did not check, and there are a lot of those, because the entire value proposition was that they would not have to check. Repair has to reach the back catalogue, not only the current mistake.

Which means an apology is close to useless here. An apology addresses one incident. The user's actual question is about a category.

Detect, and make correction cheap

Before anything else, know it happened.

Teams find out in one of four ways, and the order is roughly a ranking of how bad the situation already is. The system's own check caught it. The user corrected it in the product. The user complained. Somebody outside noticed publicly.

Design owns exactly one lever here and it is a big one. Make in-product correction the easiest path available.

Every correction is a detection you would not otherwise have had, and it arrives with context attached. If reporting a wrong answer takes more effort than shrugging and moving on, you have chosen not to know, and the choice was made in a design review where nobody framed it that way.

In healthcare a confident wrong answer isn't a shrug. At Commure we added a quiet way to flag a bad output and braced for a trickle. It exposed a whole class of error we didn't know existed. The one bug we set out to fix was just the tip.

Admit, in the same place

Put the correction where the wrong answer appeared. Not in an email. Not in a changelog. Not in a banner on a different screen.

Two rules for the copy. Say wrong, not "may not have been fully accurate." And say it in the first sentence, because people have decided how they feel by the second one.

Write this copy in advance. During an incident nobody writes well, and the version that ships is whatever somebody typed in a hurry with legal reading over their shoulder.

Contain, then correct where they can see it

Tell them the blast radius before they ask, because the unasked question is always what else was wrong.

Teams get this part backwards. Naming what was not affected does more work than naming what was. Say the usage summary between the third and the ninth was wrong and that billing and exports were not touched. Without the second half, the user assumes everything, and the assumption is free for them and expensive for you.

If you do not know the radius yet, say that and say when you will know. A stated "still checking, update by four" holds far better than silence. Silence in this window is read as either incompetence or concealment, and the user gets to pick.

Then the correction itself.

Show the corrected value, labelled as corrected. Show what changed, not just the new state. If the person acted on the wrong answer, give them the path to unwind that action, which is a lot easier if the work in Undo Is a Design Primitive is already done.

Never quietly replace the wrong value with the right one. Silent correction is the fastest way to lose a user who noticed, because they now know the product edits history without telling them, and there is no coming back from that particular piece of knowledge.

Prevent, in a way they can see

This is the step teams skip and the one that does most of the repair.

One sentence about the mechanism. A check that catches date range mismatches before the summary renders. Not a promise about caring more. "We take accuracy seriously" repairs nothing, and every user has read that sentence forty times from companies that did not.

A named check repairs a lot, because it tells the person the class is closed rather than the instance.

The strongest version is not a message at all. It is a new state they can see in the product: a confidence signal where there was none, a source citation on that output, a confirmation step that was not there last week. Design owns whether the prevention is visible, and visible prevention is the difference between a company that says it learned something and a company that shows the shape of what it learned.

Change the posture

After a confident error, the product's voice on that surface changes. Not forever and not everywhere, but on that class of output for a while.

Concretely: move up a rung on the ladder in Disclosure Without Overwhelm, add the source, drop the assertive phrasing. When the eval shows the class is fixed, move back. Put the revisit date in the decision record so moving back is a decision somebody makes rather than a thing that quietly never happens.

This step is what separates repair from apology. It costs something. That is exactly why it is believable.

Two things still sink a repair that did all six steps, and both are quiet. Repairing in a channel the user does not read, for one. A status page records a repair. It does not perform one.

Apologizing more than correcting, for the other. People notice the ratio of message spent on the fix versus message spent on how sorry you are. Aim for something like three to one in favour of the fix, and read your draft out loud to check, because the sorry parts are the parts that feel safest to write.

Decide severity before you need it

Three tiers, decided on a calm Tuesday. Low is wrong, easy to spot, no action taken, admitted inline and signed off by design. Medium is wrong, the user likely acted, recoverable, admitted in place plus a notification, signed off by design and PM. High is wrong with an irreversible action taken, or money, access, or safety involved, which means in place plus notification plus direct contact, signed off by a named executive.

What that table buys you is that during an incident it stops an argument about process at the exact moment you cannot afford one. It also belongs in the record of decisions the practice keeps, which is the argument in The Receipt and the Work.

This week: write the four-sentence script for the surface where a confident error would cost you the most. Plain statement that it was wrong, what was affected and what was not, the corrected state and how to unwind anything done on the old one, and the specific check now in place. Then show it to whoever would have to approve it during a real incident. Getting that approval before you need it is worth more than any amount of polishing the wording.

The full sequence with owners per step, the severity table, and the script template are in The Trust Repair Sequence. The short version is What do you do in the thirty seconds after your AI is confidently wrong?

---

Chapter 11 of a series on the design operating model. Next: eval-driven design, and how to write taste into a rubric that executes without you in the room.