Nielsen Says Treat a Model Change as a Redesign. The Redesign Your Users Feel Is the Harness Commit.

Jakob Nielsen's new piece on the expectation economy closes on a guideline I agree with and want to move one layer down. This is The Rewrite: treat model changes as redesigns becomes treat every harness commit as a redesign.

The short version

Nielsen's November 4 piece turns user expectations into a ledger: conventions you borrow, consistency that banks familiarity capital, violations that levy a surprise tax, and a novelty budget you spend on changes that deliver value. His 37 guidelines end with the AI set, and the last one is to treat model changes as redesigns, with the August 2025 evidence: OpenAI replaced GPT-4o with GPT-5, users revolted, the older model was back within three days, and Altman said he had underestimated what users valued. The ledger is right. The layer is wrong. The user of an agent product never sees the model; what they feel is the harness. Lesson 08 of Becoming Forward Deployed Engineer has a prompt shortened on a Thursday, four lines removed, seven fewer answers out of 42, noticed two weeks later. Vivek Trivedy moved one model about thirteen points by changing only the harness while the two models in Lesson 06's review were three points apart. And the library that governs my own agents has 51 commits since March 30, at least 21 with an agent co-author line and 30 I cannot attribute, none of them tested for two to four weeks and none with a classic path. By Nielsen's own ledger those were redesigns. The Rewrite puts six lines under every harness commit: score, rows, name, undo, the previous version selectable for the test window, and a slice before the fleet. The model swap gets the same six. It is just the smallest redesign you ship.

The ledger Nielsen wrote

The piece is long and most of it is the history of why conventions work: Jakob's Law, that users spend most of their time on other sites; the logo study where moving it from top-left to center raised one-click homepage failures from 4% to 24%; Norman's system image and the thermostat model a third of Americans hold; the principle of least astonishment back to the 1967 PL/I Bulletin; the representativeness heuristic and Fogg's 2,684 participants whose credibility comments were 46% about visual design; signifiers; mere exposure; Loewy's most advanced yet acceptable.

What it adds for AI is the ledger. Familiarity capital accrues through consistency and is spent by redesigns, mostly among loyal users. Each surprise is a tax of hesitation, wrong clicks, recovery, and lost trust, and it compounds across funnels. Innovation draws on a novelty budget, and the two failures are cowardice, cloning the competitor, and shock therapy, changing everything at once. The change-management guidelines follow from that: keep landmarks stable, make small changes, keep classic paths, remove bridge conventions once users have crossed but keep fossil signifiers that still work, stage astonishment with previews and undo, run redesign A/B tests for two to four weeks, and never cite habituation as a defense.

Then the last five guidelines, for AI. Adopt the emerging conventions and re-survey them quarterly. Give the prompt box signifiers, because roughly half the population in rich countries lacks the writing skill to articulate complex intent. Calibrate the AI's costume. Give agents previews, permission tiers, logs, and undo. And treat model changes as redesigns, with the case: GPT-4o replaced by GPT-5 in August 2025, users revolted, the old model back in three days, Altman admitting he had underestimated what users valued.

Where the redesign lives

Take the guideline literally and ask what the user of an agent product has ever seen change. Not the model. The model is a name in a settings page, if that. What changes the product in the user's hands is the harness: the system prompt, the skills that load, the retrieval budget, the tools the agent may call, the stop rule.

Lesson 08 of Becoming Forward Deployed Engineer is the shape, and The Prompt Commit: Score, Rows, Name, Undo Under Every Prompt Change already works through it. An insurer's prompt lives as a triple-quoted string. Someone shortens it on a Thursday to save tokens. The PR shows four lines removed and reads fine. Two weeks later the adjusters say the answers got worse, with nothing connecting the two. The lesson's own number is that one of four wordings answers seven fewer questions out of 42. That is a redesign by every entry in Nielsen's ledger. Familiarity capital spent on the loyal users, the adjusters who had learned what the thing could do. Surprise tax paid over two weeks of hesitation and wrong answers. No preview, no undo, no two-to-four-week test, and the only landmark, the model, untouched.

Which Model Is the Smallest Knob in the Architecture Review has the sizes. Vivek Trivedy at LangChain took one model from 52.8 to 66.5 on Terminal Bench by changing only the harness. The two candidates in Lesson 06's review were three points apart. Thirteen points from the layer nobody treats as a redesign, three from the one Nielsen's guideline names.

My harness, by Nielsen's ledger

The library that governs what my agents do, the rules and skills they load before they act, has 51 commits since March 30. I counted them for 43 Skills, 39 Auto-Activate. Kill the Skill Count. Keep the Four You Type. and the count has not improved. At least 21 carry an agent co-author line. 30 I cannot attribute to a person or an agent. Not one ran for two to four weeks against the version before it. Not one kept the old version selectable. The same post has Dan McAteer's supervisor mod speaking on 99 of his 111 turns before he started skimming it, which is the surprise tax in its purest form: the harness changed how often it interrupted, and the person adapted by reading less.

By the ledger in Nielsen's piece, each of those 51 commits was a redesign. The GPT-4o reversal became a three-day story because the model change had a name, a date, an announcement, and a vendor who could be shouted at. The 30 unattributed commits in my harness had none of that, and they moved the product more than any model swap I have made.

The Rewrite

Treat every harness commit as a redesign. The guideline stays. The layer moves.

Under each commit, six lines. Four are the ones The Prompt Commit already asks for:

Score. The eval result on this exact text.

Rows. The row set it was scored on, count and date, because a classifier I shipped scored 89% on held-out data and 61% in production within six weeks on the same prompt.

Name. Who changed it, person or agent, and who read the diff.

Undo. The version this rolls back to and who can pull it. Undo Is a Design Primitive makes the design case: undo has to cover the thing the system did while you slept, and a harness change is exactly that thing.

Two more come straight from Nielsen's change-management block, applied one layer down:

The classic path. The previous version stays selectable for the two to four weeks his A/B guideline asks for. Not a rollback you can request; a path the user or the operator can take. OpenAI's three days were the classic path restored after the fact. Ship it with the change instead.

Staged astonishment. The change goes to a slice before the fleet. Preview, old default held, one-click undo, in his words. For a harness that means a slice of agents or a slice of users, with the score and the rows reported for the slice before the rest moves.

The model swap gets the same six lines. It is a redesign too. It is just the smallest one you ship, by the measurements above, and the only one that already arrives with a name and a date.

The Operating Cadence(coming Jan 27) says a practice that is not on a calendar with a name next to it is an intention. The design review that covers every screen and none of the harness commits is that intention. Under AI Product Management the claim is that AI collapsed the cost of the parts PMs were trained for and left the parts nobody trained for. Deciding when a change to the harness is a redesign is one of those. Nielsen wrote the test. It only needs to be run on the right layer.

This week: open the commit log of whatever governs your agents' behavior, the prompts, the skills, the rules. Count the commits since the last design review. Count the ones with a name. The difference is the number of redesigns your users absorbed without a test, a classic path, or an undo.

Related answer: Should a change to an agent's harness be treated as a redesign?

Sources: Jakob Nielsen, "Mental Models, Conventions, and Expectations in UX Design: The Expectation Economy," UX Tigers, November 4, 2026, read in full; the logo study, the 2,684 participants, the 46%, and the GPT-4o account are as he reports them. Falk Gottlob, "The Prompt Commit: Score, Rows, Name, Undo Under Every Prompt Change" (October 26, 2026), where Lesson 08's seven of 42 is read in the lesson's public part; "Which Model Is the Smallest Knob in the Architecture Review" (October 19, 2026), where Vivek Trivedy's 52.8 to 66.5 on Terminal Bench is as his LangChain post reports it; "43 Skills, 39 Auto-Activate. Kill the Skill Count. Keep the Four You Type." (November 2, 2026); and the Design Operating Model chapters "Undo Is a Design Primitive" and "The Operating Cadence(coming Jan 27)."