Latency Is Interface

The short version

AI latency is an interface problem, not a spinner problem, because the user cannot tell whether anything is happening, whether it will finish, or whether they are still needed. Six duration bands, and each one requires a different answer: under 100ms needs nothing, a second needs stillness, ten seconds needs the work named in place, two minutes needs honest progress and a cancel, hours needs a job with a lifecycle, and open-ended agent work needs legibility instead of a loading state. Pick the band by the realistic slow case, never the median, because the median is fine by definition and the tail is where people decide the product is broken. The copy for each band is a design deliverable, and "Loading" means nobody has decided yet.

Waiting used to be a solved problem with a solved answer. The work was deterministic, the duration was roughly known, and a spinner covered the gap between a click and a response that was going to arrive in about the same time it arrived last time.

Generated output broke all three of those. Duration varies by a factor that would have been logged as a bug in any previous product. Same request, two seconds on Tuesday and forty on Thursday. And the person waiting has no way to tell a system that is working hard from one that is stuck from one that finished and failed to say so.

So a rendering detail becomes the thing that decides whether the product feels reliable. People will happily wait for something consistently slow and abandon something usually fast that occasionally is not.

Design for variance, not just duration.

The band comes from the slow case

Most of the damage in this area is done before any state gets designed. Somebody asks how long it takes, somebody answers about three seconds, and everything downstream gets built for three seconds.

Three seconds is the median. And the median is the case where nothing needed designing.

What you want is the realistic slow case, roughly the slowest response a normal user hits in a normal week, and engineering can give it to you in one query. Ask for it before the design review, not after. In my experience the number comes back one or two bands higher than anyone on the design side assumed, and a surface designed two bands away from its actual behavior is exactly what people mean when they say a product feels unreliable.

The six bands

Under 100ms feels instant. No state needed, and adding a loading indicator here makes the product feel slower rather than more responsive.

From 100ms to a second, the wait is perceptible but not worth explaining. Keep the interface still. Do not move layout, do not flash a spinner in and out, do not disable the control the person just used. Most of the damage in this band is self-inflicted, because a loading state that appears and vanishes reads as a flicker and flicker reads as a bug.

From one to ten seconds, the user is watching, and a generic spinner starts costing you. It says something is happening and nothing else, and by second six they want to know what. Tie the indicator to the thing they asked for, in place rather than in a corner. Name the work in text. Keep the rest of the interface usable, because blocking the whole screen for eight seconds converts a wait into an outage.

And if you can stream the output, stream it. Streaming turns a ten-second wait into a two-second wait plus reading time, which is a different experience rather than a faster version of the same one.

From ten seconds to two minutes, they will look away. Design for the return.

Progress with real substance means steps completed or what is being worked on now, never an invented percentage. Give them a cancel and say what cancelling loses. Make the result survive navigation, and tell them when it lands if they are elsewhere in the product.

Past two minutes it is a job, and you should stop designing a loading state altogether.

A job has a lifecycle. The user can close the product and lose nothing. It is listed somewhere with its status, findable without a link somebody sent them. Completion arrives through a channel they actually watch, and failure arrives the same way with the same prominence, because silent failure on a long job is the fastest way to lose someone's trust in the entire feature. Show partial results if partial results are useful.

Then there is the sixth band, which is barely latency at all. An agent working across sessions, overnight, or indefinitely. Legibility is the design problem: one sentence answering what it is doing right now, a running account of what it has already done, a summary waiting at the start of the next session before anything else competes for attention, and a stop control that is honest about what stopping leaves half finished.

That running account is the same surface as the one in Undo Is a Design Primitive. Build it once. A product that has a good list of what the agent did already has most of its long-running latency design finished, which is a rare case of two hard problems collapsing into one.

Four rules that hold everywhere

Never lie about progress. A bar that moves smoothly and then parks at ninety percent teaches people that the interface is decorative, and that lesson generalizes to everything else you show them, including the things that are true.

Name the work, not the mechanism. "Checking the last thirty days of usage" is information. "Querying warehouse" is engineering's mental model wearing a user's clothes.

A failed wait needs more design than a successful one. After twenty seconds, "something went wrong" is close to an insult. Say what failed, what survived, and what to do now. Most teams write the happy path copy in the design file and let the failure copy get written by whoever catches the exception.

And slow is survivable, unpredictable is not. Say that in front of an engineering team that wants to optimize the median, because optimizing the median while leaving the tail alone can make the product feel worse.

Which brings up the thing that keeps getting left off the list. Every band above one second needs words. Words are the design output for this surface, more than the layout is.

The template has a table with a column for whether the copy is written, and that column is not a formality. Go look at your product. If a generated surface still says "Loading" then a decision has not been made, it has been deferred into a default, and defaults are where products lose their voice one screen at a time.

This is also the cheapest place to apply the range thinking from You Cannot Mock a Distribution. A duration is one sample. What you actually specify is the best case, the slow case you would still ship, and the point at which the wait becomes a failure that needs its own state.

This week: get the realistic slow case for your three most-used generated surfaces from engineering. Put each one in a band. If any lands two bands away from the state you actually built, you have found the reason people call the product flaky.

The six bands, the required elements for each, the fill-in table, and the four rules are in Latency States by Duration Band. The short version to hand an engineer is How do you design for AI latency and long waits?

---

Chapter 9 of a series on the design operating model. Next: how a product says no in a way that reads as competence rather than limitation.