What is a gate ledger?

A table with one row per human gate in an agent system: approval prompts, review queues, exception readers, and undo windows. Each row has the action class, asks per week per person, approval rate, median seconds to approve, and the date of the last refusal. Undo rows add the window and the typical gap between that user's sessions.

When is an approval gate just theater?

Nielsen's rule of thumb is a median decision time under 2 seconds or an approval rate above 95%. I add the last-no date: a gate that nobody has refused in a quarter is either guarding nothing or being waved through.

Should agents use undo and skip approval?

For reversible actions, undo is usually better, with two conditions. The notification has to name the change and carry the undo control, and the window has to outlast the gap between the user's sessions. Irreversible external actions such as payments, outbound messages, permanent deletions, and permission grants keep a person in session.

Is there an accepted industry solution?

No. Kathy Baxter of Salesforce wrote on October 7, 2026, summarizing workshops with 21 organizations, that the industry currently lacks a definitive solution to the tension between human oversight and the autonomy agents are deployed for. A ledger does not resolve that tension. It shows which gates still have a working human in them.

Does Human in the Loop Make an AI Agent Safe?

Does keeping a human in the loop make an AI agent safe?

THE SHORT ANSWER

Not as a sentence in a safety case. Jakob Nielsen's 2026 review of vigilance research shows two failures. A person watching for rare events detects 10 to 15% fewer within 30 minutes. A person answering approval prompts habituates, so the request that deserved a no is approved with the rest; in a study by Ting Yan that he reports, users who wrote their own ask-me-first rules approved 67% of the resulting prompts and blocked less overreach than users who judged every action by hand. A human gate works when it is reserved for irreversible, external actions, shows the context for the decision, and is measured. Keep a gate ledger: one row per place a human is supposed to catch something, with asks per week, approval rate, median seconds to approve, and the date of the last no. Nielsen's thresholds for theater are an approval rate above 95% or a median decision under 2 seconds. For gates replaced by undo, record the undo window next to the gap between the user's sessions, because a window that closes while the user is away protects nothing.

The phrase promises a person who catches the bad action. The research Jakob Nielsen collects says that person stops catching things on a schedule: within half an hour when watching, and within a few dozen routine approvals when answering prompts. Neither failure is about effort, and telling people to pay attention has never fixed either.

What works is fewer, better gates and a count of what each one does. Reserve the interrupt for actions that can't be taken back and that leave the account. Give reversible actions an undo with a window the user will be present for. Then measure every gate, because the only evidence that a human is in the loop is a refusal.

The argument, the ledger columns, and where undo stops helping are in Kill 'Human in the Loop' as a Control. Keep a Gate Ledger. The design of the undo itself is in Undo Is a Design Primitive.

Last reviewed 2026-10-09 · 1 min read

Does keeping a human in the loop make an AI agent safe?

SOURCES

THE LONG VERSION

RELATED ANSWERS