Sierra's Voice Agents Fix a Third of the Details They Catch. Read Back the Diff.

Soham Ray at Sierra published a benchmark on October 22 for the part of a voice call that never makes the demo: getting a name, a phone number, or an email address right. The numbers are worse than the demos suggest, and the worst one is not the capture rate. It is what happens when the agent goes back to check.

The short version

Sierra's τ-Entity benchmark, as Soham Ray describes it, runs 200 tasks across ten entity types with a cooperative caller who spells on request, and scores Pass³, the same task passed under regular, noise-heavy, and speech-heavy audio. The leading agent captured the value on 60.3% of regular calls pooled across six languages and 34.0% across all three conditions; English alone was 74.0% and 47.5%. Of the wrong values the agents went back to verify, only 24% to 37% were corrected; the rest repeated the mistake or swapped in a different wrong value. Three fields at 75% each became a 53% call. The same week Jakob Nielsen wrote that satisfaction is performance minus the promise, and Databricks published a 90-day voice pilot recipe that records cost per call, handle time, and satisfaction before the pilot starts. None of the three says what the read-back should sound like or which number opens the pilot. The rewrite: "verify the field" becomes "read back the diff, and score the call." The repair is spoken as old value, new value, and which one is on file, which is the design handbook's never-silently-replace rule applied to audio. The pilot opens with Pass³ per call across every field the call needs, before cost per call. And the copy promises what the agent does nine times in ten, which at 53% is "spell it and we read it back."

What Sierra measured

The setup is deliberately easy. One missing value per call. A caller who cooperates and spells accurately when asked. Most calls under two minutes. Success means the stored value exactly matches the expected one.

Even there, the agents struggle. GPT Live 1 leads the pooled leaderboard at 60.3% on regular calls and 34.0% on Pass³. Gemini 3.8 Live and Grok Voice Think Fast 2.0 reach 43.8% and 44.2% on regular calls and fall to 19.5% and 24.0% across all three conditions. English is kinder: 74.0% and 47.5% for the leader. Korean Pass³ runs 3.5% to 8.5% and Mandarin 12.5% to 24.5%.

Then the part about knowing when to slow down. Sierra measured verification effort as read-backs and spelling requests, and compared it across easy and hard values, familiar and unfamiliar entities, clean and noisy audio, and different caller voices. The agents verified hard and unfamiliar values more carefully. Only Grok verified more under noise. None spent more effort on the caller voice it struggled with most.

Prescribing the sequence helps. Ask, have the caller spell or read it back, confirm and correct, submit. That lifts Pass³ by 14 to 31 points depending on the configuration, at a cost of 21 to 28 seconds per call.

And then the number this post is about. Of the initially wrong captures the agents did go back and verify, only 24% to 37% ended up corrected. Ray's own words: sometimes the agent repeats the same mistake, other times it replaces the original value with a new incorrect one. Asking a follow-up question is only useful if the agent can incorporate the answer.

Fields compound. In the multi-field study, three-field calls succeeded 53% of the time while individual fields stayed 75% accurate. Validating each field and allowing retries moved one setup from 64% to 82%, and Sierra is careful to say it cannot separate the two interventions.

What the other two posts leave out

Nielsen's section on October 26 is about expectations. Satisfaction tracks performance relative to what was promised more than raw quality; exceed the promise and the extra reads as a bonus, fall short and the gap reads as a swindle. Every marketing claim is a loan against the product's future performance. Calibrate the promise to what the product delivers nine times out of ten, and when performance varies, say so in the copy, "usually within a minute."

Databricks' voice post on October 28, by Rohit Bhagwat, Usman Zubair, and Sathish Gangichetty, is a recipe. The agent "confirms who they are by checking your systems rather than taking their word for it." Start with a 90-day pilot on one channel, one team, and your highest-volume routine call. Before it begins, record cost per call, handle time, and customer satisfaction, so you can compare at the end.

Both are right as far as they go. Neither says what the agent should say when it catches itself, and neither says which number the pilot opens with. Sierra's data is the reason those two gaps matter: an agent that re-asks and then swaps one wrong value for another will pass the Databricks pilot on handle time and fail the caller on the thing the call was for.

The rewrite, part one: the repair is a diff you can hear

Recovery and Trust Repair has one rule for a product that was confidently wrong, and it is the rule most teams break by instinct. Never quietly replace the wrong value with the right one. Show what changed, not just the new state. A user who catches a silent edit now knows the product rewrites history, and there is no coming back from that piece of knowledge.

At Commure we added a quiet way to flag a bad output and braced for a trickle. It exposed a whole class of error we did not know existed. The one bug we set out to fix was the tip. The silent fix is how a team chooses not to know, and the choice is made in a design review where nobody frames it that way.

In voice, the rule becomes a script. Old value, new value, which one is on file.

"I had L-E-E. You said L-E-I-G-H. I now have LEIGH. Is that right?"

That is the only shape in which "replaced it with a new incorrect one" is audible to the caller before submit. A read-back that only says the new value sounds identical whether the agent fixed the error or moved it. A read-back that says both lets the caller hear the swap, and lets the transcript show it, which is how the pilot team finds out which 63% to 76% of repairs failed.

The script costs seconds, and Sierra priced those seconds at 21 to 28 per call for the full prescribed sequence. Spend them on the fields that gate the call, the ones that authenticate the caller or route the ticket, and skip them on the rest. The Shape of Uncertainty is the chapter on saying "I am not sure" without sounding broken; a spelled read-back is that move in audio.

The rewrite, part two: score the call

The pilot's opening number is Pass³ per call across every field the call needs. Not accuracy per field.

Sierra's 75% and 53% describe the same agent. Only one of them is what the caller experienced. A pilot that reports per-field accuracy beside cost per call is reporting the number that flatters the agent, and the renewal conversation six months later will be about the other one.

One Support Agent, Three Resolution Rates is the same argument for text. I priced a support agent per resolved ticket on a 48-hour definition that held for 98% of cases, and the other 2% became more than 200 contested cases a week. The definition decides the number. For voice, the definition is the call: every field the workflow needs, captured correctly, under the audio conditions your callers actually have. Write that definition before the Databricks baseline is recorded, score the first hundred pilot calls against it by hand, and put the result on the same line as cost per call.

Then take Nielsen's loan seriously. At 53% on a three-field call the promise cannot be "we will verify you." It has to be the thing the product does nine times in ten, which is "spell it, and we read it back." That sentence is not a hedge. It is the promise the agent can keep, and the read-back diff is how it keeps it.

Under Enterprise AI Agents the claim is that what matters is how many deployed agents still complete production work after ninety days, and what each outcome costs. A voice agent that cannot repair a surname is not completing the work; it is handing the caller to a person with a wrong value already on file.

This week: take ten recorded calls where your voice agent corrected a field. Listen for whether the caller could hear the old value and the new one. Count how many of the ten corrections were right. If you cannot tell from the audio, the script is the first fix, and the count is the pilot's first number.

Related answer: How should a voice agent read back a corrected name or number?

Sources: Soham Ray, "τ-Entity: Can voice agents get the details right?," Sierra blog, October 22, 2026, read in full; the τ-Elicitation paper it summarizes was not opened, and every figure is as Ray reports it. Jakob Nielsen, "UX Roundup: Flow | AI & Creative Jobs | AI Price | Too Many Options | Joint Cognitive Systems | Proactive AI | Satisfaction | Carousels | Activity Feeds | Pagination | Robotaxi Safety | Integration," Jakob Nielsen on UX, October 26, 2026, the Satisfaction section read in full. Rohit Bhagwat, Usman Zubair, and Sathish Gangichetty, "Voice Agents are here. Context turns them into real customer value.," Databricks blog, October 28, 2026, read in full. The Commure flag is in Recovery and Trust Repair; the 98% definition and the 200 contested cases are in One Support Agent, Three Resolution Rates.