How do you measure an AI support agent's resolution rate?

THE SHORT ANSWER

Three ways, on the same conversations. In the benchmark Gorgias runs, written up by Jason Lemkin at SaaStr on October 6, 2026, the top five of 13 vendors resolve 64% to 75% and the median resolves 48%, counting only zero-touch conversations. Vendor reporting is looser: per Lemkin, Gorgias's own counts an interaction once 72 hours pass without a human agent helping, so a customer who gave up counts. I priced per resolved ticket with a 48-hour definition that held for 98% of cases; the other 2% reopened at 60 to 72 hours and became more than 200 contested cases a week. So take 30 conversations your agent closed last month and score each for zero touch, for the invoice definition, and for no related follow-up in seven days. Report all three rates.

Score the same conversations three ways. One rate hides too much.

Jason Lemkin's write-up of the benchmark Gorgias runs has the headline: 13 vendors, 212 live ecommerce stores, the top five resolving 64% to 75%, the median 48%. The benchmark counts a conversation only when the AI handled it with zero human touch and no push out of the channel.

Vendor dashboards use looser rules. Per Lemkin, Gorgias's own customer-facing reporting counts an interaction as automated once the AI resolves it and 72 hours pass without a human agent helping. A customer who gave up counts there. And resolved is the billing unit, at $0.90 a conversation.

I priced that unit once. Resolved meant no escalation within 48 hours, and it held for 98% of cases. The other 2% came back at 60 to 72 hours with a related follow-up, which at volume was more than 200 contested cases a week. We rewrote the definition at month 20 to no follow-up in 7 days. It cost some revenue and cut disputes by 60%.

So the measurement is a sheet with three columns. Pull 30 conversations your agent closed last month, chosen by you and weighted to your real ticket mix. Column one: zero touch. Column two: billable under your contract's definition. Column three: no related follow-up from that customer within seven days, on any channel. Report three rates side by side.

The gap between the first two is quiet you are counting as success. The gap between the last two is what you paid for that didn't stay resolved.

Run it again next month. In the benchmark, nine of ten vendors lost automation in four weeks.

The full piece is One Support Agent, Three Resolution Rates.

SOURCES

THE LONG VERSION

RELATED ANSWERS

Last reviewed 2026-10-07 · 2 min read