Last quarter I did something that felt slightly absurd. I gave my AI agents performance reviews. Same structure I would use for a person: a scorecard, a look at the trend, and a verdict of keep, coach, or fire. Three got fired. I am telling you this because the way the industry talks about agents is dishonest. They are sold as magic that either works or does not, and if it does not you tweak the prompt until the demo looks good again. That is not how anything doing real work gets managed, and it is why so many teams have a fleet quietly making them slower.
An agent is a worker, not a feature
An agent that drafts your release notes or triages your tickets is not a feature you shipped. It's a worker you hired. It has a job, a track record, and a cost, and the cost is not the API bill. It's the human time spent checking its work plus the damage when a check gets missed. Once you see agents as headcount, the management questions write themselves. Is this one good at its job? Is it getting better or worse? Would I be faster without it?
The four-metric scorecard
I grade every agent on precision, escalation rate, time-to-output, and trust trend. Escalation rate is the one nobody tracks and the one that matters most. An agent that never escalates isn't confident, it's dangerous. It's making calls it should be handing up. I'd rather have an agent that escalates too much than one that has never once said "I'm not sure."
Why I fired three
The three I shut down didn't fail because they never worked. They failed on the metric that gets people hurt: they were confidently wrong in ways I couldn't detect at review time. One was a competitive-intel summarizer that produced beautiful, structured briefs with occasional invented details I only caught because I happened to know the source. The output quality is exactly what made it dangerous. A worse-looking agent would have made me check. This one earned a trust it hadn't actually built. Fired.
Run the review on your fleet this quarter. Grade every agent on the four metrics, and look hardest at escalation rate, because that's where the dangerous ones hide. I'll bet you have at least one agent you've kept alive out of sunk cost that you'd never re-hire today. That's the one to fire first.