Last quarter I did something that felt slightly absurd. I gave my AI agents performance reviews. Same structure I would use for a person: a scorecard, a look at the trend, and a verdict of keep, coach, or fire. Three got fired. I am telling you this because the way the industry talks about agents is dishonest. They are sold as magic that either works or does not, and if it does not you tweak the prompt until the demo looks good again. That is not how anything doing real work gets managed, and it is why so many teams have a fleet quietly making them slower.
The reframe: an agent is a worker, not a feature
An agent that drafts your release notes or triages your tickets is not a feature you shipped. It is a worker you hired. It has a job, a track record, and a cost, and the cost is not the API bill. It is the human time spent checking its work plus the damage when a check gets missed. Once you see agents as headcount, the management questions write themselves. Is this one good at its job? Is it getting better or worse? Would I be faster without it?
The four-metric scorecard
I grade every agent on precision, escalation rate, time-to-output, and trust trend. Escalation rate is the one nobody tracks and the one that matters most. An agent that never escalates is not confident, it is dangerous, because it is making calls it should hand up. I would rather have an agent that escalates too much than one that has never once said "I am not sure."
Why I fired three
The three I shut down did not fail because they never worked. They failed on the metric that gets people hurt: they were confidently wrong in ways I could not detect at review time. One was a competitive-intel summarizer that produced beautiful, structured briefs with occasional invented details I only caught because I happened to know the source. The output quality is exactly what made it dangerous. A worse-looking agent would have made me check. This one earned a trust it had not actually built. Fired.
Run the review on your fleet this quarter. Grade every agent on the four metrics, and look hardest at escalation rate, because that is where the dangerous ones hide. I will bet you have at least one agent you have kept alive out of sunk cost that you would never re-hire today. That is the one to fire first.