I have built 39 agent blueprints and shipped most of them. The toolkit makes it look like a clean track record. It is not. For every agent that worked, there is one or two that did not ship, broke in production, or got quietly turned off after a month. Here are the patterns from ten specific autopsies, because the AI-native product conversation is too bullish and I contribute to that when I only publish the wins.
Pattern one: distribution shift
Five of the ten failed because training data, user populations, or source documents did not match production. The customer sentiment classifier scored 89% on a held-out set and 61% in production within six weeks, because the held-out set was English-heavy and non-English customers were systematically scored as angry regardless of content. The internal docs RAG confidently answered from 2021 wiki pages sitting next to 2024 ones, giving both equal weight, and two HR incidents traced back to it. Source quality matters more than model quality, and every RAG system is primarily a data hygiene project. Evals against the real distribution would have caught most of these early.
Pattern two: correctness that was beside the point
Three of the ten failed because the agent did exactly what it was asked and the asking was wrong. The PR-drafting agent produced fine drafts that all sounded the same, until competitors started referencing the generic AI-drafted tone of our announcements. Some categories of output are valuable because they are distinctive, and agents produce median-quality output efficiently, which is bad when your brand depends on above-median. The onboarding walkthrough agent was technically correct about where users got stuck, but users perceived it as surveillance and activation dropped below the control group. Technical correctness is not sufficient. Ask what behavior the agent's presence causes.
Pattern three: cost-compounding autonomy
Two of the ten failed because an agent got authority to act with no human gate, creating downside far bigger than the upside. The auto-approve expense agent approved anything under $500 with matching receipts, until an engineer figured out that "dinner with customer" and a vague receipt passed every time. It approved about $2,400 that should not have been, and finance turned it off. The autonomous pricing tester found a local maximum and optimized toward it with compounding effect, anchoring on a price 14% below benchmark. Recovery took two quarters and cost about $180,000 in annual recurring revenue. Autonomous optimization in a domain with long feedback loops and anchoring effects needs a human approval gate at every iteration.
The pre-flight checklist
The common thread across all ten is that agents got authority before the eval system was load-bearing. Before any agent ship, ask three questions. Am I evaluating on the real production distribution, not the training one? What is the social, emotional, or brand cost of this agent being technically correct but culturally wrong? Where are the human gates, and what would compound if they were not there? Run those three against the next agent on your roadmap before it ships.