AI AgentsNew·Falk Gottlob··8 min read

Every Check Was Green. Four Sources Were Dead.

My source monitor reported 43 of 45 feeds healthy. Four had published nothing for 72 to 612 days. Every one returned HTTP 200 and parsed perfectly.

AI agentsagent reliabilitysilent failuremonitoringhealth checksstalenessagent fleetFalksterobservabilityevals
Helpful?

AI Agents green Falkster cover: a board of identical status lights all glowing the same green, with one wire running down from a light in the middle to a frayed severed end that connects to nothing.

Six of the people I read published something last week explaining how to build an AI agent. Paweł Huryn shipped a multi-tenant SaaS app without writing code. Maja Voje wrote up what she automated in her first month with an agent stack. Lenny Rachitsky ran two pieces on designers building prototypes from their phones. Basia Kubicka walked through Google's ADK end to end. Aakash Gupta took apart an $8.3B company's product stack. Rich Holmes covered the tooling.

All six were right. The floor has dropped and it is not close.

Not one of them wrote about month seven.

The short version

An AI agent takes an afternoon to build and never stops needing maintenance, and almost all published writing covers the afternoon. This week my own source-monitoring agent reported 43 of 45 feeds healthy. Four of those healthy feeds had published nothing for 72, 356, roughly 550, and 612 days. Every one returned HTTP 200, parsed as valid XML, and carried between 10 and 100 well-formed items. The health check I had written asked whether the source was reachable and parseable, which was the easy question. The question that mattered was whether it had said anything lately. That is the characteristic failure of agent systems: not the crash, which gets fixed on Tuesday, but the confident well-formed output from an input that stopped being true. The fix is to record the newest item date per source, compare it to the run date, and threshold it against how often that source normally publishes.

What green meant

I have 47 agent blueprints published on this site, 44 of them tagged to a stage in the fleet. I am not short of experience with the building half.

This week I added another one: a monitor that reads 45 sources and turns the week into post ideas. Straightforward. An afternoon, as advertised.

I wrote a health check for it, because I have been doing this long enough to write a health check. It fetches every source, confirms the response parses, counts the items, and prints a line per source. Forty-three came back ok. Two are behind logins and get handled by hand.

Green. Ship it.

Then I looked at the dates, for an unrelated reason, and found this:

SourceItems in feedNewest itemSilent for
Chip Huyen102025-01-16612 days
Zendesk1002024-03-20~550 days
Airtable152025-09-29356 days
Dan Shipper, Chain of Thought502026-07-1072 days

Every one of those returned HTTP 200. Every one parsed as valid XML. Every one contained a healthy number of well-formed items with titles, links, and dates.

Every one of them was dead, and my monitor called them healthy, because I had written the check that was easy to write.

Is it reachable. Does it parse.

The check I needed was: has it said anything lately.

The failure nobody demos

This is the part that gets skipped in all six of those excellent build posts, and I do not think it is because the authors are hiding it. I think it is because the failure is boring to describe and invisible while it happens.

An agent that crashes is a good agent. It crashes, you get an alert or a blank output, somebody fixes it, total cost one day.

An agent that quietly reads a dead source produces exactly what a working agent produces: a well-formed result, on time, with the right shape. In my case it would have produced a weekly digest that said "Dan Shipper: quiet this week" every week, forever, and every week that sentence would have been false in a way no human would catch, because the sentence is plausible. People do have quiet weeks.

That is the whole problem. The output of a silently failing agent is indistinguishable from the output of a working one, right up until someone checks the underlying world by hand.

And the entire premise of building these things is that nobody checks by hand. That is what you bought.

So the cost is not the failure. It is the failure multiplied by however long it takes someone to notice, and you have specifically engineered the conditions for nobody to notice.

No news and no data are different states

The concrete fix is small. Record the newest item date from every source on every run. Compare it to the run date. Compare that gap to how often the source normally publishes, and flag it when the gap gets absurd.

I run two thresholds. A source that normally publishes weekly, silent for 35 days, gets flagged. A source that publishes rarely, silent for 120 days, gets flagged. The flagged ones show up in the output with a recommendation to demote or drop.

The numbers are not the point and I would not defend them hard. The point is that the default threshold is infinity, and anything finite beats it.

What the fix really does is split one state into two. Most pipelines collapse "this source is alive and had nothing to say" and "this source stopped existing" into the same empty result set. They are not the same. The first is information. The second is a defect. If your system cannot tell them apart, it will report the defect as information every single run, confidently, in a nice table.

There is a second-order version of this that took me longer to see. When the monitor flagged Dan Shipper at 72 days, the obvious read was that he had gone quiet. The likelier read, given that Every publishes constantly, is that I have the wrong feed URL. Those need opposite fixes: one is a roster decision, the other is a one-line correction. So the flag says which it suspects and why, and the fix that ships is the fix for the actual problem rather than the first plausible one.

This is the same shape as the enforcement argument from a different angle. There I was saying you can only gate what is observable at a chokepoint. Here the observable thing is a date, sitting in the payload, that I was not reading. The chokepoint existed. I was walking past it.

Why the build content wins

I want to be fair to the six posts, because "here is the unglamorous part" is an easy pose and usually a cheap one.

Build content dominates for structural reasons, not vanity ones. It is easy to write, because the thing you did has a beginning and an end. It performs, because a reader can do it today. The result is visible the same afternoon, which makes for a good screenshot. Maintenance content has none of that: no clean arc, no artifact, and the honest version is a table of dates showing that you were wrong for two months.

So the public record of this moment is going to be badly skewed toward the first afternoon. Not because anyone is lying. Because one half is publishable and the other half is a Tuesday.

Which means if you are learning agents from what is published, you are learning the half that takes an afternoon and skipping the half that takes forever, and you will find the second half yourself, later, in production, in a way that costs more than reading about it would have.

Pick one thing this week

Take one agent you trust. Not the flakiest one, the one you would defend.

Ask what its freshest input actually is. Not whether the pipe is open. The date on the last real thing that came through it.

If you cannot answer that in under a minute, you do not have monitoring. You have a green light.

And if you are about to build your fourth agent, the useful question is not what to build next. It is what is already quietly wrong in the three you have, how you would find out, and who is looking.

Sources: Feed staleness figures measured against the 45-source roster in this site's own monitoring script on 2026-09-20. The six build posts referenced are from Paweł Huryn, Aakash Gupta, Lenny Rachitsky, Maja Voje, Basia Kubicka on LinkedIn, and Department of Product, all published in the week of 2026-09-14.

This one belongs to the running argument on Enterprise AI Agents: agent count is a vanity metric, and what matters is how many deployed agents still do real work after ninety days. A source that has been dead for 612 days while reporting healthy is one of the ways that number is quietly lower than you think.

Related answer: How do you detect a silently failing AI agent?

Share this post

Also on Medium

Full archive →

Frequently asked

What is silent failure in an AI agent?+

When an agent keeps producing well-formed output from an input that has stopped being valid. Nothing errors, nothing alerts, and the output looks exactly like working output. It is more expensive than a crash because a crash is fixed the same day, while a silent failure runs for as long as it takes a human to notice, and these systems are built so that nobody is watching.

Why do health checks miss dead data sources?+

Because reachability and freshness are different questions and only the first one is easy to check. In my own source monitor, four feeds returned HTTP 200, parsed as valid XML, and contained between 10 and 100 well-formed items. They had published nothing for 72, 356, 612, and roughly 550 days. Every liveness check passed. The data was dead.

How do you detect a stale source in an agent pipeline?+

Record the newest item date from each source on every run, compare it to the run date, and compare that gap to how often the source normally publishes. A weekly source silent for 35 days is a signal. A monthly one silent for 120 days is a signal. The thresholds matter less than having any at all, because the default threshold is infinity.

What is the difference between no news and no data?+

No news means the source is alive and had nothing to say this week, which is information. No data means the source stopped and you are reading a corpse, which is a defect. Most pipelines collapse both into an empty result set, so a broken source and a quiet week produce the same output and the broken one never gets investigated.

How long does it take to build an AI agent versus maintain one?+

An afternoon to build, indefinitely to maintain. That asymmetry is why build content dominates: it is easy to write, it performs, and the result is visible the same day. Maintenance content is unglamorous and nobody shares it, so the public record of this moment is badly skewed toward the first afternoon.

What should you check before adding another agent to a fleet?+

What breaks in the ones you already have, how you would find out, and who is looking. If the answer to the last one is nobody, adding another agent multiplies an unmonitored surface. The useful question is not what to build next, it is what is already quietly wrong.

THE SHORT ANSWER

About the author

Falk Gottlob

Falk Gottlob

Product Executive · Founder, Falkster.AI

Thirty years shipping product, from Microsoft Research and Adobe to Salesforce, where he grew Quip into what became Slack Canvas. Four startups, five exits, including a $6.5B healthcare platform and a company Microsoft bought. Four-time Chief Product Officer. Now founder of Falkster.AI, an agentic AI company run by its own agents. This notebook is written from inside the build, not above it.

Comments (0)

Sign in with LinkedIn to leave a comment.

Sign in with LinkedIn
  • Be the first to comment.

Keep Reading

Posts you might find interesting based on what you just read.