
In May I wrote IDSD Is Spec-Driven Development With a New Acronym. Kill Both. It was a confident post. Part of it has held up. Part of it has not, and I would rather say which part than quietly move on.
The short version
- IDSD (Intent-Driven Software Development) and spec-driven development are both selectively useful, and in May I said to kill both. For small changes I still say skip the document: prompt, diff, eval.
- For large changes I was wrong, and Uvik Software's 50-ticket benchmark shows a short written intent helps once a change crosses a boundary.
- I was right about the ceremony, because the heaviest framework in that data was the slowest and the most expensive and bought no quality over the lightest one.
- The bigger lesson came from building Heidi: intent is discovered by reading real outputs, and the failures that hurt most lived in the runtime, where no document could see them.
- The rule I use now is to size the page to the change, write the eval rows first, read 50 real outputs, and put the gate in the runtime.
What I said in May
Kapil Viren Ahuja proposed Intent-Driven Software Development as the replacement for spec-driven development. I agreed with his diagnosis of SDD and rejected the cure. An intent document, I argued, is a PRD with a new name, and every methodology from waterfall to SAFe arrives as the same package: a vocabulary, a set of templates, a training program, and an invoice.
My advice was to run a two-week parallel test. One pod uses the intent-document workflow, the other uses prototype plus eval, and you compare.
I never ran that test at a scale worth publishing. I am one builder with a fleet of agents, not two pods. Somebody else ran it.
Somebody ran the test
In October, Uvik Software gave the same 50 production Python tickets to five workflows: GitHub Spec Kit, Kiro, BMAD, OpenSpec, and a no-spec control. Same model in every arm, 45 minutes to spec, 45 minutes to build, and a senior engineer reviewing each patch blind. Paul Francis published the results and the raw CSV.
Read it with the caveats they list themselves. One team, one language, one model, one run, and Uvik sells engineering services. Their noise band is ten points either way, so most of the framework-versus-framework rankings mean nothing.
Three results are bigger than the noise, and they do not all point the same way.
Small changes do not need a spec. On bug fixes the control merged 90% of tickets, level with the best spec arms. On one-to-two-file changes it merged 85%, again level. And it got there in a median of 29 minutes, against 36 to 55 for the spec workflows.
Large changes do. On tickets touching six or more files, the control merged half. Every spec arm merged 80%. That is ten tickets per arm, so hold the exact numbers loosely, but a 30-point gap is not noise. Features and data pipelines showed the same pattern.
Reviewers found about half as many defects in the spec arms: 0.46 per merged ticket against 0.86 for the control.
Then there is the result I care about most. BMAD produced the longest specs (a median of 188 lines, 28 minutes to write) and spent close to 44% of its tokens before any code existed. OpenSpec produced the shortest (93 lines, 12 minutes). OpenSpec merged more tickets, cost less per merged ticket, and drifted less. And across every framework, a spec that covered at least six of eight completeness items merged 89% of the time, against 60% for one that did not.
89% vs 60%
What predicted success was whether the page was complete. Which framework produced it barely mattered, and length worked against you.
What I got wrong, and what I got right
Wrong: "kill the document." Once a change crosses files, services, or people, the agent needs the boundary written down, and so does the reviewer. I underweighted the reviewer. A prototype shows you what was built. It does not tell the person reading the diff what was supposed to stay untouched.
Right: the methodology layer is overhead. Nothing in that benchmark rewards the vocabulary, the phases, or the training. It rewards one complete page.
That leaves Kapil and me closer than my headline suggested. His claim was that a full up-front spec is the wrong unit, and that humans should own intent and constraints. The data backs him on that. Where I still disagree is the packaging. The moment the page becomes a method with a name, somebody starts selling the name.
What Heidi taught me that no benchmark could
A benchmark ticket has a known right answer. Product work with a model inside it usually does not, and that is where six months of building Heidi changed my thinking more than the data did.
You do not know your intent until you read the outputs. I wrote Heidi's first eval set from the task description in an afternoon: permission scope, reversibility, receipt logging, budget. Every check passed. Then I watched real runs and found agents taking actions that were permitted, reversible, logged, and in budget, on the strength of one stale data point. Nothing I could have written up front named that failure, because I did not know I cared about it until I saw it. Shreya Shankar calls these bottom-up evals. I wrote that up in The Wince Is the Spec, and it led directly to evidence grading and a three-axis autonomy gate.
This is my real objection to intent-first methods now. They assume intent exists before the work and only needs to be articulated. In my experience half of it gets discovered during the work.
The failures that cost the most were not in any document. Two examples. Slack stopped delivering events to Heidi for 12 days. Slack's own dashboard showed the endpoint as verified the entire time. Separately, a single override setting forced every chat turn onto one model for 11 days, and the admin page built to explain model routing never showed it. Both were written down correctly somewhere, and the running system ignored both.
An intent document would have caught neither. A watchdog that alerts on silence caught the first once I built it. Surfacing the override on the page caught the second.
Prompts and fallbacks are changes nobody specs. I counted the commits in my own agent rule library and could not attribute most of them to a reviewed decision. That became The Prompt Commit: score, rows, name, and undo under every prompt change. Model fallbacks have the same hole, which I covered in the fallback nobody reviewed.
The rule I use now
Four parts. None of it needs a framework.
1. Size the page to the change. Bug fix, refactor, new test, one or two files: no spec. Prompt, diff, eval. If the change touches six or more files, alters a schema or an API, involves more than one contributor, or sits in regulated code, write one page. Uvik's own Spec Fit Test is a fine checklist for this and it is free.
2. Cap the page and check it for completeness. Under 100 lines. It must say what done looks like, what must not change, which boundaries it crosses, and how it will be verified. If you cannot finish it in 15 minutes, you do not understand the change yet, and that is worth knowing before an agent spends your tokens finding out.
3. Write the eval rows before the build, then go read. Five rows that can fail. After the first working version, read 50 real outputs and write down every wince. Convert three of them into yes or no checks. That second pass is where your actual intent shows up, and it goes back into the page.
4. Put the gate in the runtime. Anything the page says must hold needs a check that runs in production and fails loudly: a watchdog on silence, a visible flag on every override, a threshold that travels with every model swap. A requirement that only exists in a document will drift, and the document will not tell you.
If you lead a team
Do three things this month.
Pull your last 20 merged changes and sort them by size. For the large ones, ask whether the reviewer had anything in writing about what should not change. If the answer is mostly no, that is your defect rate.
Ban the framework debate. Pick the lightest tool your engineers will use and measure one thing: is the page complete. The benchmark could not separate the frameworks from each other. It separated complete pages from incomplete ones by almost 30 points.
Then ask the harder question about your product surfaces with a model inside. Who read 50 outputs last month, and what changed because of it? If nobody did, you have a spec for something you have not looked at.
Where this goes
Spec Kit passed 149,000 GitHub stars this month, so the tooling is not going away, and I no longer think it should. But watch the direction. The winning spec in that benchmark was the shortest one, and each new model release takes more of the planning off the page. The written part keeps shrinking toward the things only a human can supply: what matters, what must not break, and what done means.
What does not shrink is the record of what happened when the thing ran. Which outputs made someone wince, which actions got reversed, and which rules survived contact with real traffic. I argued in Context Is King. Written Context Is Rented. that the written layer ports and the earned record does not. The same holds for how you build. Anyone can copy your spec template. Nobody can copy six months of graded outputs.
So in May I told you to kill the document. My advice now is to keep it to one page, write it only when the change is big enough to need it, and spend the hours you save reading what your system produces.
More on this thread in AI Product Management and the handbook chapter The Eval Is the Spec.
Related answer: Did Uvik's 50-ticket benchmark show that spec-driven development works?
Sources:
- Paul Francis, Uvik Software, Spec-Driven Development Benchmark 2026 (run October 15, 2026, updated November 2, 2026; raw data published CC BY 4.0)
- Kapil Viren Ahuja, The Method That Replaces Spec-Driven Development: IDSD, Activated Thinker on Medium (May 2026)
- Shreya Shankar, on top-down versus bottom-up evals
- GitHub Spec Kit; star count as reported by Kunal Ganglani, Spec-Driven Development vs Waterfall for AI Agents (November 10, 2026)
- Falk Gottlob, IDSD Is Spec-Driven Development With a New Acronym. Kill Both. (May 28, 2026)
Frequently asked
What is IDSD (Intent-Driven Software Development)?+
IDSD is a method Kapil Viren Ahuja proposed in May 2026 as a replacement for spec-driven development. Instead of a complete up-front spec, the team writes down intent, expectations, context, and constraints, and lets the agent work out the implementation.
Does spec-driven development work?+
Selectively. In Uvik Software's October 2026 benchmark of 50 production Python tickets, every spec workflow merged 80% of large tickets against 50% for a no-spec control, and review defects were roughly half. On bug fixes, refactors, and test writing the spec did not pay back.
What did you get wrong in the May 2026 IDSD post?+
I said to kill the document. Once a change crosses files, services, or people, the agent and the reviewer both need the boundary written down. What held up was the claim about ceremony: the heaviest framework in the benchmark was the slowest and most expensive, with no quality edge over the lightest.
When should a team write a spec for an AI coding agent?+
When the change crosses a boundary: six or more files, a schema or API change, several contributors, or regulated code. Keep it under 100 lines, check it for completeness, and write the eval rows before the build. Skip it for small, contained changes.
Which spec-driven development framework is best?+
The Uvik benchmark could not separate them: most framework differences fell inside its ten-point noise band. What separated outcomes was completeness. A spec covering at least six of eight checklist items merged 89% of the time, against 60% for one that did not.
Why is a written intent not enough for an AI product?+
Because half the intent is discovered by reading real outputs, and the costliest failures live in the runtime. At Heidi, Slack stopped delivering events for 12 days while its dashboard showed the endpoint as verified, and one override forced every chat turn onto one model for 11 days. A watchdog and a visible flag caught those, not a document.

Sign in with LinkedIn to leave a comment.
Sign in with LinkedIn