Two Months, Then One Day: OpenAI Published the FDE Scorecard

Colin Jarvis runs forward deployed engineering at OpenAI, and this morning he published what the team does, how it works, and the numbers behind it. I have been writing the FDE Transition for two months, and twice in that series I had to admit that no benchmark existed for the thing I was asking people to measure. This post is the first one from a lab.

So this is a field note, not a rewrite. Most of what Jarvis wrote I would hand to anyone running the seat. Two of his numbers go straight onto the scorecard. One of his cases needs a different reading if you are not a lab, and that is where I will spend the second half.

The short version

Colin Jarvis's "What FDE means at OpenAI" (October 22, 2026) is the first public account from a frontier lab of what its FDE team measures. Around 15 FDEs on a semiconductor customer with 10-plus use cases in mid-2025, everything in production in three to six months, around 48% time saved on directed test generation and a 22% median gain from kernel optimisation; then a rebuild on a common Codex harness that removed roughly 90% of the custom code, after which a deployment that took two months the first time reached a second team in one day. The FDE Scorecard said hours per deployment should fall every quarter and that no benchmark existed. Now one does, and those two numbers, the second team's date and the absorbed share, are what belongs on the page; "making ourselves obsolete every couple of months," Jarvis's phrase, is the feeling they produce. The case to read differently is the telecom voice deployment that scored around 60% on the key evals at scoping and reached 90-plus over six months because Research shipped model snapshots. That is a column only a lab has. For everyone else, 60% at scoping is a no with an eval number and a re-test date, written after one harness probe, because a 60% that survives a harness change is a model gap and a 60% that moves never was.

What he published

Jarvis founded the team in early 2025 to close what he calls the deployment gap: the distance between what models appear capable of and what people can reliably use to do their work. The post opens with a question someone on his team asked when Codex came out: what do we do when Codex can one-shot the use cases we spent months building? His answer is the thesis of the piece. If the team makes itself obsolete every couple of months, it is doing its job.

The first case is a semiconductor customer in mid-2025. Around 15 FDEs, 10 or more use cases, the ambition to make the customer's verification process AI-native: directed test generation, root cause analysis, an architecture assistant. Verification, he notes, can be around 70% of a project's effort in that industry. Some of the engineers had joined OpenAI days earlier and had never worked in semiconductors. They embedded with the verification team, studied chip design in their spare time, and built evals with the customer's engineers. Over three to six months everything reached production: around 48% time saved on directed test generation, a 22% median performance improvement through kernel optimisation, weeks of debugging compressed into hours. The Chief Hardware Officer asked whether they had been chip designers before.

Then the second half of the same case, which is the part the FDE Transition has been circling. Each workflow worked, and each still needed too much custom integration to extend to another team. Codex had improved. One FDE rebuilt their application around it, and the team moved every use case onto a common harness with Codex at the core. Custom applications became plugins and skills on common primitives. The separate terminal UIs and CLIs written for individual use cases were retired. Roughly 90% of the custom code per project went away. And the sentence I will be quoting for a while: a use case in August 2026 took two months for the initial deployment and one day for the second team.

The third case is a telecommunications customer, and I will come back to it.

Two lines for the scorecard

The FDE Scorecard: Count What Stayed, Not Who Was Sent runs four numbers, each as a trend against your own last quarter, and it says in so many words that it does that because no benchmark exists. The fourth number is hours per deployment for the same class of customer, which should fall every quarter. The third is generalization rate, the share of what came back from the field that shipped in the product.

Jarvis has just published both, from the one place that could.

Time from first deployment to second team: two months, then one day. That is hours per deployment collapsing in a single line, and it is the number I would ask any FDE leader to produce for their own best use case. Not the average. The best one, with the dates. If you cannot write that sentence for anything you have shipped, the loop is not running.

Share of custom code the product absorbed: about 90%. That is generalization rate measured from the code rather than from the packets, and it is a better measurement than mine because it is a diff rather than a count of decisions. The 90% is what made the one day possible. Extending to the next team, in his telling, was swapping connectors and MCP servers, building fresh evals with a skill they had for that, and starting.

Jarvis calls the result making themselves obsolete. I would not put a feeling on the scorecard. I would put the second team's date and the absorbed share, and let obsolescence be the consequence. Partly because feelings do not trend, and partly because his own sequence shows what the feeling is made of: production first, then the product improves, then the custom implementation is replaced and the value is kept, then the engineers go find the harder problem. Each of those steps has a number. Write the numbers.

The column only a lab has

Now the telecom case. The team had built a customer-service platform with a large telecommunications customer, and the customer wanted voice, which was around 90% of its service volume and more expensive to contain than chat. At scoping the model scored around 60% on the key evals used to decide whether it could do the job in production. Function calling and instruction following in customer-service scenarios were not there yet.

What OpenAI did next is the interesting part. They told the customer. They brought Research into the work. FDE built the platform and defined what the model needed to do, Research closed the capability gaps, and successive model snapshots were tested against the customer's requirements. Over six months the key evals went from around 60% to 80% and then to 90-plus. The resulting voice model now supports that deployment, other custom deployments, and managed deployments through their product.

Jarvis frames this as the second version of making themselves obsolete: turning a problem today's models cannot solve into a capability future models deliver. It is also the most honest paragraph in the post, because it names the one advantage the lab's FDE team has that nobody else's does. A column called Research.

In the Room, Not on the Quota: Who Owns the FDE Promise attaches a one-page handshake to every FDE contract. Every outcome in the proposal, sorted by the FDE into three columns: ships, builds, or no. Sales owns the number, the FDE owns the columns, and both names go on the page. Jarvis has a fourth column. A 60% eval at scoping goes into it, with the customer committed to working through it and Research on the other side. For his team that is the right call, and the six-month result says so.

If you are not a lab, that column does not exist, and pretending it does is how a pilot becomes an eighteen-month services engagement. A 60% eval at scoping is a no. Not a soft no, not a "builds" with an open-ended date. A no with two things attached: the eval number, and the date you re-test when the next model lands. Written on the handshake, with the customer's name on the rows.

One test before the no

There is a step before writing it, and it is the one most teams skip because they have already decided the model is the problem.

Which Model Is the Smallest Knob in the Architecture Review argues that an architecture review that asks which model is asking the smallest question in the room, because a harness change moves the same evals further than a model swap usually does. The probe is cheap: hold the model, change the harness, the tool set, the context assembly, the retries, and run the customer's rows again.

So before the 60% goes on the handshake as a no, run that. A 60% that survives a harness change is a model gap, and if you have no Research column it is a no. A 60% that moves to 75 under a different harness was never a model gap, and it belongs in builds with a date. Jarvis's team did the equivalent inside the lab; the whole semiconductor rebuild is a harness story with the model held roughly constant. The difference is that when the harness probe came back flat on the telecom case, they had somewhere to send it.

Under AI Product Management the argument is that the build got cheap and the scarce work is knowing which part to build. Jarvis's post is that argument from inside the deployment gap: the engineers were never the constraint, the product's ability to absorb what they learned was, and the day it could, two months became one day. The scorecard question for everyone else is whether you can write that sentence, and the handshake question is what you do with a 60% when there is no fourth column to put it in.

This is a standalone post alongside The FDE Transition.

This week: find your best deployment and write the two numbers next to it, the date the second team went live and the share of custom code the product took back. If either line is blank, that is the scorecard. Then open your last three pilot scopes and find the eval number at scoping. If any of them was in the sixties and went ahead without a re-test date, you have found the services engagement.

Related answer: What should an FDE team do when the model scores 60% at scoping?

Sources: Colin Jarvis, "What FDE means at OpenAI," OpenAI, October 22, 2026, read in full. Every figure attributed to OpenAI (the 15 FDEs, the 10-plus use cases, the three to six months, the 48%, the 22%, the roughly 90% of custom code, the two months and one day, the 60 to 80 to 90-plus on the telecom evals, the 7,000 tax returns and about 50% throughput) is as Jarvis reports it; I did not open the John Deere, Thrive Holdings, or Presence material he links. Falk Gottlob, "The FDE Scorecard: Count What Stayed, Not Who Was Sent" (October 8, 2026), "In the Room, Not on the Quota: Who Owns the FDE Promise" (September 29, 2026), and "Which Model Is the Smallest Knob in the Architecture Review" (October 19, 2026).