One Interface, Several Providers Is the Easy Half. The Eval Threshold Travels.

Lesson 07 of Becoming Forward Deployed Engineer landed yesterday with the best one-line brief for the FDE seat I have read in a while: the customer will change model providers halfway through the engagement, and the question is how much of your code finds out.

The lesson is paid past its preview, so I am working from the two sections anyone can read, the problem and the settings. They are enough to argue with, and the argument is not with the code. The code is right. It is with what the lesson treats as the interface's job.

The short version

Lesson 07 of Becoming Forward Deployed Engineer (October 22, 2026) opens on an insurer whose CISO will not let claim data leave the network while its VP of Claims wants hosted credits spent on hard questions, and its visible fix normalises exactly two settings across providers, a token limit and a temperature, passing everything else through by native name. That is the right amount of abstraction and the easy half. Which provider answers which call is not an interface property. It is a routing-table row per surface, with the model, the fallback, the eval threshold that triggers a swap, and the date it was last run. The same week, LangSmith's LLM Gateway shipped fallback rules across models and hosts enforced for every application, and Kevin Stumpf at Databricks described cutting frontier access at 70% of a team's budget and routing the rest to open-source. Neither attaches an eval to the switch. Three points separated the models in Lesson 6's review, a harness moved one model 13.7 points at LangChain, and a harness doubled the bill in HarnessTax, so a fallback is a provider switch nobody reviewed. The rewrite: one interface, several providers, and one eval threshold per surface, run before a candidate enters the fallback list and again the night the fallback fires.

What the lesson says, in the part you can read

The insurer is a good case because both stakeholders are reasonable. The CISO's rule is that claim data stays on the network, so the component that answers questions about a claim runs on a local model. The VP of Claims has hosted credits and wants them spent on the questions the local model gets wrong. Neither requirement is going away, and the lesson is clear that neither person is wrong.

Then the two failure modes. Write the code for one provider and the switch costs a week. Write a layer that smooths over every difference between providers and you get a client that can only do what the weakest provider does.

The fix in the settings section is the kind of thing I would hand to any engineer on the seat. The shared interface normalises two things, the token limit and the temperature. That list is fixed as a tuple in the code so it cannot grow quietly. Everything else passes through as an options mapping under its native name, the local model's context and prediction limits on one side, the hosted model's seed and logit bias on the other. And a test asserts that the two native sets do not intersect, which is the lesson's way of saying that a single combined settings class would silently drop most of its fields on any given call.

Two settings travel. Everything else stays local to the provider. Good.

The half the interface cannot hold

Here is the question the preview sets up and does not answer: when a claim question comes in, which provider answers it?

Not the interface. The interface makes the call the same way on either side. The decision about which side is a routing decision, and it belongs to the surface, not the client. The CISO's constraint is one row: claim-bearing surfaces, local model, no hosted fallback. The VP's wish is another row: the hard-question surface, hosted model, local fallback, and a threshold that says when "hard" has been reached. Two rows. One client.

I keep that table per surface, and Gross Margin Is Your Job Now has the version I use: for each surface, what model, what fallback, what eval threshold triggers a swap. I review it quarterly. About once a quarter I find a surface routed to the flagship for no reason other than that it shipped that way. The routing table is where the FDE's customer layer meets product, and The FDE Codebase Contract is where that layer is defined.

So far this is a disagreement about where a decision lives. The reason it matters this week is that two platforms moved the decision to runtime.

The switch moved to runtime, without the eval

Martha Janicki and Ramon Petgrave's post on the LangSmith LLM Gateway describes a governance layer between agents and the models they call. Among its controls: define fallback rules across models and hosts, then enforce them for all applications, so when a provider is down, rate-limited, or unavailable, agents switch to another model without fallback logic in each application. Spend caps sit at four levels, organization, workspace, API key, and user, and an agent that hits one gets a 402.

Kevin Stumpf at Databricks gives the budget-driven version in his interview on AI cost management. His example: cut off default access to frontier models once a team has burned through 70% of its budget, and route the rest of their requests to open-source models. His estimate is that roughly 5 to 10% of tasks need a frontier model. He also names harness routing as the next layer beyond model routing, because the same model through a different harness can produce different costs.

Both are real improvements over fallback logic copied into every application. And neither post says what the fallback is allowed to answer with, or how anyone finds out whether it did.

That is the gap. The gateway knows the provider was rate-limited. It does not know whether the model it switched to passes the surface's eval. The budget rule knows the team is at 70%. It does not know which of that team's surfaces are in the 5 to 10% that needed the frontier model.

How far a switch can move the result

The numbers are already on this site, and they are not small.

Which Model Is the Smallest Knob in the Architecture Review collected them: the two candidates in Lesson 6's architecture review sat three points apart on a leaderboard that measured a different task; a harness change at LangChain moved one model 13.7 points on Terminal Bench 2.0; UC Berkeley's HarnessTax found the same model within about a point across three harnesses, at twice the cost in one of them.

A fallback changes the model, usually the host, and often the harness, all at once, at the moment the primary is unavailable. Three points from the model is the smallest of those movements. Nobody reviewed the other two.

I have run the loop without the check. The Stop Rule tells the story: an autonomous pricing agent with no stop rule drifted 14% under benchmark in five weeks, and it took two quarters to recover, because nothing re-ran the eval after the agent started making its own calls. A fallback that fires at 3 a.m. and answers until the primary comes back is that drift with a start time.

The rewrite

One interface, several providers. Keep the lesson's tuple. Two settings travel and the rest stay native.

And one eval threshold per surface, which is the thing that actually travels across providers, because it is the only part of the contract written in the customer's terms rather than the provider's.

Two runs. A candidate model runs the surface's sliced eval before it enters the fallback list, the same 200-row eval with named slices from The Eval That Caught What the Demo Did Not, because a fallback that passes on average and fails on the slice that is a fifth of traffic is the demo all over again. And the threshold runs again the night the fallback fires, with the result waiting in the morning digest next to the fact that it fired.

The routing row gets a fourth column: the date the threshold was last run against the fallback. A row with a model, a fallback, a threshold, and a blank date is a provider switch nobody reviewed.

Under AI Product Management the argument is that the build got cheap and the scarce work is knowing which part to build. The provider layer is the cheap part now, and the lesson shows it in two settings. The routing row with a dated threshold is the part nobody builds unless a PM writes it down.

This is a standalone post alongside The FDE Transition.

This week: open your routing table, or write the first row of one. Model, fallback, eval threshold, date last run. If the fourth column is blank for any surface that has a fallback, run the threshold against the fallback before Friday, because the provider outage will not wait for the quarterly review.

Related answer: What has to travel when a customer changes model providers mid-engagement?

Sources: "Lesson 07: One Interface, Several Providers," Becoming Forward Deployed Engineer, October 22, 2026; the post is paid past its preview and only the problem section and the settings section, which are public, are cited here. Martha Janicki and Ramon Petgrave, "LangSmith LLM Gateway: Runtime Controls for Agents," LangChain blog (the page is dated July 30, 2026; the feed carried it on October 22, 2026), read in full. "The new economics of AI cost management," an interview with Kevin Stumpf, Databricks blog, October 22, 2026, read in full. Falk Gottlob, "Which Model Is the Smallest Knob in the Architecture Review" (October 19, 2026), for the three points, the 13.7 points at LangChain, and HarnessTax's 2x, each as reported there; "The Stop Rule" (September 29, 2026), for the 14% drift; "Gross Margin Is Your Job Now," for the routing table.