Who writes the eval in a forward deployed engineering deployment?

The forward deployed engineer writes it, before building, and the customer's domain experts supply the labels. Colin Jarvis's October 22, 2026 account of OpenAI's FDE team has the engineers building evals with the customer's engineers before building the system, and a skill for writing fresh evals by the time a rebuilt deployment reached the next team. Saurabh Singh's method in the Vibe Engines handbook is the first-week version: two or three of the customer's experts label about 40 real cases independently, the disagreements become a written rubric, and the rubric becomes the labeling guide, the LLM-judge prompt, and the acceptance language in the statement of work. The eval is the deployment artifact. It is a frozen, versioned set of the customer's real cases with a target set against the human baseline, an error budget for the mistakes that are forbidden at any rate, the judge's agreement rate with human graders, and cost and latency limits. It travels into the handback to product, rides on the harness pull request, becomes the customer's independence test at month two, and is the customer's property in the contract. The product manager writes a second eval later, the one the generalized version has to pass, from the same rows. Aneel Bhusri's standard for AI running payroll, that almost right isn't good enough, only means something if someone wrote down the number for almost.