Marco Di Santo's Five Vendor Questions Come Back Described. Ask for the Day-60 Page.

Marco Di Santo published five questions to ask an AI agent vendor before you sign, and they are the right five. This is a Rewrite: the questions stay, the sheet stays, and the thing you ask for changes, because the evidence the sheet wants already exists on one page the vendor's forward deployed engineer wrote.

The short version

• Di Santo's November 10 post says the demo shows the agent on its best day and you are buying every day after go-live, so he gives the buyer five questions: an agent in production and how long it has run, the cost in its worst month, who fixes it after go-live and how fast, what it got wrong in the first weeks, and a call with the process owner rather than the sponsor.
• Each answer gets marked Shown, Described, or Missing, and two Missing is a reason to slow down. In the room, the sales side answers all five from memory and the sheet fills with Described, because sales does not hold the evidence.
• The vendor's forward deployed engineer does, on the day-60 page I use when an FDE's attention drops from one of three customers to one of twelve accounts: the promise, the signal (the customer's eval rows on a schedule, with the go-live pass rate as the floor), the watch list, and the keys (the named person at the customer who can change a prompt without calling the vendor), with a recall rule under it.
• That page answers questions 1, 3, 4, and 5 with names and dates. Question 2 sits on the per-outcome line.
• So send the five questions, then ask for the day-60 page of one account that went live two quarters ago, and mark the page, not the meeting.

What Di Santo published

The setup is a vendor meeting with a demo that works, because the vendor chose the inputs and rehearsed the agent, and a purchase of something else: "the agent on its two hundredth day, on your data, after the model under it has changed and the first odd cases have arrived." He cites the 2026 Gartner CIO survey for the gap: 17% of organizations have deployed agents and more than 60% expect to within two years.

Five questions, in order, each with a strong answer and a warning sign.

Show me an agent in production and tell me how long it has run. Production means real users inside a real process with an owner who would notice within a day if it stopped. A strong answer "names the process, the start date and the person who owns it on the client side." A warning sign is a list of logos.

What did it cost in its worst month. He cites IDC's survey for DataRobot from December 2025, in which 92% of organizations implementing agents said costs ran higher than expected and 71% said they have little control over where the costs come from, and Augment Code's example of a 20-step loop that bills 210,000 input tokens instead of 20,000 because the whole history goes back each step. A strong answer gives a typical month, the worst month, the reason, and who pays the difference under the contract.

Who fixes and improves it after go-live, and how fast does a fix arrive. A strong answer "names a role, describes the process and gives the date of the last fix it shipped to a client," and says what you would need to run it yourself if you stopped working with them. He notes OpenAI's Agent Builder deprecation, announced June 3 with shutdown on November 30, as the kind of change that arrives whether or not anyone owns it.

What did it get wrong in the first weeks. He cites an eight-week study of one production assistant agent that root-caused 22 incidents and counted at least 28 cases where the error signal never reached a human in a usable form, a class the authors call fail-plausible. A strong answer describes two or three specific mistakes and what changed. "No issues so far" means nobody is looking or nobody is telling you.

Can I talk to the process owner at one of your clients, as well as the sponsor. Sponsors talk about the decision. Process owners "know whether the agent still saves time in month six."

Then the sheet. Send the questions before the meeting. Mark each answer Shown (a system, a date, a person), Described (the vendor told you), or Missing. Two or more Missing, slow down. "A sheet full of 'described' tells you what to ask for in the next meeting."

Every one of those is a question I would ask, and the sort is the right sort. My disagreement is with the next meeting.

Why the sheet comes back Described

Picture the meeting. The account executive and a solutions engineer on the vendor side. Question 1: yes, we have three customers in production, one for about a year. Question 3: our customer engineering team, usually within a week. Question 4: some edge cases early on, all resolved. Question 5: we can arrange that.

4 of 5

Four Described and one Shown, if the logo slide counts. Not because the vendor is hiding anything. Because the people in the room do not hold the evidence. The start date, the last fix shipped, what broke in week two, and the name of the person who owns the process on the client side are not in the deck. They are with the vendor's forward deployed engineer, who lived through them and, if the vendor runs the model honestly, wrote them down on the day they rolled off the account.

I have written that page. Month Two: The FDE Leaves, the Workflow Has to Stay is the handoff I use when an FDE's attention drops from one of three customers to one of twelve accounts, held by a CSM who cannot read the configuration. Four lines.

The promise. The handshake's three columns, updated to what actually went live.

The signal. The customer's own eval rows, the ones from the pre-contract week and the handback packet, running on a schedule against production. The pass rate at go-live is the floor. The FDE Writes the Eval: Almost Right Has a Number is why those rows exist before the build starts.

The watch list. What this customer is likely to switch off, override, or route around once nobody who built it is in the building, and, by day 60, what it did.

The keys. The named person at the customer who can change a prompt or a routing rule without calling the vendor. If the keys line is blank at day 60, the deployment has not landed.

Under the four lines, a recall rule. Three triggers (the rows fall under the floor, something on the watch list happens, the customer wants a new workflow) and two business days for the FDE's manager to answer.

The page over the five questions

Lay the page over Di Santo's sheet and four of the five questions are already Shown.

Question 1 is the promise line. It carries the process, the go-live date, and the owner on the client side, which are the three things his strong answer asks for.

Question 3 is the keys line plus the recall rule. Who can fix it is the name on the keys line, on the customer's side, and who the vendor sends when the customer cannot is the recall rule, with the date of the last recall. That is also the answer to his follow-up, what you would need to run it yourself: the keys line is the person who already does.

Question 4 is the watch list. By day 60 it is a list of what the customer switched off and what the vendor changed in response, which is his "two or three specific mistakes" with dates on them.

Question 5 is the name on the keys line. The person who can change the prompt without calling the vendor is the process owner by construction, and a vendor who has the page can offer the call without going back to check who that is.

Question 2 does not live on the page, and should not. The worst month is a line on the per-outcome table I described in Three Token Bills, One Unit: per workflow, outcomes, tokens per outcome, cost per outcome, realized price, and margin, by month. A vendor who keeps that line shows you the worst month in ten seconds and can say who paid for it. A vendor who does not keep it will give you a price per seat, which is Di Santo's warning sign.

The rewrite, then. Send the five questions, as he says. Then ask for one artifact: the day-60 page of one account that went live two quarters ago, names redacted if they must be. Mark the page, not the meeting. A vendor with agents in production has one per account, and the AI Product Management thesis is that AI collapsed the cost of the parts PMs were trained for and left the parts nobody trained for; writing this page is one of the untrained parts, and it is where a vendor's FDE earns the seat. A vendor without the page answers all five Described, which is the sheet Di Santo describes, and you have learned that in one ask instead of two meetings.

What the dashboard said

The reason I trust the page over the meeting is that I have been on the vendor side of a dashboard that said Shown.

In the pricing migration I wrote up in Field Report: What Broke When We Killed Our Per-Seat Tier, 12 accounts sat on plan on the dashboard while their usage ran under the committed minimum. Nobody escalated. The behavior moved first and the dashboard never did. We lost two of them.

Every one of those accounts would have passed question 1 in a vendor meeting. Real users, a real process, a start date. What they did not have was a keys line, a named person on their side who was changing anything, because nobody on their side was. A page with a blank keys line would have said Missing, which was the true answer, two quarters before the renewal said it for us.

Di Santo's process owner question is the same instinct from the buyer's chair. The page just makes it a line somebody has to fill in on a date, instead of a call somebody has to arrange.

This week

If you are buying: send the five questions, then ask for the day-60 page of one account. If you are the vendor: pull one account that went live two quarters ago and try to write the page from what you have. Count the blanks. Each blank is a question your next buyer will mark Described, and the keys line is the one to fill first.

Related answer: What should you ask an AI agent vendor before you sign?

Sources: Marco Di Santo, "The Demo Is the Least Useful Thing an AI Vendor Can Show You," Operational AI, November 10, 2026, read in full; the Gartner, IDC, Augment Code, OpenAI, and arXiv figures are as he cites them and I did not open those sources. The one-of-three to one-of-twelve, the four lines, the three triggers, the two business days, and the 12 accounts and two losses are my own, from the posts linked above. This post is a standalone companion to The FDE Transition; the closest part is the month-two page.