What should you ask an AI agent vendor before you sign?

Answered by Falk Gottlob · Reviewed · 1 min read

THE SHORT ANSWER

Send Marco Di Santo's five questions from his November 10, 2026 post (an agent in production and how long it has run, the cost in its worst month, who fixes it after go-live and how fast, what it got wrong in the first weeks, and a call with the process owner rather than the sponsor), and mark each answer Shown, Described, or Missing as he says. Then ask for one artifact the sales side does not hold: the day-60 page the vendor's forward deployed engineer wrote for one account that went live two quarters ago. The page I use has four lines, the promise, the signal (the customer's own eval rows on a schedule, with the go-live pass rate as the floor), the watch list (what the customer switched off), and the keys (the named person at the customer who can change a prompt without calling the vendor), with a recall rule of three triggers and two business days underneath. It answers questions 1, 3, 4, and 5 with names and dates; question 2 sits on the per-outcome line. A vendor with agents in production has one per account. In my own pricing migration, 12 accounts sat on plan on the dashboard with usage under the minimum and two were lost; the page would have shown a blank keys line two quarters earlier.

The five questions stay. The thing you ask for changes, from five answers across a table to one page the vendor's FDE already wrote.

Di Santo's five questions and his Shown, Described, Missing sort, laid over the four-line page, are in Marco Di Santo's Five Vendor Questions Come Back Described. Ask for the Day-60 Page.. The page itself is Month Two: The FDE Leaves, the Workflow Has to Stay, and the contract lines that go with it are in Buying From an FDE Company: Write the Move-Out Into the Contract.

SOURCES

THE LONG VERSION

RELATED ANSWERS