Agents Don't Answer Interviews. Amplitude's Left a Note.

In May I published a discovery playbook for products whose customers are agents, and its short version says, flatly, that agents don't answer interviews. On October 6 an engineer at Amplitude described an agent answering one. It took a single optional parameter.

The short version

Chanaka Perera's post on consolidating Amplitude's MCP server includes a detail most readers will skip: every tool has an optional parameter, Tool Rationale, asking why the agent is calling it, and agents fill it in about three quarters of the time without being told to. Amplitude used it to explain a funnel in which, of 18,703 users who queried data, 20% rendered a chart and less than 5% saved it. The rationales showed agents were trying to save and failing to finish a three-step chain, so the chain became one call and 16 chart tools became four. My May post, Customer Discovery When Your Customer Is an Agent, said agents cannot be interviewed and started its six methods with telemetry. This is the method it was missing. Add the field, build the funnel, read the rationales where sessions stop, and treat each one as a claim to check.

What Amplitude found

I read Perera's post in full. Most of it is about tool count, and that half is good too. By June, connecting to Amplitude's MCP server loaded 96 tools, roughly one per API endpoint. For one customer on a 200K context window, listing those tools used 57% of the window before a single call. The team rebuilt the catalog around outcomes: one tool per object, so cohorts went from nine tools to one with eleven actions.

The part I want is smaller. Creating a chart in Amplitude takes three steps, and the tools copied them: query the data, render the chart, save the edit. Of 18,703 users who queried data, 20% went on to render a chart, and less than 5% saved it.

A funnel like that says where sessions stop. On its own it supports two opposite stories. Maybe agents only wanted the data. Maybe they wanted the chart and could not get there.

Amplitude could tell which, because every tool on the server carries one optional parameter called Tool Rationale, a "Brief explanation of why you are calling this tool." Agents fill it in about three quarters of the time, unprompted. On the query calls where the chain stopped, the rationales read like "save chart edit for dashboard analysis."

The agent meant to save. It never made it through the chain. So the agent now sends a definition and gets data back in one call, and 16 chart tools became four.

What I got wrong in May

Customer Discovery When Your Customer Is an Agent builds on Teresa Torres' weekly interview habit and asks what replaces it when the user is a Claude instance or an MCP-wired workflow. I wrote that the agent "doesn't answer interview questions," and I listed six methods. The first was telemetry: which endpoints, in what sequence, where the errors and the unexpected usage are. The second was interviewing the human who deployed the agent, because you cannot interview the agent.

I also wrote that I expected to revise some of it within six months. This is month five.

The telemetry method has a hole, and the chart funnel is a clean picture of it. Sequence data gives you the what. It gave Amplitude 20% and 5%. It could not give them the sentence that changed the design. The operator interview would have, eventually, for the handful of operators you can book.

The rationale is the agent's own account, written at the moment of use, on most calls, for every customer. I did not think that was available. It is, and it costs one parameter.

The drop

For anyone who owns an MCP server or an agent-facing API.

1. Add one optional string parameter to every tool: why are you calling this. Use Amplitude's wording or your own. Keep the description short, since tool descriptions load on every connection, a cost I went through in Context Is King. Written Context Is Rented..
2. Do not make it required. Three quarters unprompted is plenty. A required field gets text written to pass the schema.
3. Build the funnel across calls. Pick the chain that represents a finished job in your product and count how many sessions reach each step.
4. Read the rationales on the last call before the stop. Keep reading until the pattern repeats. You are looking for the gap between what the agent called and what it says it was trying to do.
5. Treat each rationale as a claim. It is generated text. Check it against what the agent did next.

Step five is the one I would not skip. In The Receipt Gate the point was that an agent saying "all checks pass" is a sentence, and the log is the evidence. Intent works the same way. A rationale can be wrong, vague, or copied from the tool description. Amplitude's finding holds because two things agreed: sessions stopped at the query, and the rationales on those calls said save.

One more number from the post is worth copying as a metric. During the rollout Amplitude watched recovery rate: after a failed call, did the agent make a successful one within five minutes? A tool that fails with a clear message is fine. One that leaves the agent guessing is not. That is a usability score for a user who will never fill in a survey.

This belongs to the running argument on AI Product Management: AI collapsed the cost of the parts of the job PMs were trained for and left the parts nobody trained for. Discovery with a customer that is software is one of those parts, and the first real method for it arrived as a footnote in an engineering post.

Pick one thing this week. Add the field to one tool, the one at the start of your most important chain, and read what comes back on Friday.

Related answer: How do you find out why an AI agent is calling your tool?

Sources: Chanaka Perera, "Lessons from consolidating our MCP server," Amplitude Blog, October 6, 2026. Customer Discovery When Your Customer Is an Agent, falkster.com, May 20, 2026. Teresa Torres, Continuous Discovery Habits, for the weekly interview practice the May post builds on.