The short version
The switch interview is one of the best tools in discovery, and AI just gave us something better for half of what we used it for. When customers use an AI feature, they type what they want in plain language, at the exact moment they want it, across your entire user base. That is a switch interview with no recruiting, no scheduling, no recall bias, and a sample size of everyone. Prompt logs are the richest unmet-needs dataset most product teams already own and are not reading. They will not tell you why a customer feels something, so interviews are not dead. But for what customers are trying to do and where your product fails them, the logs beat the call. Stop booking eight interviews to guess at demand you can already see verbatim.
I ran switch interviews for years. They work. Reconstruct the timeline of a customer's decision, find the forces, and you get the real job instead of the feature request they opened with. Bob Moesta built a career on it for good reason.
Here is what I could never fix about them: they are slow, they sample almost nobody, and they ask a human to accurately remember what they were thinking three weeks ago. I would spend two weeks to learn from eight people what they thought they wanted. By the time I had the insight, the eight had moved on.
Then we shipped AI features, and I realized the interview I had been working so hard to schedule was already happening thousands of times a day, in writing, for free.
What a prompt actually is
When a user types a request into an AI feature, look at what they just handed you. The job, in their own vocabulary. The trigger, because they typed it the second they hit the wall. And the failure, captured the instant your product could not do the thing.
That is the exact payload a switch interview tries to reconstruct from memory. Except the customer is not remembering it. They are doing it, live, while you watch. No recruiting. No 45-minute call. No "walk me back to the moment you decided." The moment is in the log.
A prompt is a customer narrating the job out loud, in their own words, at the exact second they wanted it done. That is the thing interviews spend two weeks trying to recover.
Where the logs win, and where they do not
I want to be precise, because the easy version of this take is wrong. Prompt logs do not replace interviews. They replace a specific job interviews were doing badly.
Logs win on what and where. What is the customer trying to do, and where does the product fall down. That is unbeatable in the logs because it is literally the text. You see the request your product cannot serve, the rephrasing when it fails, the workaround they narrate in frustration.
Interviews still win on why. Why does this matter to them, what is the emotion, what is the context around the moment. A log shows you a user asked the same thing three different ways and gave up. It does not show you that they gave up because their boss was waiting and they were embarrassed. That is still a conversation.
So the model is not logs instead of interviews. It is logs for breadth, interviews for depth, in that order.
The four signals I read first
When I open a prompt dataset, I am hunting four things, in priority order.
Requests the product cannot do yet. These are unmet jobs stated in the customer's own words. No translation needed. If forty users a week ask the AI to do something it refuses, that is a feature brief that wrote itself.
Reformulations. When a user rephrases the same ask three times, they are not confused, they are blocked. Repeated rephrasing is the cleanest friction signal I know, and it never shows up in a survey.
Vocabulary mismatches. When customers consistently name the job differently than your UI does, you have a positioning problem hiding as a discovery one. The words in the logs are the words that belong on your landing page.
Rising intents. When a request type grows week over week, you are watching demand form months before it reaches a sales call or a quarterly review. This is the signal that makes the whole exercise a continuous discovery engine rather than a one-time audit.
How to actually run it
You do not need a data team. You need consent, a de-identification step, and a clustering pass.
Start with aggregated, de-identified prompts. Strip anything personal. You are looking for the shape of demand, which lives in the aggregate, not in any single private session. Treat this data with the same care you would treat session recordings, because that is what it is.
Then cluster by intent and let an agent do the grouping. This is exactly the kind of work I hand to a listening agent: ingest the week's prompts, cluster them by what the user was trying to do, flag the rising and the unmet, and surface the top patterns every Monday. The PM reads a ranked list of real jobs instead of a stack of call recordings.
Then, and only then, book the interviews. Now they are surgical. You are not calling eight random users to discover the problem space. You are calling four people who triggered a specific rising intent to learn the why behind a pattern you already measured. That is the outcome-to-prototype loop running on real demand instead of guesses.
Logs tell you where to point the interview. That one change turns discovery from fishing into targeting.
The uncomfortable part
If your product has an AI feature, you are already sitting on the richest unmet-needs dataset your company has ever had, and most teams are not reading it. They are still recruiting for interviews to discover demand that is sitting verbatim in a table nobody queried.
That is the part that should sting a little. The research you are scheduling has, in many cases, already been volunteered. Your customers told you exactly what they want, in their own words, the moment they wanted it. The only question is whether anyone on your team is listening to the channel they are actually using.
Pick one AI surface in your product this week. Pull a de-identified sample of what people typed. Cluster it by intent. I will bet the top three clusters contain at least one job you did not know was on your roadmap, and one word your marketing should have been using all along.
Sources: Bob Moesta & Chris Spiek, The Re-Wired Group (switch interviews) · Teresa Torres, Continuous Discovery Habits · Intercom, on mining real customer language
Also on Medium
Full archive →How to Avoid Survivorship Bias in Product Management
Lessons from the British bomber study, applied to PM customer interviews and analytics.
Why Users Don't Know What They Want Until You Show Them
Visionary product development, when prototypes beat interviews, and how to know the difference.
Frequently asked
What is a switch interview in product discovery?+
A switch interview is a Jobs to Be Done research technique where you interview a customer about the moment they switched to or away from a product, reconstructing the timeline of the decision to find the real job and the forces behind it. Bob Moesta and Chris Spiek popularized it. It is powerful and it is slow: you recruit, schedule, run a 45-minute call, and rely on the customer accurately remembering what they were thinking weeks ago.
Why are prompt logs better than interviews for some discovery questions?+
Because a prompt log is the customer telling you what they want in their own words at the exact moment they wanted it, with no recruiting, no scheduling, and no recall bias. When a user types a request into an AI feature, they are narrating the job live. You get the real vocabulary, the real trigger, and the real failure when your product could not do it. Interviews reconstruct intent after the fact. Prompt logs capture it as it happens, across every user instead of the eight you could book.
Do prompt logs replace customer interviews entirely?+
No. Prompt logs are unbeatable for what customers are trying to do and where the product falls short, because that is literally what they type. They are weak on why, on emotion, and on the context around the moment. The right model is prompt logs for breadth and signal detection, then a small number of targeted interviews to understand the why behind the patterns the logs surface. The logs tell you where to point the interview, which makes the interview far more efficient.
How do you mine prompt logs for product discovery without violating privacy?+
Work with aggregated, de-identified prompts and explicit consent, never raw logs tied to a named user without permission. Strip personal data, cluster the requests by intent, and analyze patterns rather than individuals. The goal is to see the shape of demand, which lives in the aggregate, not to read one customer's private session. Treat prompt data with the same care you would treat session recordings, because that is effectively what it is.
What signals should product managers look for in prompt logs?+
Four. Requests for things the product cannot do yet, which are unmet jobs in the customer's own words. Reformulations, where a user rephrases the same ask three times, which signal friction or a capability gap. Vocabulary mismatches, where customers name the job differently than your UI does, which is a positioning and naming problem. And rising intents, where a request type grows week over week, which is a roadmap signal arriving months before it would show up in a sales call.
Comments (0)
Sign in with LinkedIn to leave a comment.
Sign in with LinkedIn