Downside Exposure, Extended: When the Model Is Wrong About You

Downside exposure asks what happens when an AI feature fails a user. It needs a second half now: what happens when an AI system fails to represent you accurately to someone else.

The short version

Downside exposure, extended. The original framework was first-party risk: your product's AI feature gives a wrong answer to your own customer, and you rank features by how fast a silent quality drop would cost you customers. That still holds. The second half is second-party risk: as more of the discovery and evaluation layer runs through AI systems you do not control, comparison tools, procurement assistants, general assistants fielding which-vendor questions, the model can get you wrong and you never find out until a deal dies for a reason no one traces back to you. You cannot instrument it the way you instrument your own product. What you can do is treat your documentation and positioning as an eval target: periodically ask a general model, cold, to describe your product, and fix the documentation wherever the answer is vague, wrong, or generic.

The original framework was built for first-party risk, your product's AI feature gives a wrong answer to your own customer, and you need to know the blast radius before you ship. That still holds, and ranking AI features by downside exposure is still the first move. But as more of the discovery and evaluation layer around your product runs through AI systems you do not control, comparison tools, procurement assistants, general-purpose assistants fielding which vendor should I use questions, there is a second failure mode that the original framework does not cover: the model gets you wrong, and you never find out until a deal falls through for a reason nobody can trace back to you.

This is a harder problem than first-party downside exposure because you cannot instrument it the way you instrument your own product. You do not get the error logs. You do not get to run an eval against a competitor's model or a general-purpose assistant that a prospect happened to ask about you. What you can do is treat your own product documentation, positioning, and differentiation as an eval target, not just a marketing asset, structured and specific enough that a reasonably capable model, reading it cold, describes your product correctly and does not need to guess at what you do or why you are different. This is where AI evaluators become a real PM audience, not a metaphor.

I would extend downside exposure with a specific practice: periodically ask a general model, with no special access, to describe your product and your differentiation as if a prospect had asked. Do it the way you would run any other eval, looking for where the answer is vague, wrong, or generic. Where it is wrong, the fix is not a press release. It is fixing the underlying documentation the model is drawing from, because that documentation is now doing sales work whether you designed it to or not. If you need a format for the eval itself, the five-row eval template is enough to start.

The governance conversation happening around AI right now is almost entirely about what your product does to users. This is the other half, what other people's AI does to your product's reputation without you in the room. Nobody is building a framework for that yet. I think we should.