A question worth asking of any AI product you rely on: which model served that request?
For most consumer AI apps there is no way to find out. Not because the information is secret, but because the interface was never built to surface it, and the answer changes more often than users imagine.
Three reasons the answer moves
Model families are not single models. A vendor announces a name and ships several variants under it — a fast one, a careful one, sometimes a cheap one for free tiers. The marketing name is the family. The thing that ran is a member of it.
Capacity routing. Under load, requests get moved. A request that would normally go to the careful variant goes to the fast one, or to a different region, or to an older pinned version that has spare capacity. This is sensible engineering and it is invisible.
Silent experiments. Products run A/B tests on models the same way they run them on button colours. If you are in the treatment group on Tuesday and the control group on Thursday, you experienced a model change that does not appear in any changelog.
None of these are scandals. All three are reasonable. Together they mean that “the model” you are using is a distribution, not a thing.
Why this breaks real work
For casual use it does not matter. For anything you have to reproduce, it is the whole ballgame.
The concrete failure: you produce something good, you need three more like it next month, and you cannot get there. You still have the prompt — prompts live in text boxes and text boxes are easy to copy. You do not have the variant, the tier, or the route. So you re-run and get something different, and you attribute the difference to luck, because luck is the only explanation the interface has left available.
The second failure is diagnostic. When output quality shifts, you need to know whether the model changed, your prompt changed, or your expectations changed. Without a version string all three look identical, and teams burn days re-writing prompts to fix a routing change.
The third is commercial. If you are building on top of a vendor and your users report degradation, “we cannot tell which model served you” is not an answer you want to give.
What to ask for
A version string in the output. That is genuinely all.
Not a dashboard, not telemetry, not an API. One identifier, displayed with the result, showing what the inference call actually returned rather than what the client requested. Those differ precisely when it matters.
It costs about an hour to implement and it converts an unfalsifiable complaint into a lookup. I think it is entirely reasonable to treat its absence as a signal about how seriously a product takes reproducibility — not as a scandal, but as information about what the product is for.
The case that makes this concrete
The sharpest version of this is when a vendor ships two models at the same price and the consumer app picks for you without saying which.
Same cost, different trade-off — one fast, one with a higher ceiling. There is no budget reason to prefer either, so the choice is purely about the job. And it is exactly the choice a chat interface takes away, while also declining to tell you which way it went. Users end up forming theories about the model’s “mood” that are really theories about routing.
That is the situation I built around: an interface that names the model on every result prints the identifier the API returned, plus the tier, the delivered size and the elapsed time, with the raw response a click away. The picker itself is documented on the GPT Image 2.5 Image Generator comparison page, which sets the two variants out by the job each suits rather than as a quality ladder — because at equal price, a ladder is the wrong mental model.
If you take one thing from this: the next time output quality seems to have changed, check whether you can check. The answer tells you a lot about the tool.
Disclosure: I build that site. Independent, not affiliated with OpenAI.