Where the difference is
It is tempting to assume models disagree about the facts. In our measurements the larger difference is earlier: whether a model produces a useful answer for the seller at all — the product named, or an official URL present.
Share of completed questions that produced a recorded outcome for the seller, 04.08.2026 to 16.09.2026:
| Provider | Model | Share | | --- | --- | --- | | Google | gemini-2.5-flash | 83% | | Perplexity | sonar | 64% | | Anthropic | claude-haiku-4-5-20251001 | 57% | | OpenAI | gpt-4o-mini | 53% | | xAI | grok-4-latest | 52% |
Among the 54 questions where three or more models did produce a usable answer, 87% got the same outcome from all of them (study). Useful answers tend to agree; whether you get one depends on the model.
Why models differ
- They search differently. Some answer from a fresh web search every time,
some from what they learned in training, some mix the two. The same question reaches different pages.
- They cite differently. The share of citations pointing at the seller's own
site ranged from 36% to 52% across models; one model returned its own search redirects in 74% of cases (study).
- They change under the same name. A provider can update the model behind a
name without announcing it. A result without the exact model name and a date is not checkable a month later.
What to do with that
Do not treat one model as a sample of AI. Ask the same question of several, name each model in full, date each answer, and repeat on a schedule. A single answer is an anecdote; the same question across models and weeks is a measurement.
How to run that yourself: /learn/how-to-check-what-ai-says-about-your-product.