Does AI change its mind when comparing brands?

A repeated matchup is one brand pair compared on the same prompt, criterion, and engine in at least two answers. An answer-level verdict is the one unambiguous winner in one answer. One engine picked both brands as the winner in separate answers for 383 of 2,438 repeated matchups, or 15.7%.

The matched set covers 1,091 prompts, 1,507 cleaned brand names, and 6,777 answer-level verdicts. One answer is not enough to establish a stable comparison winner.

Grouping repeated comparisons without holding the prompt constant produces 887 winner changes across 3,260 matchups, or 27.2%. Holding the prompt, engine, brand pair, and criterion constant produces 383 across 2,438, or 15.7%.

BrightEdge compares brand sets across engines. SparkToro measures complete recommendation lists and order. Conductor measures brand-list overlap and lead-brand stability. This study measures the narrower decision of which brand wins one fixed comparison.

Of the 383 reversing matchups, 354 have one answer for the less frequent winner. In 29 matchups, each brand wins at least twice.

A single reversal can be an isolated answer. Repeated wins for both brands identify comparisons that need closer monitoring, but they do not explain why the decision changed.

Google AI Overviews changes the winner in 192 of 949 repeated matchups, or 20.2%. ChatGPT is at 23 of 138, or 16.7%. ChatGPT Search is at 132 of 930, or 14.2%. Google AI Mode is at 36 of 421, or 8.6%.

The four engine samples cover different time windows and matchup mixes. The rates show where repeated checks matter in this window; they are not a synchronized engine experiment.

Ease-of-use comparisons change the winner in 127 of 625 repeated matchups, or 20.3%. Price is at 16.4%, features 14.0%, security 11.1%, performance 10.6%, and support 8.9%.

A single overall rate hides meaningful differences by criterion. Competitive monitoring should preserve what the brands were compared on, not only which names appeared.

We restricted ChatGPT Search and Google AI Mode to the same 127 prompt, pair, and criterion matchups. ChatGPT Search changes the winner in 13, or 10.2%, while Google AI Mode changes it in nine, or 7.1%.

The matched set removes a different matchup mix as the full explanation. Both rates still show that a repeated answer can reverse a fixed brand comparison.

Jira Software and Linear change winners eight times across 63 repeated matchups and 216 answer-level verdicts. Asana and Trello change seven times across 13 matchups. DraftKings and FanDuel, Shopify and WooCommerce, and HelloFresh and EveryPlate each change five times.

The table requires at least 10 repeated matchups and 30 answer-level verdicts. A high count identifies comparisons to investigate; it does not show that either winner was factually correct.

The measured set contains 240,719 explicit comparison statements. We resolved decisive comparisons to cleaned brand names and grouped related labels into the six criteria used in this report. Of 29,278 grouped answer comparisons on those six criteria, 385 name both brands as the winner and are excluded.

Keeping only higher-confidence comparisons produces 366 reversals across 2,367 matchups, or 15.5%. Keeping every original criterion label separate produces 173 across 2,176, or 8.0%. The criterion grouping is a significant analysis choice, so quarterly reruns must preserve it or report the change.

Rerun the same comparison prompt on the same engine. Record the criterion and answer-level winner. Treat a single result as one observed answer, not a durable competitive verdict.

Investigate comparisons that reverse repeatedly. Check whether the answers use different facts or definitions before changing positioning, content, or competitive claims.