How closely do brand recommendations from memory-only endpoints agree with recommendations from live-search endpoints?
We ran 858 matched buyer questions across ChatGPT Memory (gpt-4.1-mini) and ChatGPT Search (gpt-6), plus 189 matched buyer questions across all three frontier lab pairs (OpenAI, Anthropic Claude, and Google Gemini with memory vs live web search) and 123 matched by-name brand questions from October 15 through November 19, 2026.
ChatGPT Memory and Search chose different #1 brands for 80.7% of the 778 matched buyer questions where both named a brand. The two endpoints use different models.
ChatGPT Memory and Search chose different #1 brands on 80.7% of questions
Across 858 matched buyer questions answered by ChatGPT Memory (gpt-4.1-mini) and ChatGPT Search (gpt-6), both endpoints named at least one brand on 778 questions. They agreed on 150 top picks (19.3%) and differed on the remaining 628 (80.7%). The model and search configuration both differ in this comparison.
The endpoints had similar shortlist lengths (6.88 brands in Memory and 6.93 in Search) and shared 1.89 brands per question on average. Search named a brand on 49 questions where Memory named none. Memory named a brand on 19 questions where Search named none.
- Different #1 brands on ChatGPT Memory and Search
- 80.7%
Only 150 of 778 kept #1 (19.3%)
858 matched buyer questions comparing ChatGPT Memory vs Search
- Average shared brands per question
- 1.89
Out of 6.88 (Memory) vs 6.93 (Search)
The two endpoints shared fewer than two of the seven brands named per question
- Highest 3-lab #1 agreement (memory vs web)
- 21.7%
Claude 21.7% · OpenAI 20.9% · Gemini 20.5%
Measured on 189 matched buyer questions across all six endpoints
- Longer brand briefs under ChatGPT Search
- 7.8x
5,417 vs 696 average characters
Across 123 matched by-name brand identity questions
- Questions keeping the same #1 brand
- 19.3%
150 of 778 questions naming brands on both
- Average shared brands per answer
- 1.89
Out of 6.88 (Memory) and 6.93 (Search)
- Questions naming a brand on one endpoint only
- 49 vs 19
49 Search-only branded vs 19 Memory-only
Takeaway
Top brand picks differed across memory and search endpoints at all three labs
This turnover isn't unique to OpenAI. On 189 buyer questions matched across all six memory and live-web configurations from OpenAI, Anthropic, and Google, roughly four in five top picks differed within each lab. OpenAI kept #1 in 20.9% of pairs (36 of 172), Anthropic Claude kept #1 in 21.7% (33 of 152), and Google Gemini kept #1 in 20.5% (34 of 166).
Full-shortlist Jaccard overlap stayed equally low across all three labs: 0.185 on OpenAI, 0.154 on Google Gemini, and 0.151 on Anthropic Claude. Response sizing diverged by lab: OpenAI stretched answers 3.3x longer with search (2,245 to 7,481 characters) at flat brand counts (7.48 to 7.43), while Gemini shortened prose slightly (4,285 to 3,725 characters) even as it expanded average brands named from 8.98 to 11.65.
Share of 189 matched questions keeping the same #1 brand (memory vs live web)
- Anthropic Claude (189-Q)21.7%33 of 152 · 0.151 Jaccard
- OpenAI ChatGPT (189-Q)20.9%36 of 172 · 0.185 Jaccard
- Google Gemini (189-Q)20.5%34 of 166 · 0.154 Jaccard
Takeaway
83.3% of brand-question appearances belong exclusively to Memory or Search
Across the 858-question ChatGPT panel, we observed 9,721 total brand-question appearances. Only 1,619 pairs (16.7%, representing 684 distinct brands) appeared in both Memory and Search.
Nearly identical volumes appeared strictly on one side: 4,047 Search-only appearances (41.6%, covering 2,304 distinct brands) and 4,055 Memory-only appearances (41.7%, covering 1,977 distinct brands). Search expanded the pool of distinct brands named exclusively in one mode by 16.5% over Memory (2,304 vs 1,977).
Brand-question appearances across ChatGPT Memory vs Search (9,721 pairs)
- Memory Only4,055 pairs41.7% · 1,977 brands
- Search Only4,047 pairs41.6% · 2,304 brands
- Both Memory & Search1,619 pairs16.7% · 684 brands
Takeaway
Search surfaced newer AI-era tools from zero Memory mentions to #1 wins
The biggest Search gainers on the 858-question ChatGPT panel are recently launched or rebranded products that didn't appear in Memory answers. Google Cloud Security Operations jumped from 0 memory mentions to 41 search mentions (4 #1 picks), Bunny from 0 to 28, Cribl from 0 to 26 (2 #1 picks), TryProfound from 0 to 25 (17 #1 picks), Peec AI from 0 to 24 (3 #1 picks), Otterly AI from 0 to 19, and Soar from 0 to 16 (6 #1 picks).
Live retrieval lifted these products directly from zero Memory mentions onto Search buyer shortlists and #1 slots.
Top ChatGPT Search gainers absent from Memory (858-Q panel)
Search mentions · 7 results
- Google Cloud Security Operations41
- Bunny28
- Cribl26
- TryProfound25
- Peec AI24
- Otterly AI19
- Soar16
Explore the data (7 rows)
| Google Cloud Security Operations | 0 | 41 | 4 |
| Bunny | 0 | 28 | 0 |
| Cribl | 0 | 26 | 2 |
| TryProfound | 0 | 25 | 17 |
| Peec AI | 0 | 24 | 3 |
| Otterly AI | 0 | 19 | 0 |
| Soar | 0 | 16 | 6 |
Takeaway
Memory named some policy-sensitive and legacy brands more often
Several newer names appeared more often in Search. Some policy-sensitive and legacy names appeared more often in Memory. Policy-sensitive categories dropped sharpest: Betway went from 18 memory mentions (7 #1 picks) to 0 under live search, and SIG Sauer plunged from 19 memory mentions (10 #1s) to 1 (0 #1s).
Legacy infrastructure labels and older market staples also faded once live web results took over. Azure Data Lake Storage dropped from 23 mentions to 0, Elastic Stack from 23 (2 #1 picks) to 0, Amber Road from 19 (8 #1 picks) to 0, and Trello fell from 28 mentions (8 #1 picks) to 10 (2 #1 picks).
Brands dropping sharply between ChatGPT Memory and Search (858-Q panel)
Takeaway
Search makes ChatGPT more comparative while Claude and Gemini get more decisive
Across 9,690 brand-level evaluations on the 189-question three-lab panel (Parse's standardized brand-evaluation rubric, answer-matrix-v4 / jev-1.14.0), live search pushed ChatGPT in the opposite direction from Claude and Gemini. On ChatGPT, active recommendations fell 21.1 points from 77.6% in Memory (1,475 judgments) to 56.5% in Search (1,611 judgments), while direct comparisons more than tripled (2.4% to 8.1%) and alternative framing climbed from 5.8% to 9.7%.
Claude and Gemini both became more willing to recommend once search ran. Active recommendations climbed from 67.3% to 71.6% on Claude (1,262 to 1,264 judgments) and from 66.7% to 75.9% on Gemini (1,782 to 2,296 judgments), while compared mentions dropped sharply on both labs (7.7% to 4.5% on Claude; 5.4% to 2.2% on Gemini).
Active recommendation share across 3 AI labs: Memory vs Live Search
- Memory
- Live Search
- ChatGPT
- Memory77.6%1,475 judgments
- Live Search56.5%1,611 judgments
- Claude
- Memory67.3%1,262 judgments
- Live Search71.6%1,264 judgments
- Gemini
- Memory66.7%1,782 judgments
- Live Search75.9%2,296 judgments
Takeaway
On by-name brand checks, ChatGPT Search writes 7.8x longer briefs than Memory
The memory vs search split hits branded queries just as hard. Across 123 matched by-name brand identity questions, ChatGPT Memory delivered short, sparse summaries averaging 696 characters.
With live search enabled, ChatGPT Search expanded those by-name briefs 7.8x to 5,417 characters by pulling live product pages, pricing, and reviews. Over the same prompt set (122 matched answers), Google AI Mode averaged 1,588 characters.
Average answer length on 123 matched by-name brand questions
- ChatGPT Search5,417 chars123 Q
- Google AI Mode1,588 chars122 Q
- ChatGPT Memory696 chars123 Q
Takeaway
Explore every 3-lab metric, treatment shift, and exclusivity bucket
Use the tabs below to inspect the complete three-lab comparison table, the six-endpoint brand-treatment shift, and the ChatGPT exclusivity buckets from the dataset.
Memory vs live search dataset explorer
#1 brand agreement, brand counts, and answer lengths comparing pre-training memory vs live web search.
Same #1 (%) · 4 results
- Anthropic Claude (189-Q panel)21.7%
- OpenAI (189-Q panel)20.9%
- Google Gemini (189-Q panel)20.5%
- ChatGPT (858-Q panel)19.3%
Explore the data (4 rows)
| Anthropic Claude (189-Q panel) | 189 | 152 | 21.7 | 6.44 | 6.3 | 1,410 | 1,705 |
| OpenAI (189-Q panel) | 189 | 172 | 20.9 | 7.48 | 7.43 | 2,245 | 7,481 |
| Google Gemini (189-Q panel) | 189 | 166 | 20.5 | 8.98 | 11.65 | 4,285 | 3,725 |
| ChatGPT (858-Q panel) | 858 | 778 | 19.3 | 6.88 | 6.93 | 2,257 | 7,888 |
Takeaway
What to do with these numbers
Measure each live-search endpoint your buyers use. The ChatGPT Memory and Search endpoints chose different #1 brands on 80.7% of the matched questions where both named one.
If your brand is new or recently repositioned, focus on live-retrievable proof right now: public product docs, third-party comparisons, and accurate pricing pages. As TryProfound (0 to 25 mentions, 17 #1 picks) and Peec AI (0 to 24 mentions, 3 #1 picks) show, newer products can win #1 once live search runs.
Inspect your by-name brand answers in ChatGPT Search too. A 5,400-character live brief pulls every stray review and missing spec onto the page.
Takeaway
Study scope and counting rules
All comparisons run on matched buyer questions executed between October 15 and November 19, 2026. The primary OpenAI comparison matches 858 buyer questions answered by both ChatGPT Memory (gpt-4.1-mini, first answer) and ChatGPT Search (gpt-6, latest answer), of which 778 named brands in both modes and generated 9,721 brand-question appearances (#1 brand = smallest mention position).
The three-lab panel evaluates 189 matched buyer questions executed across all six configurations (OpenAI memory and search, Anthropic Claude memory and web search, Google Gemini memory and web search), yielding 9,690 brand-level judgments under Parse's standardized brand-evaluation rubric (answer-matrix-v4 / jev-1.14.0). Note that OpenAI's memory endpoint uses gpt-4.1-mini while ChatGPT Search uses gpt-6, whereas Anthropic and Google pair same-family memory and web-enabled endpoints directly; roughly four in five top picks differed within each lab. The OpenAI pair measures endpoint disagreement. Estimating the effect of retrieval alone would require holding the model and the rest of the configuration fixed.
By-name identity checks cover 123 matched brand questions on ChatGPT Memory and ChatGPT Search (122 on Google AI Mode).
How we measured
858 matched buyer questions answered by both ChatGPT Memory (gpt-4.1-mini) and ChatGPT Search (gpt-6), 189 matched buyer questions answered across all three lab pairs (OpenAI, Anthropic Claude, Google Gemini memory vs live web search) with 9,690 brand judgments under Parse's standardized brand-evaluation rubric, and 123 matched by-name brand identity questions, October 15 through November 19, 2026.
- Questions with different #1 brands on ChatGPT Memory and Search
- 80.7%
Only 19.3% (150 of 778) kept #1 between Memory and Search
- shared brands per question out of ~6.9 named
- 1.89
6.88 average in Memory vs 6.93 in Search (858 matched questions)
- 3-lab #1 agreement (memory vs web)
- 20.5% to 21.7%
Gemini 20.5%, OpenAI 20.9%, Anthropic Claude 21.7%
- brand-question pairs appearing in both modes
- 16.7%
1,619 shared vs 4,047 Search-only and 4,055 Memory-only