How often do AI-cited pages block access?

An access-limited page is a cited URL whose latest eligible check returned HTTP 401, 403, or 429. We kept one latest result per page and required that check to occur at or after the page's latest recorded citation. Of 154,986 eligible pages, 2,435 limited access. The exact rate was 1.6%.

Ahrefs reports that 67% of ChatGPT's top citations are off-limits to marketers. Its study defines off-limits as not pitchable or influenceable through traditional outreach. It does not mean 67% of cited servers refused a request. Audit outreach opportunity and direct page access as separate questions.

The latest eligible result was a successful response for 144,774 pages, or 93.4%, and a redirect or not-modified response for 6,367, or 4.1%. Together they account for 151,141 pages, or 97.5%. Access-limited results account for 1.6%, page-not-found or gone results for 0.66%, server errors for 0.20%, other 4xx responses for 0.04%, and other responses for 0.001%.

A page can be hard to influence, omitted from later answers, or disallowed for a named crawler while still returning a normal response to a direct check. Semrush measures declared crawler rules in robots.txt. This study measures the direct page response after a citation.

HTTP 403 accounts for 2,236 of the 2,435 access-limited pages, or 91.8%. HTTP 429 accounts for 185, or 7.6%, and HTTP 401 accounts for 14, or 0.57%.

Most access-limited results were explicit refusals rather than rate limits or authentication prompts. The status code identifies the next diagnostic step, but it does not reveal the publisher's intent or the rule that produced the response.

Google AI Mode had 2,249 access-limited pages among 136,583 checked pages, or 1.6%. ChatGPT Search had 226 among 20,095, or 1.1%.

Keep the two engines separate during a citation audit. The difference is descriptive because the same page can be cited by both engines and the engines cite different sources.

Among source groups with at least 1,000 checked pages, government pages were at 4.0%, directories 3.0%, news 2.4%, social 2.3%, health 2.0%, and general web pages 2.0%. SaaS pages were at 0.73% and developer pages at 0.18%.

Source type changes the baseline, but no displayed group reached 5%. Use the group rate to prioritize checks, not to assume that every page in the group limits access.

The table requires at least 50 checked pages and at least three access-limited pages. Reddit had 55 access-limited pages among 63 checked pages, or 87.3%. Quora had 49 among 117, or 41.9%. Byrdie was at 20.6%, Food & Wine at 15.9%, Serious Eats at 11.0%, and NerdWallet at 8.8%.

Recheck the exact cited URL and response before treating a whole source as unavailable.

The eligible window contained 2,469 pages that returned 401, 403, or 429 at least once. On the latest check, 2,435 were still access-limited and 34 returned a successful, redirect, or not-modified response. The later-normal-response rate was 1.4%.

A recheck can clear a temporary response, but most access-limited pages in this window remained limited on the latest result. Confirm the latest response before escalating to the publisher or changing content strategy.

The two-engine corpus contained 1,823,717 cited pages. We excluded 1,557,618 without a check in the window, 14,854 without a valid HTTP result, and 96,259 checked before their latest recorded citation. That left 154,986 pages.

The 1.6% result applies only to eligible checked pages. A strict 403-only definition gave 2,236 of 154,986, or 1.4%. An expanded definition that also includes 402, 406, and 451 was 2,453 of 154,986, or 1.6%. The result should not be applied to cited pages Parse did not check, to robots.txt policy, or to the access behavior of an AI engine.

The Parse index isolates 2,435 pages with a latest authentication-required, forbidden, or rate-limit response. The domain and source-group results show where those pages concentrate.

Check three things separately. Read robots.txt for declared crawler rules, request the exact cited URL to see the current response, and rerun the prompt to see whether the engine still cites it. Fix only the one that failed, then run the same check again.