Anthropic applied two layers of indexing control on shared chat pages:
The conflict: Because robots.txt blocked Google's crawler from accessing the pages entirely, Google's crawler never saw the noindex header. However, if the URLs were linked elsewhere on the web (e.g., on social media, forums, or other sites that Google could crawl), Google could still discover and index the URLs based on those external references. When the crawler is blocked by robots.txt from fetching a page, it cannot read the noindex directive — so it may index the URL from external context alone . Essentially, the
robots.txt disallow made the noindex tag invisible and therefore ineffective.
Key pattern: In all three cases, the root cause is that AI companies' "share" features generate public, guessable URLs, and users post those links on the open web — after which search engines index them. Anthropic's 2025 incident was essentially the same issue .
site: queries Google had previously estimated it indexed "just under 600" Claude conversations in the prior incident before they were removed . The current status is in flux — Anthropic had not publicly addressed the matter at the time of reporting
.
Anthropic's official help center provides these steps:
Review your shared chats:
Unshare (revoke) a chat:
Delete a conversation entirely: