Google, OpenAI, Microsoft and Anthropic Back One AI Training Opt-Out

Google, Microsoft, OpenAI and Anthropic said on Friday, Oct. 30, 2026, that they will honor one shared AI training opt-out that website owners can place in their robots.txt files, replacing a patchwork of company-by-company crawler rules. The line, a “policy token” called __usage_ai-training, comes from a draft filed the same day with the Internet Engineering Task Force (IETF), and lets a site tell every participating AI developer at once that it does not want its pages used to train generative models.

How the shared AI training opt-out works

Robots.txt is the plain-text file, standardized by the IETF as RFC 9309 in 2022, that tells automated crawlers which parts of a site they may visit. Until now, a publisher that wanted to keep its work out of AI training had to name each company’s crawler or token separately. The new approach keeps the familiar format but adds a purpose-based entry. In the example given in the IETF draft, a site writes “User-Agent: __usage_ai-training,” then “Disallow: /” to refuse training on everything, with optional “Allow” lines for folders it is happy to share, such as a press kit.

The draft’s author is Kevin Kelley, a standards lead at Anthropic. It says the token is a statement about how content may be used, not an instruction about crawling, so a company’s normal crawler rules still decide what its bots fetch. Two details matter for site owners. A catch-all “User-Agent: *” block does not count as a training opt-out, so a site has expressed no preference unless it names the new token. And private licensing deals may override the signal.

The document is an individual submission, not an adopted standard. The IETF datatracker notes that it has no formal standing in the standards process, and the draft itself expires on May 3, 2027, unless it is revised.

Who is in, and when

The four companies did not pick a common start date. Here is what each one has committed to, based on their own posts and the reporting below.

OpenAI’s post called the move “voluntary” and said operators previously had to “identify individual crawlers and maintain separate rules for each and every provider.” It promised a further update once it implements the signal and urged other AI providers to honor it.

Google’s Search Central post, signed by Search Advocate John Mueller, said that when a site opts out with the shared token and does not mention Google-Extended, Google plans to read it as a Google-Extended block. Google-Extended is the training control Google introduced in 2023. Google also said the plan is consistent with the EU’s Code of Practice for general-purpose AI models under the AI Act, which treats robots.txt as a primary way to state machine-readable preferences.

Training versus grounding

The biggest practical wrinkle is the difference between training a model and “grounding,” where an AI assistant pulls live web pages into an answer and links to them. PPC Land’s analysis of Google’s documentation found that Google-Extended governs both Gemini training and grounding in Gemini Apps and Vertex AI, so a shared-token opt-out could also remove a site from those cited answers. Google’s post says sites that explicitly allow Google-Extended remain eligible to appear and receive clicks through grounding in Gemini Apps.

Microsoft took the opposite route and split the two. Its Bing blog post introduces a page-level grounding setting with a new “DisallowExtended” option that keeps content in Microsoft’s own products, such as Copilot, while excluding grounding services it sells to other companies. Microsoft said the choices do not change ranking in standard Bing results and warned that “neither choice guarantees inclusion, traffic, or payment.” Its timeline runs from Dec. 15, 2026, when transition settings open in Bing Webmaster Tools, to March 1, 2027, after which the older NoArchive and NoCache tags no longer count as AI preferences.

Why publishers are watching

The announcement lands amid a long fight over whether AI companies may train on published work. The Verge noted that publishers have turned to tools such as the Really Simple Licensing standard, which lets sites charge crawlers, and Cloudflare’s default blocking of AI bots. Meta communications chief Andy Stone told the outlet that “creating a common signal that conveys such preferences to multiple providers at once is an important step,” but Meta did not commit to reading it. Amazon and Apple were not among the companies announcing support.

The signal has no enforcement mechanism. Under the draft, the preference that counts is the one in a site’s robots.txt when a page is retrieved, and the text does not address material gathered earlier. Courts are still sorting out the legal side, as the appeals ruling in Thomson Reuters v. Ross showed. Meanwhile the same companies keep shipping models trained on web data, from Google’s Gemini 4 Carbon to OpenAI’s GPT-6.1 Terra.

Sources

• IETF: draft-kelley-aipref-training-preference-signal-00 (Oct. 30, 2026)
• OpenAI: Making it easier for websites to express AI training preferences (Oct. 30, 2026)
• Google Search Central Blog: Supporting a shared AI training control for websites (Oct. 30, 2026)
• Microsoft Bing Blog: Your content, your choice (Oct. 30, 2026)
• The Verge: The biggest names in AI are backing a new way to opt out of scraping (Oct. 30, 2026)
• PPC Land: Google, Microsoft, OpenAI and Anthropic back one AI training opt-out (Oct. 31, 2026)