A website owner reads the headlines on the last day of October. Google will let her opt out of AI training. Microsoft is splitting its AI controls in two. She has years of articles, product guides and case studies on her site, and a vague worry that someone is using them to build products she’ll never see a penny from. Her first instinct is to open the robots.txt file, paste in a new line and be done with it.
That instinct is understandable, and it could cost her more than she expects. The new controls don’t all do the same thing, and some of the ways of saying no can quietly remove a business from the AI answers that customers increasingly use to find and compare suppliers. Saying no to training isn’t the same as saying no to being cited, and neither is the same as leaving search.
The decision matters because content is an asset. For a publisher it’s the product. For a brand it’s the way customers learn what you do and why to trust you. Whether AI systems can learn from it, quote from it or both is a strategic choice with trade-offs on each side, not a switch to flip on a whim.
The timing is helpful and a little deceptive. Most of what was announced on October 30 doesn’t take effect until 2027, so there’s no emergency. But some preparation has to happen this year, and the businesses that understand the options now will make better decisions than those who scramble in the spring.
In this post we’ll explain what Google and Bing announced, the three separate decisions hiding inside “AI opt-out”, the trade-offs of saying no, and a practical plan. We’ll finish with a checklist and answers to common questions.
What Google Announced
On October 30, 2026, Google said in a Search Central blog post that it plans to support a shared AI training control in robots.txt. The control is based on work from the AI Preferences working group at the Internet Engineering Task Force, and it builds on Google-Extended, the AI training control Google has offered since 2023.
Google says the aim is “a shared AI training control in robots.txt.”
The idea is a single policy token that lets a site state its training preferences across multiple AI providers in one step, instead of maintaining separate rules for each. If a site uses the token to opt out and doesn’t separately specify Google-Extended, Google plans to treat that as though the site had disallowed Google-Extended. The mechanism comes from an Internet-Draft published the same day, an individual proposal authored by Kevin Kelley of Anthropic.
It’s a plan, not a live feature. Google’s blog post says it aims to support the control in 2027, and its crawler documentation says early 2027. Google-Extended continues to work in the meantime, and Google’s documentation says using it has no effect on a site’s inclusion or ranking in Google Search.
What Bing Announced
Microsoft went further on the same day. In a post from Bing’s product team, it said it will adopt the shared training preference, extend it to individual pages and add a separate AI grounding preference.
Bing now separates AI training from AI grounding, so a page can be closed to model training while still informing AI-generated answers, and Microsoft will begin honouring the new preferences on February 1, 2027.
The training preference can be set in robots.txt or at page level, and a page-level setting overrides robots.txt for that page. The grounding preference is page-level only and has a middle option, called DisallowExtended, that allows grounding in Microsoft’s own tools, such as Copilot and Bing’s AI Mode, but not through grounding services Microsoft supplies to other businesses. Microsoft says either grounding restriction will limit a brand’s visibility in AI answers and won’t affect its listing in Bing search results. It also notes it was the first company to offer publisher controls for AI training and grounding, back in 2023, through its NoArchive and NoCache tags.
Three Different Decisions
The most useful thing to take from all this is that “opting out of AI” is really three separate choices.
Training. Whether your content may be used to build or improve AI models. This is what the shared robots.txt control and Google-Extended address.
Grounding. Whether your content may be used to help an AI system compose a live answer, for example when an assistant looks up a page and cites it. Bing now has a dedicated setting. At Google, grounding for the Gemini app is tied to Google-Extended, while AI Overviews and AI Mode are controlled by a separate setting in Search Console.
Search. Whether your pages appear in ordinary search results. Neither company links these new controls to search listings. Google says Google-Extended doesn’t affect Search inclusion or ranking, and Microsoft says its restrictions don’t affect Bing listings.
Treating the three as one is the easiest way to get a result you didn’t intend.
The Trade-Offs of Saying No
Each “no” has a price, and it’s worth knowing the price before you pay it.
Blocking training may be easy to justify if your content is your product. But it doesn’t, on its own, stop AI answers citing you through other routes, and it may not be what you want if you rely on being recommended.
Restricting grounding is where visibility is most clearly at stake. Microsoft says it will limit your brand’s presence in AI answers. On Google’s side, one analysis points out that Google-Extended covers both Gemini training and Gemini app grounding in a single token, so a site can’t refuse Gemini training while staying in Gemini app answers. The same analysis warns that pasting the shared token into robots.txt without a Google-Extended rule could end Gemini app grounding once Google reads it, perhaps without the owner realising. That’s one analyst’s reading, so confirm the details in Google’s documentation before acting.
There’s a similar trade-off outside Google and Microsoft. OpenAI’s documentation says sites opted out of its search crawler won’t be shown in ChatGPT search answers, and OpenAI has said a common training signal wouldn’t replace its existing crawler controls. Where you set the line depends on how much you value being cited.
It’s also worth remembering that these signals are preferences, not locks. Robots.txt and similar tags rely on crawlers choosing to comply, so they express your wishes without being able to enforce them.
A Timeline to Mark in Your Calendar
- October 30, 2026: Google and Microsoft announce their controls.
- December 15, 2026: Bing’s temporary grounding setting opens in Bing Webmaster Tools for sites that currently use NoArchive or NoCache.
- February 1, 2027: Microsoft begins honouring the new training and grounding preferences.
- March 1, 2027: NoArchive and NoCache stop counting as AI training or grounding preferences, and pages without a new grounding setting default to allowing grounding until recrawled.
- 2027, and early 2027 in its crawler documentation: Google aims to support the shared training token.
- September 30, 2027: Bing’s temporary grounding settings expire.
How to Decide
Start with what the content is for. If it’s your product, such as journalism, research or paid material, protecting it from training may matter more than being cited. If it’s marketing, such as guides, case studies and product pages, being cited and recommended may be the whole point.
Then think about each of the three decisions separately and write down your answer. Some businesses will block training but allow grounding. Others will allow both. Others will treat different sections of their site differently, with premium material restricted and public guides open.
Consider your legal position too. Google notes that its approach is consistent with the EU Code of Practice for general-purpose AI model providers, which recognises robots.txt as a way to express machine-readable preferences. Our EU AI Act checklist covers related obligations. This isn’t legal advice, so take advice if you have licensing or contractual commitments.
A Practical Plan
Audit what’s already there. Look at your robots.txt for existing AI-related rules, and check whether any pages use NoArchive or NoCache tags. You may have set preferences years ago and forgotten.
Decide by content type. Separate the material you want cited from the material you want protected, and decide each on its own.
Prepare the implementation. Training preferences can go in robots.txt or on individual pages. Grounding preferences, for Bing, go on each page. Our web and e-commerce team can help with the technical changes, including page-level tags across large sites.
Handle the Bing deadline. If you rely on NoArchive or NoCache, you’ll need both the new page-level setting and the temporary setting in Bing Webmaster Tools before March 1, 2027. Check that your site is verified in Bing Webmaster Tools before December 15.
Keep your existing rules. Don’t remove current crawler controls just because a shared standard is coming, since the new signal doesn’t replace them everywhere.
Review your wider AI strategy. Training controls sit alongside choices about AI Overviews, AI Mode and AI answers more generally. We covered that side in our post on Google’s AI Overviews opt-out, and our digital marketing team can help you weigh visibility against protection.
Monitor what happens. Watch your logs and analytics for changes after you act, bearing in mind the attribution gaps we described in our post on AI referral tracking.
An AI Content Controls Checklist
- Reviewed your robots.txt for existing AI-related rules
- Checked pages for NoArchive and NoCache tags
- Decided separately on training, grounding and search for each type of content
- Noted the key dates, including December 15, 2026, February 1, 2027 and March 1, 2027
- Confirmed your site is verified in Bing Webmaster Tools
- Checked Google’s documentation before adding the shared token, especially if you want to stay in Gemini app answers
- Kept existing crawler controls in place
- Asked legal advisers about any licensing or contractual commitments
- Set a date to review the decision and its effect on visibility
Frequently Asked Questions
What did Google announce on October 30, 2026? Google said it plans to support a shared AI training control in robots.txt, based on IETF work, and to treat it as a Google-Extended disallow when a site opts out and doesn’t name Google-Extended separately. It aims to support it in 2027.
What is the difference between AI training and AI grounding? Training is using content to build or improve a model. Grounding is using content to help an AI system compose a live answer. Bing now has separate preferences for each.
Will opting out of AI training remove me from Google Search? No. Google says Google-Extended has no effect on Search inclusion or ranking, and Microsoft says its AI restrictions don’t affect Bing listings.
Does Google-Extended control AI Overviews? No. AI Overviews and AI Mode are controlled by a separate setting in Search Console. Google-Extended covers Gemini training and Gemini app grounding.
When do the new Bing controls take effect? Microsoft says it will begin honouring them on February 1, 2027, and NoArchive and NoCache stop counting as AI preferences after March 1, 2027.
Can a crawler ignore these settings? Yes. They’re preferences that depend on voluntary compliance, so they express your wishes but can’t enforce them.
Where to Start This Week
If you only do a few things, begin with these.
- Read your robots.txt. See what AI-related rules exist today and who added them.
- Sort your content. List which parts of your site you’d protect, which you’d want cited and which you’re unsure about.
- Check your Bing setup. Confirm your site is verified in Bing Webmaster Tools, and note whether any pages use NoArchive or NoCache.
- Book a decision meeting. Put a short session in the calendar with marketing, legal and whoever manages the website, and aim to agree a policy before the end of the year.
None of this needs new software. It needs the people responsible for content, legal and the website to talk to each other, which is often the step that gets skipped.
Decide Deliberately, Then Revisit
The new controls are a welcome sign that publishers will get clearer choices over how their work is used. But clearer choices bring clearer responsibility. The worst outcome isn’t opting out or staying in. It’s drifting into a setting by accident, because someone pasted a line into a file without realising what it did.
It’s also a decision that won’t stay fixed. Standards are still being drafted, companies are still announcing their plans, and the balance between protecting content and being found in AI answers will shift as these systems become a bigger part of how people search. A policy you set now should come with a review date. What matters is that it’s a decision your business made on purpose, understood by the people who run its website, and written down.
If you’d like help working through the options for your own site, our digital marketing team and Kilowott Intelligence can help you weigh visibility against protection, our web and e-commerce team can handle the technical changes, and automation can keep the monitoring running. You can see how we approach decisions like this across other brands in our case studies.
