Google

Google-Extended

User-agent token: Google-Extended
Purpose: Training
Status: active
Last verified: 2026-07-30

Google-Extended is a control token, documented by Google, that lets a website opt out of having its content used to train Gemini and Vertex AI generative APIs — without affecting Google Search indexing, which is governed separately by Googlebot.

Why this one is easy to get wrong

Disallowing Googlebot blocks Search indexing entirely. Disallowing only Google-Extended leaves Search indexing intact while opting out of generative-model training use. Confusing the two is one of the most common policy conflicts CrawlPact’s conflict detector looks for.

Example robots.txt configuration

To disallow Google-Extended specifically, without affecting any other crawler:

User-agent: Google-Extended
Disallow: /

If no dedicated User-agent: Google-Extended group exists in a domain's robots.txt, this crawler falls back to whatever the wildcard User-agent: * group says (RFC 9309) — see robots.txt syntax basics for how group selection works.

Official source: https://developers.google.com/search/docs/crawling-indexing/google-common-crawlers

Verified against the source above as of 2026-07-30 — see how CrawlPact verifies crawler information.

Related guides

Google-Extended vs. Googlebot: what each one actually controls
A decision guide for separating AI-training opt-out from Search indexing when configuring robots.txt for Google's crawlers.

Related crawlers

See how this applies to your own site

Run a free audit to check your declared AI crawler policy against your own domain, or use the AI crawler checker to check this one crawler specifically.

Audit a domain