CrawlPactMeta-ExternalAgent
- User-agent token
- Meta-ExternalAgent
- Purpose
- Training
- Status
- active
- Last verified
- 2026-07-01
Meta-ExternalAgent is documented by Meta as a crawler used to gather content for training AI
models and for improving Meta’s AI products and features.
Site-owner controls
Disallowing Meta-ExternalAgent affects only Meta’s AI-training use of this content. It does not
affect Meta-WebIndexer (search), Meta-ExternalAds (advertising validation), or
Meta-ExternalFetcher (user-triggered agent fetches) — Meta documents each as its own distinct
token, and CrawlPact evaluates them independently. See /limitations for what a
robots.txt rule can and cannot guarantee.
Distinguishing from Meta’s other crawlers
Meta operates several distinctly named crawlers for different purposes (link previews,
indexing, and AI training). CrawlPact’s registry tracks Meta-ExternalAgent specifically as a
training-purpose crawler; other Meta tokens are evaluated separately as the registry expands.
Example robots.txt configuration
To disallow Meta-ExternalAgent specifically, without affecting any other crawler:
User-agent: Meta-ExternalAgent
Disallow: /If no dedicated User-agent: Meta-ExternalAgent group exists in a domain's robots.txt, this crawler falls back to whatever the wildcard User-agent: * group says (RFC 9309) — see robots.txt syntax basics for how group selection works.
Verified against the source above as of 2026-07-01 — see how CrawlPact verifies crawler information.
See how this applies to your own site
Run a free audit to check your declared AI crawler policy against your own domain, or use the AI crawler checker to check this one crawler specifically.
Audit a domain