About CrawlPact

CrawlPact audits and monitors what a website's own declared signals — robots.txt, meta robots tags, X-Robots-Tag headers, llms.txt, RSL, and Content Signals — say about AI crawler access, and reports that honestly against a versioned, source-backed registry of documented AI crawlers.

What CrawlPact is not

CrawlPact does not use an AI model to produce its results. Every finding traces to a deterministic parser and rules engine evaluating your site's own declared files against a registry entry with a cited official source — see methodology. CrawlPact also does not modify your website, and it does not guarantee that any crawler obeys a declared instruction — see limitations for what a result can and cannot prove.

CrawlPact is also not a web application firewall, a live crawler blocker, a server-log analytics service, or a general-purpose SEO crawler — it reads and reports on declared public policy files, it does not sit in front of your traffic or process your server logs. See the full list on limitations.

Principles

• Vendor neutral. The crawler registry is built from each operator's own published documentation, not any single operator's product interests.
• Evidence based. Every finding cites the specific matched rule, line number, and source behind it — not a summary judgement without a trail back to the evidence.
• Honest about uncertainty. A crawler CrawlPact cannot verify against an official source is marked unverified, not guessed. An unimplemented or disabled feature says so, rather than fabricating a result.

Operator

CrawlPact operates the CrawlPact service.

Questions

See security, privacy, status, and contact for operational, privacy, security-reporting, and content-correction details, or run a free audit to see the methodology applied to a real domain.