Limitations

CrawlPact audits declared policy

CrawlPact reports what a website's public configuration asks crawlers to do (declared policy) and what CrawlPact itself was able to retrieve (observed response). It does not, and cannot, prove what an external crawler actually does (actual crawler behaviour) —robots.txt and related signals are voluntary, unenforced declarations.

CrawlPact is not

• A web application firewall or reverse proxy
• A live crawler blocker
• A server-log analytics service
• A full-site SEO crawler or general-purpose SEO suite
• An AI-search ranking tracker or brand-mention tracker
• A legal service or compliance certificate
• A guarantee that bots will obey instructions or that content will appear in AI answers

Evidence can be incomplete

A target may block CrawlPact's scanner, return an unexpected response, or expose only partial signals. When evidence is insufficient, CrawlPact shows "Incomplete" rather than a misleading score or result.

Registry coverage is not exhaustive

CrawlPact's crawler registry only includes crawlers with a reliable official source and a verified token. An unlisted or newly introduced crawler will not appear in a report until it is verified and published.