WebDecoyBot is the crawler operated by WebDecoy. It fetches pages for three purposes only: verifying that a customer's WebDecoy install is serving correctly, reading the robots.txt of a site a customer added to their account, and running bot-readiness scans of sites whose owners have verified domain ownership with us. It never scans a site its owner did not ask us to look at.
Every request carries this User-Agent:
Mozilla/5.0 (compatible; WebDecoyBot/1.0; +https://bot.webdecoy.com)
Requests are also signed with HTTP Message Signatures (Web Bot Auth). Our public key is published in the key directory on this host, so any verifier can confirm a request claiming to be WebDecoyBot really came from us. A request with our User-Agent but no valid signature is an impersonation.
WebDecoyBot,
or the * group, is read before any page is fetched. A disallow
ends the scan, even when the site's owner requested it./robots.txt and nothing else, at most
once a day per site, to show a customer what their own file asks of AI
crawlers.You can block WebDecoyBot by user agent or by rejecting requests that fail signature verification. Questions or abuse reports: bot@webdecoy.com.