WebDecoyBot

WebDecoyBot is the crawler operated by WebDecoy. It fetches pages for three purposes only: verifying that a customer's WebDecoy install is serving correctly, reading the robots.txt of a site a customer added to their account, and running bot-readiness scans of sites whose owners have verified domain ownership with us. It never scans a site its owner did not ask us to look at.

How it identifies itself

Every request carries this User-Agent:

Mozilla/5.0 (compatible; WebDecoyBot/1.0; +https://bot.webdecoy.com)

Requests are also signed with HTTP Message Signatures (Web Bot Auth). Our public key is published in the key directory on this host, so any verifier can confirm a request claiming to be WebDecoyBot really came from us. A request with our User-Agent but no valid signature is an impersonation.

Behavior

Blocking or contact

You can block WebDecoyBot by user agent or by rejecting requests that fail signature verification. Questions or abuse reports: bot@webdecoy.com.