In my mailbox today
To whom it may concern:
We’d appreciate your forwarding this to the appropriate contact at [url]
….. we have noticed that you have blocked the Ask.com agent from crawling your site in the robots.txt file.
We definitely understand that granting our crawler access to your site is an act of trust, which we will use with utmost integrity. So we are asking you to reconsider allowing our crawler access to your site. If there is some reason you are blocking us, please share that with us and allow us to address it. Or if there is no reason, then we ask you to go ahead and remove the line in your robots.txt file that is blocking us. Our goal is to have the same access you have granted other search engines.
The line in question is:
Therefore “Jeeves” used to be listed as
robot-owner-name: Leon Brocard
robot-owner-email: [email protected]
robot-purpose: indexing maintenance statistics
robot-useragent: Jeeves v0.05alpha (PERL, LWP, [email protected])
robot-description: Jeeves is basically a web-mirroring robot built as a
final-year degree project. It will have many nice features and is
already web-friendly. Still in development.
robot-history: Still short (0.05alpha)
robot-environment: researchmodified-by: Leon Brocard
I don’t know why the bot was on the “evil list” but it explains why this old site had it blocked.
I had always had my eye open for
…but never just “Jeeves” – but it is definitely an alias.
I thought it was awesome for Ask.com to take the time to crawl these files and send out notes.