Search engine crawler
Yahoo Slurp
Crawlers that index the web for search engines. Blocking them costs rankings.
Last updated
At a glance
| Category | Search engine crawler |
|---|---|
| User agent match | contains "slurp" |
| Executes JavaScript | No |
| Respects robots.txt | Generally yes |
| Identity verifiable | Yes, by reverse DNS |
| IVT class | GIVT (general invalid traffic: declared automation) |
What Yahoo Slurp is
Yahoo Slurp crawls pages to build a search index. It is declared automation of the most ordinary kind: every site on the web is visited by it, and its traffic should be filtered out of your human numbers, never billed to a campaign and never blocked.
Can the name be trusted?
Yahoo Slurp can be verified by reverse DNS: the visiting address must resolve back to a hostname the operator controls, and that hostname must resolve forward to the same address. TrueVisit runs this check on every visit claiming the name. A failed lookup is reported as unverifiable rather than as impersonation, because a DNS timeout is not proof of fraud.
Famous crawler names attract impostors, and a confirmed impostor is one of the few verdicts that carries intent. The routine for catching one is in how to spot fake Googlebot traffic.
Should you block it?
Do not block Yahoo Slurp. Blocking a search engine crawler removes your pages from its index, and no bandwidth saving is worth that. The real risk with a famous crawler name is the opposite one: scrapers impersonate it to walk past filters, which is why verification matters more than blocking here.
How TrueVisit reports it
Visits from Yahoo Slurp appear in your bot report under this exact name, never as an anonymous percentage, and every verdict stores the reason it was reached. Because this client never executes JavaScript, a browser-only tracker cannot see it at all; it is visible through server-side collection, which is how the visits reach your report.