Stop Bad Bots from Crawling Your Website

Posted on March 21, 2019 at 10:26 am

A robots.txt example to block bad bots from crawling your website:

User-agent: *
Allow: /
 
User-agent: CuteStat
Disallow: /
 
User-agent: AhrefsBot
Disallow: /
 
User-agent: SemrushBot
Disallow: /
 
User-agent: GrapeshotCrawler
Disallow: /
 
User-agent: WebsiteOutlook
Disallow: /
 
User-agent: MJ12bot
Disallow: /

Receive updates via email

Other Posts

Updated Posts