I hope everyone is having a safe 4th of July, especially my fellow Americans celebrating our country's birthday.

I wanted to drop you a quick note about a new Cloudflare setting that is available to all levels including the free tier that I use.

It is a bad bot blocker.

Let me back up a second. 

Most of us have a robots.txt file or something similar that is in our websites root directory, and it tells bots what they can and cannot do. 

Like the Google bots can index certain pages but not others. That is a good bot. One that adheres to the rules you spell out in that Robot.txt file.

Well, not all bots are good ones. 

Some, like these newer AI bots, will scrape websites even though you might have a robot.txt file in place to tell these bots to stay out of your stuff.

Okay, back to this new Cloudflare feature.

When you log into your Cloudflare account, go to your site (this feature is site-specific) settings, in the left sidebar, click on Security then bots. The new feature called Block AI Scrapers and Crawlers. Turn that on and your site is now safe from getting scrapped by those bots.

Cloudflare is constantly adjusting for new AI bots that pop up so your sites content will remain yours and not become training content for that particular AI tool.