AI companies use bots to scrape the web, in order to gather data to train their models. Anubis is a program designed to block these bots from scraping self-hosted sites.

I'm on a mission to review 1,000 marketing software tools and share my findings with over 100,000 small business owners ...

Internet infrastructure company Cloudflare has placed the control with content creators and publishers for content that is scraped by AI bots.

News publishers are building fences around their content in an effort to cut off crawlers that don’t pay for content.

Starting Tuesday, every new web domain that signs up to Cloudflare will be given the option to allow — or block — AI crawlers.

Cloudflare, a company that runs 20% of the web, just flipped a switch that could end the open internet as we know it, forcing AI companies to pay for the content they’ve been taking for free.