![](https://digiday.com/wp-content/uploads/sites/3/2023/03/ai-thoughts-digiday-gif.gif?w=1030&h=579&crop=1)
This article is a WTF explainer, in which we break down media and marketing’s most confusing terms. More from the series →
Publishers have a new tool in their efforts to limit AI’s threat to their businesses. And it’s from the company behind one of the predominant threats.
In August, OpenAI announced that website owners can now block its GPTBot web crawler from accessing their webpages’ content. Since then, 12% of the 1000 most-visited sites online have done so, according to Originality AI. The list of sites shutting themselves off to OpenAI’s web crawlers includes publishers such as Bloomberg, CNN and The New York Times.
As Digiday has covered, publishers have had a hard time protecting against generative AI tools like ChatGPT sidestepping their paywalls and siphoning their content to inform the large language models. OpenAI’s announcement, however, makes that undertaking much easier.
For those unfamiliar with what a web crawler like OpenAI’s GPTBot is, not to mention how websites are able block their access, check out the explainer video skit below.
More in Media
![](https://digiday.com/wp-content/uploads/sites/3/2025/01/tiktok-aftermath-digiday.jpg?w=439&h=277&crop=1)
How U.K.-based creators took advantage of the U.S. TikTok outage
In the week following the United States’ Jan. 19 TikTok outage — the seven days between Jan. 19 and Jan. 25 — video creators in the U.K. posted 15 percent more content than they did during the seven days leading up to the ban.
![](https://digiday.com/wp-content/uploads/sites/3/2024/09/audio-lifeline-digiday.gif?w=439&h=277&crop=1)
How podcasters are measuring videos’ role in growing incremental audiences
Without a proper way to measure videos’ effectiveness in growing incremental audiences, podcast orgs and publishers are drawing correlations between a podcast’s video audience growth and overall audience growth.
![](https://digiday.com/wp-content/uploads/sites/3/2023/03/netflix-dynamite-digiday.jpg?w=439&h=277&crop=1)
Netflix readies pan-European rollout of its in-house ad tech amid pricing competition
Buyers eye discounts of as much as 20% as Google, The Trade Desk and others join its much-vaunted expansive rollout.