Arena Group, BuzzFeed, USA Today Co, Vox Media join RSL’s AI content licensing efforts

The RSL Collective now has over 50 partners, co-founder Doug Leeds told Digiday. People Inc., Ziff Davis, Yahoo, Reddit, Medium and Quora had already joined the RSL Collective when it launched in September. The partnerships are non-exclusive.

RSL’s mission is to stop AI crawlers from scraping publishers’ sites without compensation or permission. The idea is that publishers and other digital companies will come together to define, and automate licensing terms for their content. Publishers can add machine-readable terms to their robots.txt files through the RSL protocol, including licensing, usage and royalty terms — rather than just the traditional “allow” or “disallow” language telling AI bots if they can scrape their sites. 

Publishers can ask AI companies for a pay-per-crawl or pay-per-inference fee through the RSL standard, for example. The RSL Collective is working with the CDN company Fastly to allow AI bots to scrape websites if they’ve agreed to license content.

“As USA Today Co. focuses on clear attribution and compensation of our valued content, we are evaluating every opportunity to ensure fair value, not just for the USA Today Network, but for the industry at large as we seek to sustain quality reporting and trusted journalism,” Lark-Marie Anton, USA Today Co.’s chief communications officer, said in an email statement.

RSL isn’t the first to try to do this. The IAB Tech Lab has its own working group of publishers and tech companies, who meet weekly to develop what it’s calling the Content Monetization Protocols (CoMP) framework. Fastly is also an active partner in the IAB Tech Lab working group. Meanwhile, in September, CDN company Cloudflare added a Content Signals Policy standard to update robots.txt — letting publishers communicate how they do and don’t want AI crawlers like Google’s to use their content once it’s scraped — in addition to the AI bot blocking tool it launched this summer.

The IAB Tech Lab and RSL aren’t competing with each other though. Leeds said they are collaborating on how each standard can dovetail and complement the other. Details on how are yet to be sketched out. 

But here’s the catch: none of these licensing standards have real enforcement behind them. They only work if AI companies honor the rules. In theory, these standards give publishers leverage, but only if the AI companies agree to respect the terms.

The idea is that this collective effort, backed by major digital media companies, will create enough pressure that AI companies will have to comply.

Eric Aledort, svp of partnerships and business development at Arena Group, acknowledged that without a way to enforce AI companies to comply with frameworks like RSL’s, their participation in the organization’s efforts might not amount to much. But they were impressed by RSL’s approach to the existential problem of AI crawlers scraping their content for free, as well as the large digital publishers that had already backed RSL.

“We support anything that will help publishers survive,” Aledort said. “It seemed crazy to say no. There’s no downside to participating.”

IAB Tech CEO Anthony Katsur has stressed that it’s in the interest of the LLMs not to have to negotiate with thousands of different sites and risk being blocked too. He expects that in time, this will also prove an additional incentive for LLMs to play ball. 

Leeds agrees it’s a win-win. “If [an AI company] wanted to license everything, [they’d] have to… pick up the phone and find out who their business contact is, and then get to the lawyers, talk to the lawyers and draft the contract. And that is impossible to do [at scale]. It is not impossible to do if you have a blanket license and a collective rights organization,” he said. He compared the arrangement to digital rights organizations like the music rights group ASCAP, which collects licensing fees and shares them with members.

Leeds isn’t fazed by the fact AI companies are yet to comply with the RSL standard. The standard, he said, can help lower the cost and friction of licensing content at scale for AI companies. But he punted off enforcement to technical, legal and regulatory organizations, such as CDNs, lawsuits and government bodies.

“AI companies can have the ability to get all the content they need from wherever they need it with one agreement,” Leeds added. “That product doesn’t exist right now. It can’t just be the U.S. It has to be a global footprint.”

RSL is now focused on getting more buy-in from publishers, particularly in international markets like Europe and Asia, before approaching AI companies. “Our goal right now is really on the supply side and getting that scale before we go to the demand side of this. But that will come soon,” said Leeds.

The RSL Collective has a steering committee made up of its publisher partners, who are developing a licensing agreement that they plan to publish publicly by the end of this year, Leeds said. 

RSL is a nonprofit organization founded by Leeds, who is a former CEO of IAC Publishing  and Ask.com, and Eckart Walther, who is the co-creator of the Really Simple Syndication (RSS) standard.

Because it’s a nonprofit, no one working for RSL is getting paid yet. As the licensing agreements take shape, royalties will come in, and the RSL Collective will take a cut of those, Leeds said. The organization recently hired an economist to work out the revenue model, which will likely be a percentage of revenue, with some minimum thresholds – similar to models like ones set by Spotify and Apple, he added.

It’ll really come down to the value publishers provide to AI companies. “Taylor Swift does not get paid the same every time her song gets played as the garage band down the street,” Leeds said.