is a news writer who covers the streaming wars, consumer tech, crypto, social media, and much more. Previously, she was a writer and editor at MUO.
Posts from this author will be added to your daily email digest and your homepage feed.
An open licensing standard that aims to make AI companies pay for the content they vacuum up across the web is now an official specification. Really Simple Licensing 1.0 — or RSL for short — gives publishers the ability to dictate licensing and compensation rules to the web crawlers that visit their sites.
The RSL Collective announced the standard in September with backing from Yahoo, Ziff Davis, and O’Reilly Media. It’s an expansion of the robots.txt file, which outlines the parts of a website a web crawler can access. Though RSL alone can’t block AI scrapers that don’t pay for a license, the web infrastructure providers that support the standard can — a list that now includes Cloudflare and Akamai, in addition to Fastly.
RSL’s 1.0 release lets publishers block their content from AI-powered search features, like Google’s AI Mode, while maintaining a presence in traditional search results. Currently, Google doesn’t give websites an individual option to opt out of AI-powered features without booting them out of traditional search, too.
“RSL provides exactly that missing layer,” RSL Collective cofounders Doug Leeds and Eckart Walther say in an emailed statement to The Verge. “Using RSL, Google can respect a publisher’s preferences at the use case level, which means a publisher can stay fully available in traditional search, while opting out of AI training, grounding, or generative answers.”
Google is currently facing an investigation from the European Commission, which is looking into whether the company has violated antitrust policies by using web publishers’ content in AI search features “without offering them the possibility to refuse such use of their content.”
The RSL Collective says more than 1,500 media organizations and brands now support RSL. In addition to Reddit, Quora, WikiHow, Stack Overflow, and Medium, publishers like The Associated Press, Vox Media (The Verge’s parent company), The Guardian, Slate, BuzzFeed, and Men’s Journal publisher Arena Group have also endorsed the standard.
“With this release, and the support for it across the internet ecosystem, RSL 1.0 becomes the expected and trusted way to communicate how content may be used in AI systems, giving those signals real weight in both practice and legal interpretation,” Leeds says.
The RSL Collective also worked with the Creative Commons to add a new “contribution” payment option for nonprofit organizations and individuals behind the webpages, code repositories, and datasets that make up “the shared pool of freely available knowledge and creative work on the internet.”