OpenAI’s new deepfake machine, Sora, has proven that artificial intelligence is alarmingly good at faking reality. The AI-generated video platform, powered by OpenAI’s new Sora 2 model, has churned out detailed (and often offensive or harmful) videos of famous people like Martin Luther King Jr., Michael Jackson, and Bryan Cranston, as well as copyrighted characters like SpongeBob and Pikachu. Users of the app who voluntarily shared their likenesses have seen themselves shouting racial slurs or turned into fuel for fetish accounts.
On Sora, there’s a clear understanding that everything you see and hear isn’t real. But like any piece of social content, videos made on Sora are meant to be shared. And once they escape the app’s unreality quarantine zone, there’s little protection baked in to ensure viewers know that what they’re looking at isn’t real.
The app’s convincing mimicry doesn’t just run the risk of misleading viewers. It’s a demonstration of how profoundly AI labeling technology has failed, including a system OpenAI itself helps oversee: C2PA authentication, one of the best systems we have for distinguishing real images and videos from AI fakes.
C2PA authentication is more commonly known as “Content Credentials,” a term championed by Adobe, which has spearheaded the initiative. It’s a system for attaching invisible but verifiable metadata to images, videos, and audio at the point of creation or editing, appending details about how and when it was made or manipulated.
OpenAI is a steering committee member of the Coalition for Content Provenance and Authenticity (C2PA), which developed the open specification alongside the Adobe-led Content Authenticity Initiative (CAI). And in fact, C2PA information is embedded in every Sora clip — you’d just probably never know it, unless you’re the type to pore over some brief footnotes on a meager handful of OpenAI’s blog posts.
This is the label that’s supposed to appear on AI-generated or manipulated videos uploaded to YouTube Shorts, but it only applies to content around sensitive topics. Image: YouTube
C2PA only works if it’s adopted at every step of the creation and posting process, including being clearly visible to the person viewing the output. In theory, it’s been embraced by Adobe, OpenAI, Google, YouTube, Meta, TikTok, Amazon, Cloudflare, and even government offices. But few of these platforms use it to clearly flag deepfake content to their users. Instagram, TikTok, and YouTube’s efforts are either barely visible labels or collapsed descriptions that are easy to miss, and they provide very little context if you actually were to spot them. And for TikTok and YouTube, I’ve never once encountered them myself while browsing the platforms, even on videos that are clearly AI-generated, given the uploader has likely removed the metadata or not disclosed their origins.
Meta initially added a small “Made by AI” tag to images on Facebook and Instagram last year, but it later changed the tag to say “AI Info” after photographers complained that work they edited using Photoshop — which automatically applies Content Credentials — was being mislabeled. And most online platforms don’t even do that, despite being more than capable of scanning uploaded content for AI metadata.
C2PA’s creators insist they’re getting closer to widespread adoption. “We’re seeing meaningful progress across the industry in adopting Content Credentials, and we’re encouraged by the active collaboration underway to make transparency more visible online,” Andy Parsons, senior director of Content Authenticity at Adobe, said to The Verge. “As generative AI and deepfakes become more advanced, people need clear information about how content is made.”
Yet after four years, that progress is still all but invisible. I’ve covered CAI since I started at The Verge three years ago, and I didn’t realize for weeks that every video generated using Sora and Sora 2 has Content Credentials embedded. There’s no visual marker that alludes to it, and in every example I’ve seen where these videos are reposted to other platforms like X, Instagram, and TikTok, I have yet to see any labels that identify them as being AI-generated, let alone provide a full accounting of their creation.
... continue reading