2026 started with a horrifying example of generative AI's potential for abuse. Grok, the AI tool from Elon Musk's xAI, was used to undress or nudify pictures of people shared on X (formerly Twitter) at an alarming rate. Grok made 3 million sexualized images over a span of 11 days in January, with approximately 23,000 of those containing images of children, according to a study from the Center for Countering Digital Hate.
Now, competitors like OpenAI and Google are stepping up their security to avoid being the next Grok.
Advocates and safety researchers have long been concerned about AI's ability to create abusive and illegal content. The creation and sharing of nonconsensual intimate imagery, sometimes referred to as revenge porn, was a big problem before AI. Generative AI only makes it quicker, easier and cheaper for anyone to target and victimize people.
On Jan. 14, two weeks into the scandal, X's Safety account confirmed in a post that it would pause Grok's ability to edit images on the social media app. Grok's image-generation abilities are still available to paying subscribers in its standalone app and website. X did not respond to multiple requests for comment.
Most major companies have safeguards in place to prevent the kind of wide-scale abuse that we saw was possible with Grok. But cybersecurity is never a solid metal wall of protection; it's a brick wall that's constantly undergoing repairs. Here's how OpenAI and Google have tried to beef up their safety protections to circumvent Grok-like failures.
Read More: AI Slop Is Destroying the Internet. These Are the People Fighting to Save It
OpenAI fixes image generation vulnerabilities
At a base level, all AI companies have policies prohibiting the creation of illegal imagery, like child sexual abuse material, also known as CSAM. Many tech companies have guardrails to prevent the creation of intimate imagery altogether. Grok is the exception, with "spicy" modes for image and video.
Still, anyone intent on creating nonconsensual intimate imagery can try to trick AI models into doing so.
Researchers from Mindgard, a cybersecurity company focused on AI, found a vulnerability in ChatGPT that allowed people to circumvent its guardrails and make intimate images. They used a tactic called "adversarial prompting," where testers try to poke holes in an AI with specifically crafted instructions. In this case, it was tricking the chatbot's memory with custom prompts, then copying the nudified style onto images of well-known people.
... continue reading