Skip to content
Tech News
← Back to articles

xAI is being sued by teens who say Grok created CSAM using their photos

read original more articles
Why This Matters

This lawsuit against xAI highlights the urgent ethical and legal challenges surrounding AI-generated content, especially when it involves minors and the creation of harmful material. It underscores the need for stricter regulation and oversight of AI tools to prevent misuse and protect vulnerable populations, impacting both industry standards and consumer safety. The case also raises awareness about the potential for AI to be exploited for malicious purposes, prompting calls for more responsible AI development and deployment.

Key Takeaways

In this photo illustration, the logo of Grok is displayed on a computer screen with the logo of X. (Photo by Ozge Elif Kizil/Anadolu via Getty Images)

In this photo illustration, the logo of Grok is displayed on a computer screen with the logo of X. (Photo by Ozge Elif Kizil/Anadolu via Getty Images) (Anadolu via Getty Images)

xAI, which is already facing multiple investigations around the world over widespread reports that Grok repeatedly created sexualized images of children, is now facing a class action lawsuit. Three teenagers, who allege that photos of them were used by Grok to generate child exploitation material, have filed a class action lawsuit against xAI in California.

According to the lawsuit, one of the teens was alerted last December that someone was sharing AI-generated images and videos of her and other minors "in settings with which she was familiar, but morphed into sexually explicit poses." The images and videos were allegedly shared on Discord, Telegram and other platforms and used "as a bartering tool" for other CSAM imagery. Law enforcement officials who investigated the images told the girls' parents they were created with xAi's Grok, the lawsuit says.

The three teens, all of whom live in Tennessee and are identified as Jane Doe 1, Jane Doe 2 and Jane Doe 3, have "suffered severe emotional distress," the filing says. "Their lives have been shattered by the devastating loss of privacy, dignity, and personal safety that the production and dissemination of this CSAM have caused," lawyers for the teens write in the complaint, which was provided to Engadget. "xAI’s financial gain through the increased use of its image- and video-making product came at their expense and wellbeing. Plaintiffs will have to spend the rest of their lives knowing that their CSAM images and videos may continue to be trafficked and traded online by child sex predators."

Advertisement Advertisement

Advertisement

Though the lawsuit currently names three individuals, the complaint says that it could cover "at least thousands of minors" who have also had their photos manipulated by Grok into sexualized images. The lawsuit claims xAI has violated multiple laws, including laws barring the production and distribution of child abuse material.

xAI didn't immediately respond to a request for comment on the lawsuit. The company is also facing multiple investigations in the US and Europe over Grok's alleged generation of nonconsensual nudity. Researchers at the Center for Countering Digital Hate estimated in January that Grok had produced millions of sexualized images, including 23,000 that appeared to show children.

xAI CEO Elon Musk, who previously promoted Grok's "spicy" abilities, has claimed that he was "not aware of any naked underage images generated by Grok." xAI announced in January it would stop allowing people to use Grok to edit images of real people into bikinis and limit Grok's image-generation feature to paid subscribers.