GoKawiilApple and Google have been sent cease-and-desist letters that demand they remove AI-powered "nudify" apps from their respective app stores.
As reported by Wired, San Francisco city attorney David Chiu ordered the two tech giants to take down a total of 13 apps that can be used to create AI-generated deepfake nude images. The letters appeal to Apple and Google to stop "aiding and abetting" the spread of nonconsensual intimate images, and ask them to stop working with the app developers in question.
This isn't the first time Chiu has taken on deepfake platforms, having previously filed a lawsuit against 16 websites that allow users to turn images of real women and girls into pornography using AI. He told Wired that Apple and Google have likely made millions from in-app payments from the offending apps, and that their stores need better moderation to ensure they don't get approved.
Despite widespread public outcry about deepfakes, it emerged earlier this year, via a report from The Tech Transparency Project (TTP), that nudification apps were frequently making it past Apple and Google's moderators. And it went even further, alleging that some of them were even being actively promoted on the App Store and Google Play.The report also alleged that many of these apps were rated "E" for Everyone, enabling children to download them. Finding the apps, it claimed, is as simple as searching for terms like "nudify" or "undress." This is despite both companies having policies that ban sexual or pornographic material on their platforms.
Google and Apple aren't the only companies wrestling with the proliferation of deepfakes on the web. Meta's Oversight Board recently called on the company to strengthen its protections for ordinary people that are being targeted, with the existing preventative measures seemingly favoring public figures. Elon Musk's xAI has also been the subject of several lawsuits relating to nonconsensual deepfakes.
A Google spokesperson told Wired that the company had deleted "hundreds" of apps with nudification features, five of which had been alerted to them by Chiu. It also reiterated that its policies forbid sexual content and said it takes "swift action" if a reported app is found to be in violation of its rules. Wired added that Apple did not provide a comment ahead of publication.