San Francisco Cracks Down on AI Deepfake Apps

Major tech platforms are finally facing consequences for allowing AI apps that create fake nude images without consent.

San Francisco City Attorney David Chiu sent cease-and-desist letters to Apple and Google demanding they remove 13 "nudify" apps from their stores. These apps use AI to generate deepfake nude images of real people, often women and girls, without permission.

The letters ask both companies to stop "aiding and abetting" the spread of these intimate images and cut ties with the developers behind them. Chiu told Wired that Apple and Google have likely earned millions from in-app purchases tied to these harmful apps.

This isn't Chiu's first fight against deepfake platforms. He previously sued 16 websites that allowed users to turn photos of real women and girls into pornography using AI.

Despite both companies claiming to ban sexual content, a report from The Tech Transparency Project earlier this year found these apps regularly slip past moderators. Some were even promoted on the App Store and Google Play, and many carried an "E for Everyone" rating that let children download them.

The Bright Side

The legal pressure is already working. Google told Wired it has deleted "hundreds" of apps with nudification features, including five flagged by Chiu.

The company confirmed its policies forbid sexual content and promised "swift action" against rule violations. While Apple didn't comment, the public scrutiny puts both tech giants on notice that their moderation systems need serious improvement.

This action comes as other platforms wrestle with deepfake problems too. Meta's Oversight Board recently called for stronger protections for everyday people targeted by these images, not just public figures.

The San Francisco case represents a turning point where city attorneys are using their legal power to force tech companies to take responsibility. When platforms profit from harmful apps, they can be held accountable for the damage those tools cause.

Stronger enforcement sends a clear message: companies can't hide behind weak moderation systems while earning money from apps that violate people's dignity and privacy.

Real accountability from tech giants can make digital spaces safer for everyone.