San Francisco Calls on Apple and Google to Remove AI ‘Nudify’ Applications from Their App Stores

Apple and Google have received orders to eliminate apps that can “nudify” or “undress” individuals, along with directives to cease profiting from such harmful technologies, as detailed in cease-and-desist letters obtained by WIRED.

On Thursday, San Francisco city attorney David Chiu dispatched legal notices to Apple and Google, demanding the removal of 13 face-swapping applications from their app stores. These apps enable users to generate AI-created nonconsensual nude images. The letters assert that the tech giants should stop “aiding and abetting” the sale of explicit deepfake imagery and “sever” ties with the app developers involved.

“Creating non-consensual intimate images is illegal, harmful, and absolutely unacceptable,” Chiu tells WIRED. The city attorney, whose office has previously targeted 16 well-known deepfake websites legally, indicates that Apple and Google may have “made millions of dollars in fees” from nudification apps and emphasizes the need for improved moderation processes to prevent them from being listed initially.

“These companies are responsible for ensuring that the apps on their platforms do not facilitate sexual abuse,” Chiu states. The legal letters highlight that California’s laws prohibit support for services generating deepfake pornography. The apps utilize in-app purchases, from which the tech companies earn a percentage, according to the letters. “It must stop that some of the largest and most established technology companies in the world are allowing this.”

Researchers have repeatedly identified and reported apps in Apple’s App Store and Google’s Play Store that enable users to create sexual images through AI—some of which are rated appropriate for children. While new regulations aim to combat the rise of explicit deepfakes online, tech and social media companies continue to direct millions towards this harmful technology.

Apple and Google maintain developer policies that ban pornography, abuse, and harassment on their platforms. They have previously removed numerous nudify and deepfake applications following alerts from researchers and reporters.

Google spokesperson Dan Jackson informs WIRED that the company has removed “hundreds” of apps featuring nudifying characteristics for violating policies, including the five Android apps identified by Chiu’s office, among other measures to limit their accessibility.

“Google Play prohibits apps containing sexual content, and we continually take proactive measures to detect and eliminate apps with harmful content,” Jackson states. “When violations are reported, we investigate and act quickly, leading to the suspension of hundreds of offending apps and the restriction of related search terms like ‘nudify’ in our store.”

Apple did not offer a comment before publication.

In the past five years, a profitable range of deepfake “nudification” technology has surfaced online—most notably with xAI’s Grok generating millions of sexualized images in January. An array of apps, websites, and bots allow users (predominantly men) to upload images of people (mostly women and girls) and digitally “strip” clothing or place them in explicit sexual contexts.

Creating sexual deepfakes often requires just a reference photo and a few clicks, with some results available within seconds. As generative AI technology has advanced, images and videos have become increasingly realistic, with services either offering some results for free or charging minimal fees for the harmful content. Previous investigations by WIRED and Indicator Media have revealed cases in at least 90 schools where deepfake sexual abuse images involving minors have been generated.