The key facts

  • Apple updated App Store rules: a ban on AI-generated abusive content and 'bait-and-switch' apps with 'nudification' features
  • The company will remove apps that add features not declared during review
  • In July, San Francisco demanded that Apple and Google remove nudification apps

App Store rules update

Over the weekend, Apple tightened its rules for developers, banning apps that use AI to create abusive content, as well as apps that disguise themselves as ordinary programs to pass the App Store review and then turn into 'nudifiers' or bring such features to the forefront. The company updated its wording on AI and user-generated content, requiring developers to take sufficient measures to moderate unwanted materials.

Requirements for user-generated content apps

According to Apple, apps with user-generated content must filter unacceptable materials, give users the ability to report abusive content, and block bad-faith accounts. Apps with hidden or undocumented features are also banned, including those that conceal malicious capabilities — such as 'nudification' — until approval (the bait-and-switch scheme). Apple said it will remove apps that add features that were absent from the version submitted for review. In the legal section of the rules, the company also strengthened its wording banning apps that facilitate human trafficking and child exploitation.

Context and regulatory pressure

Apple reported that last year the review process blocked more than 150,000 apps and removed 193,000 developer accounts due to fraud and abuse. In July, San Francisco authorities demanded that Apple and Google clean their stores of nudification apps: city attorney David Chiu accused the companies of profiting from them through store commissions. In April, a Tech Transparency Project report showed that nudification apps were present in both companies' stores, some of which carried ratings indicating suitability for minors.