Meta sues maker of explicit deepfake app for dodging its rules to advertise AI ‘nudifying’ tech

Meta is suing Hong Kong-based Joy Timeline HK Limited, the maker of the CrushAI app, for allegedly circumventing its advertising rules to promote sexually explicit deepfakes, with over 87,000 violating ads run on Meta platforms. Meta claims the app maker created a network of fake business accounts to bypass ad restrictions, targeting users in the US, Canada, and Europe. The lawsuit follows increased scrutiny and pressure on Meta to address the proliferation of AI-generated non-consensual explicit images and the company's efforts to enhance its detection technology and collaborate with other tech platforms to combat the issue.

Analysis

Meta Platforms (META) is actively addressing the misuse of its advertising services by filing a lawsuit against Joy Timeline HK Limited, the Hong Kong-based creator of the CrushAI app, for allegedly running over 87,000 ads that violated its policies by promoting non-consensual, sexually explicit deepfake generation. This legal action, filed in Hong Kong district court, details how the app maker purportedly circumvented Meta's rules by creating a network of at least 170 business accounts and utilizing over 135 Facebook pages to display ads, primarily targeting users in the United States, Canada, Australia, Germany, and the United Kingdom. Meta has incurred $289,000 in costs related to investigation, regulatory response, and enforcement against this specific app maker. The lawsuit arises amid heightened public and regulatory scrutiny concerning the proliferation of AI-generated non-consensual explicit images, with reports indicating that CrushAI derived 90% of its traffic from Meta's platforms. This situation has drawn attention from lawmakers, including Senator Dick Durbin, questioning Meta's oversight. In response to these challenges, Meta states it has developed new technology to identify such ads, even if they lack overt nudity, by training automated systems on associated terms and phrases, and is collaborating with other tech platforms through the Lantern program to share information on abusive apps. This legal and technological response underscores Meta's ongoing battle against sophisticated actors who exploit its platforms, a challenge compounded by its previous adjustment of automated content removal systems to focus on 'high-severity' violations.

Key Decisions for Investors

• Investors should monitor Meta's legal expenses and the potential for reputational impact arising from the misuse of its advertising platform for services generating non-consensual explicit content, and the subsequent enforcement actions.
• The effectiveness of Meta's newly developed ad detection technologies and its collaborative efforts through initiatives like the Lantern program will be crucial indicators of its ability to mitigate future platform abuse and the associated operational and financial risks.
• Close attention should be paid to the evolving regulatory environment concerning AI-generated content and platform accountability, such as the Take It Down Act, as increased legislative pressure could impose further compliance burdens and operational changes on Meta.