Sora might have a 'pervert' problem on its hands

OpenAI's new Sora 2 video app is reportedly facing significant content moderation challenges, with users creating niche fetish content using the likenesses of real individuals who opted into 'cameo' features, raising ethical and consent concerns. The article highlights broader issues including unauthorized use of celebrity likenesses and potential interactions between minors and adult content, suggesting OpenAI's current moderation policies are insufficient for these 'gray-area' violations. This situation underscores growing reputational and regulatory risks for OpenAI, as the company navigates complex content governance issues beyond explicit prohibitions like nudity.

Analysis

OpenAI's new Sora 2 video app is encountering significant content moderation failures, with users exploiting the "cameo" feature to generate niche fetish content using real individuals' likenesses. Despite a ban on nudity, the platform is reportedly rife with "belly inflation," "giantess," and "foot worship" videos, raising serious ethical and consent concerns for users who opted into public likeness use. This indicates a critical gap in OpenAI's content filtering beyond explicit sexual material.

This issue extends beyond individual consent, encompassing broader intellectual property and child safety risks. OpenAI's "ask forgiveness, not permission" approach to IP is evident in its initial allowance of celebrity likenesses, leading to complaints from entities like the King family. Furthermore, the platform has facilitated potentially inappropriate interactions between minors and adult content, despite recent announcements of parental controls, highlighting an unclear and reactive policy framework.

The strongly negative sentiment and high market impact score associated with these revelations signal substantial reputational and regulatory risks for OpenAI. The ease with which problematic content can be generated, coupled with a perceived lack of proactive moderation, could deter user adoption, particularly among women, and invite increased scrutiny from regulators. This situation poses a material threat to the platform's long-term viability and investor confidence in the company's governance capabilities.

Key Decisions for Investors

• Monitor OpenAI's response to content moderation failures and any subsequent regulatory actions or legal challenges, as these could significantly impact its valuation and operational freedom.
• Evaluate the effectiveness of OpenAI's announced parental controls and broader content policy updates, as robust governance is crucial for user trust and platform scalability in the generative AI space.
• Assess potential spillover effects on the broader AI industry, as increased regulatory scrutiny or public backlash against Sora 2's issues could lead to a more cautious investment environment for similar AI platforms.