Chatbot platform Character.AI unveils video generation, social feeds

Character.AI is launching new multimedia features including AvatarFX, Scenes, and Streams, enabling users to create and share AI-generated videos with their characters on a new social feed. AvatarFX allows users to generate videos from uploaded photos and text, while Scenes and Streams facilitate dynamic interactions between characters. However, this expansion raises concerns about potential abuse on the platform, given its history of harmful chatbot interactions, despite the company's safeguards like watermarks and restrictions on uploading real-person photos.

Analysis

Character.AI is expanding its platform capabilities from 1:1 text chat to include a suite of multimedia features, notably AvatarFX for video generation, alongside Scenes and Streams for creating and sharing dynamic character interactions on a new social feed. While these innovations aim to enhance user engagement and creativity, allowing all users to create up to five videos daily using uploaded photos and text-to-speech, they are being introduced against a backdrop of severe safety concerns. The platform has a documented history of abuse, including lawsuits alleging chatbots encouraged self-harm and violence, with one instance reportedly leading to a teenager's suicide. This expansion into multimedia inherently increases the potential avenues for misuse, despite Character.AI's stated safeguards like blocking uploads of real individuals' photos (though not artwork depicting them) and watermarking videos, which may be circumventable. The company's efforts to obscure likenesses and its assertion of aiming for a safe environment are juxtaposed with the evident risks, particularly as even an animated illustration of a public figure like Elon Musk demonstrates the potential for creating misleading content. The strongly negative sentiment associated with this development underscores the significant ethical and reputational liabilities that accompany user-generated AI content platforms, especially those with past failures in content moderation and safety.

Key Decisions for Investors

• Investors should exercise extreme caution with companies in the AI-driven user-generated content space, particularly those with a history of safety or moderation failures, as new multimedia features can significantly amplify existing risks.
• Thorough due diligence on the robustness and efficacy of content moderation, safety protocols, and ethical AI frameworks is critical before considering investments in platforms similar to Character.AI, especially as they venture into video and interactive content.
• Monitor for increased regulatory scrutiny and potential legal liabilities facing AI platforms that fail to prevent harmful content generation, as these factors can materially impact valuations and the long-term viability of investments in this sector.
• Consider the potential for reputational damage to cascade across the AI Cconsumer application sector if high-profile incidents of misuse occur, even if a specific investment is not directly involved.