AI assistants make widespread errors about the news, new research shows

A joint study by the European Broadcasting Union (EBU) and the BBC reveals that leading AI assistants, including ChatGPT, Copilot, and Gemini, misrepresent news content in 45% of responses, with 81% containing some form of error. The research, which analyzed 3,000 AI outputs, found significant sourcing issues in a third of responses, notably 72% for Google's Gemini, and accuracy problems in 20% across all platforms. Given that 7% of online news consumers now rely on AI for information, these findings underscore a critical risk to public trust and data integrity, potentially leading to increased regulatory scrutiny on AI developers and impacting the perceived reliability of AI-driven information sources for market analysis.

Analysis

A joint study by the EBU and BBC revealed that leading AI assistants misrepresent news content in 45% of responses, with 81% containing some form of error across 3,000 queries in 14 languages. This extensive research, encompassing platforms like ChatGPT, Copilot, Gemini, and Perplexity, highlights significant reliability concerns in AI-generated news.

Key issues included serious sourcing errors in one-third of responses, with Google's Gemini exhibiting significant sourcing problems in 72% of its outputs, substantially higher than other assistants. Additionally, 20% of all AI responses contained accuracy issues, such as outdated information or factual errors like the incorrect reporting of Pope Francis's status by ChatGPT. While OpenAI and Microsoft acknowledge "hallucinations," and Google welcomes feedback, these findings underscore persistent challenges in factual integrity.

The EBU warns that this widespread misinformation risks undermining public trust and democratic participation, especially given that 7% of online news consumers and 15% of under-25s now rely on AI for news. This critical erosion of trust, coupled with calls for AI companies to improve accountability, suggests a heightened potential for increased regulatory scrutiny on AI developers. The strongly negative sentiment towards GOOG/GOOGL (-0.8) and negative for MSFT (-0.4) reflects investor concern over these reliability issues.

Key Decisions for Investors

• Monitor potential for increased regulatory scrutiny on AI developers (e.g., Google, Microsoft) due to reliability concerns, which could impact product roadmaps and compliance costs.
• Assess the reputational risk and user adoption impact for AI platforms, particularly for Google's Gemini given its high sourcing error rate, as public trust erodes.
• Evaluate investment opportunities in companies providing solutions for AI accuracy, fact-checking, or robust data integrity, as demand for these technologies is likely to rise.
• Exercise caution and perform enhanced due diligence on businesses whose core operations or content generation heavily depend on AI assistants, considering their exposure to misinformation risks.