Artificial intelligence chatbots like ChatGPT are making major factual mistakes nearly half the time when asked about current events, according to a large-scale study released today by Europe’s public broadcasters.

The study, coordinated by the European Broadcasting Union (EBU) and based on previous research by the BBC, tested four widely used AI applications that deliver news content — ChatGPT (OpenAI), Copilot (Microsoft), Gemini (Google), and Perplexity.

Across all languages and regions, 45% of their responses contained at least one significant issue, researchers found.

One in five responses included serious factual errors, featuring outdated information or even entirely fabricated details, the study revealed.

Of the four AI tools tested, Google’s Gemini performed worst, with “major issues” in three-quarters of its responses — more than double the error rate of the other systems. Most problems stemmed from Gemini’s “poor performance in accurately citing sources,” according to the report.

A total of 22 public media outlets from 18 countries, mostly European, participated in the research. Between late May and early June, each outlet posed the same set of current affairs questions to the four AI systems, asking them to draw answers from the outlets’ own published content.

Out of 3,000 total responses, outdated information was among the most frequent issues.

In one test, when asked “Who is the Pope?”, ChatGPT (for Finland’s Yle) and Copilot and Gemini (for Dutch broadcasters NOS and NPO) all responded: “Pope Francis.” However, in the hypothetical scenario posed by the study, Pope Francis had already died, and his successor, Leo XIV, had taken office — showing that the AI tools provided obsolete or incorrect information.

In another case, Radio France asked about a supposed “Nazi salute” by Elon Musk during Donald Trump’s inauguration in the United States in January. Gemini replied that the billionaire had made “a right-arm salute.” The chatbot had mistakenly taken a satirical column by humorist Charline Vanhoenacker as factual reporting.

“AI assistants are still not a reliable way to consume information,” concluded Jean-Philippe De Tender, EBU Deputy Director-General, and Pete Archer, Head of AI at the BBC, in their joint statement accompanying the report.

Despite this, AI chatbots are increasingly being used for news — particularly among younger audiences. According to a Reuters Institute global report published in June, 15% of people under 25 use AI tools weekly to get quick summaries of current news.

“This is the first study of its kind and scale,” said Mathieu Beauval, Head of Innovation at Radio France, speaking to the Agence France-Presse (AFP).