The key facts
- Models answered ~90% of neutral climate questions correctly, but in 8 out of 10 cases changed their position at mild user disagreement
- Accuracy in assessing new scientific evidence was 75%, policy compatibility 33.8% — close to random guessing
- Accuracy drops in languages with less training data: 78.3% in Swahili versus 88.6% in English
The desire to please is built into model design
Chatbots are trained to keep the user engaged for as long as possible, and they are rewarded for polite, non-confrontational and affirming answers, because people like such answers. A report by UNESCO and University College London (UCL), presented at the COP31 climate summit in Antalya (Turkey), calls this sycophantic behaviour, built into the design, and warns that in such a sensitive sphere as government climate policy it could distort decisions.
What the tests showed
The researchers tested up to 33 leading models on nine datasets, in 13 languages and 656 questions about climate policy, checking answers against the En-ROADS scientific simulator. On neutral questions the models answered correctly about 90% of the time. However, when a user challenged an answer or put forward plausibly sounding fabrications, the models quickly gave in: in 8 out of 10 cases they changed their position at the first mild disagreement — up to confirming false claims such as that CO2 is not the main driver of climate change.
Weakness where it is most dangerous
Despite impressive factual recall, the models proved to be an unreliable tool for developing climate policy. When given a new piece of evidence and asked to assess whether it supports or contradicts established climate science, even the strongest models were on average accurate only 75% of the time. They fared even worse at assessing how compatible policies are with one another — a key part of designing climate strategies: average accuracy was 33.8%, only slightly above the 33.3% expected from random guessing. The report therefore cautions policymakers and researchers against using AI models to develop climate policy.
Language inequality and changing disinformation
Accuracy varied markedly across languages: 88.6% in English and comparable results in Spanish, French and Portuguese versus 83.6% in Hindi and Bengali, 81.4% in Urdu and 78.3% in Swahili. According to the report, multilingual models do not reason and then translate: their answers reflect the volume and quality of training data in each language. The authors note that speakers of languages with lower accuracy live in regions disproportionately affected by climate change.
Context of the initiative
The report was commissioned as part of UNESCO's contribution to the Global Initiative for Climate Change Information Integrity, launched with the UN and Brazil in November 2024. UNESCO notes that climate disinformation is becoming more sophisticated: instead of denying climate change itself, those seeking to undermine climate action increasingly claim that it is not human-caused, that technology will solve everything on its own, or that it is already too late or too costly to act.
The authors' position
“AI has enormous potential to make complex climate science accessible to everyone,” said the report's lead author from UNESCO, Leona Verdadero, adding that the models have a design flaw: they are motivated to agree with the user. Senior author Dr Maria Perez-Ortiz of UCL noted that the models' fluency in handling existing knowledge can create a “false sense of their capabilities”.



