These LLMs are very good at refuting Russian propaganda

Heavy open source models, including Nvidia’s Nemotron and Alibaba’s Qwen, showed strong results comparable to Anthropic’s best models. GPT-5.4—the best-performing model from OpenAI—also performed well on the benchmark, giving “Example” answers on 54 percent of the questions and getting 88.9 percent.

Unsurprisingly, recent border samples showed a stronger tendency to reject Russian propaganda than samples from a few years ago. Claude 3.5 Haiku – the highest version released in 2024 – received only 73.1 points on the benchmark. That mark would put it in the bottom third of the models released in 2026 on this metric.

Detailed measurements of Google’s version of Gemini 2.5 Pro show a particular interest in negative and positive information in Russian.

But this change over time was not the same for all LLM producers. Google’s anti-falsity LLM, Gemini 2.5 Pro, is almost a year old now and has only reached a high of 82 on the benchmark, mostly due to negative wording. The latest Google tested version, Gemini 3.5 Flash, only scored 73 on the benchmark, the same as the Anthropic versions released almost two years ago.

In Support post on the Propastop blogThe agency points to a number of examples that show opposition to Russian propaganda when asked in Russian. Google’s Gemini 3.5 Flash received significantly lower benchmarks in Russian than in English, as did heavy models like Moonshot’s Kimi K2 and StepFun’s Step 3.5 Flash.

What one country sees as attractive, of course, another may see as important cultural values ​​that LLMs should support and reflect upon. A recent research from King’s College professor Gregory Asmolov examines the Russian state-through Recent technical cooperation with other BRICS countries-he wants to attract AI types by showing a certain “culturally sensitive” cultural center in Russia.