It’s one thing when a chatbot flunks Tower of Hanoi, as Apple notoriously illustrated earlier this month, and another when poor reasoning contributes to the mess of propaganda that threatens to overwhelm the information ecosphere. But our recent research shows exactly that. GenAI powered chatbots’ lack of reasoning can directly contribute to the nefarious effects of LLM grooming: the mass-production and duplication of false narratives online with the intent of manipulating LLM outputs. As we will see, a form of simple reasoning that might in principle throttle such dirty tricks is AWOL.
Here’s an example of grooming. In February 2025, ASP’s original report on LLM grooming described the apparent attempts of the Pravda network–a centralized collection of websites that spread pro-Russia disinformation–to taint generative models with millions of bogus articles published yearly. For instance, a recent article published on the English-language site of the Pravda network regurgitates antisemitic tropes about “globalists,” falsely claiming that secret societies are somehow ruling the world. Russian disinformation frequently utilizes these false claims and conspiracy theories.
No surprise there. But here’s the thing, current models “know” that Pravda is a disinformation ring, and they “know” what LLM grooming is (see below) but can’t put two and two together.
Screenshot of ChatGPT 4o appearing to demonstrate knowledge of both LLM grooming and the Pravda network
Screenshot of ChatGPT 4o continuing to cite Pravda network content despite it telling us that it wouldn’t, how “intelligent” of it
Even with that knowledge, it nevertheless often repeats propaganda from Pravda. Model o3, OpenAI’s allegedly state of the art “reasoning” model still let Pravda content through 28.6% of the time in response to specific prompts, and 4o cited Pravda content in five out of seven (71.4%) times. In an ideal world, AI would be smart enough to cut off falsehoods at the pass, reasoning from known facts, in order to rule out nonsense.
Both 4o and o3 Are Susceptible to Grooming
Our recent testing revealed that both 4o and o3 are particularly likely to exhibit LLM grooming while performing real-time searches. This is when the model searches the internet for content to use in its responses in real time. Model 4o specifically conducts these searches in response to questions that it doesn’t have a prepared answer for.
When we asked whether the atrocities in Bucha, Ukraine were staged, 4o did well. It strongly denied those lies, and cited widely respected organizations like the UN. It did not cite Pravda content or any other Russian disinformation. It did not report a real-time search, and managed to stay out of trouble.
However, when we asked 4o about the efficacy of ATACMS–a U.S.-made advanced missile system at the heart of intense political debate–in Ukraine, it did conduct a real-time search and got fooled, immediately citing Pravda network propaganda, claiming falsely that ATACMS didn’t work in Ukraine because of Russian air defense prowess. (This is a common false narrative that pro-Russia actors spread, as part of a broader effort to discredit Western military aid to Ukraine based on lies.)
... continue reading