Scientists Fool AI With Fake Disease, Journals Fix the Mess

When scientists created a completely made-up disease to see if AI would fall for it, they discovered something important about how we verify information today.

In 2024, medical researcher Almira Osmanovic Thunström and her team at the University of Gothenburg invented "bixonimania," a fake skin condition supposedly caused by staring at screens too long. They uploaded two phony research papers to a preprint server, complete with quirky references to Star Trek, The Simpsons, and The Lord of the Rings as obvious red flags.

Within weeks, major AI chatbots including Google's Gemini, ChatGPT, and Microsoft's Bing Copilot started describing bixonimania as if it were a real medical condition. The fake papers even got cited in actual peer-reviewed academic literature.

The experiment wasn't meant to shame AI or embarrass anyone. It was designed to reveal a growing challenge in scientific publishing and show where the system needs strengthening.

And here's where the story gets genuinely hopeful.

The Bright Side

As soon as the experiment went public, the scientific community sprang into action. When Nature magazine contacted journals about papers citing the fake disease, those journals immediately posted retraction notices and began cleaning up their databases.

OpenAI responded by stating their technology has gotten "better at providing safe, accurate medical information." Google and Microsoft are now using cases like this to improve how their AI systems verify medical claims before sharing them.

The experiment highlighted a real vulnerability, but it also proved that the peer-review system can still self-correct when problems come to light. Researchers are now developing better tools to catch AI-generated content and verify sources before publication.

Osmanovic Thunström's work has sparked important conversations about maintaining scientific rigor in an AI-powered world. Universities and journals are implementing new verification processes specifically because of discoveries like hers.

The fact that a fake disease with Star Trek references got through shows we have work to do, but the swift response shows the scientific community is ready to do that work and keep our knowledge systems trustworthy.