Researcher Creates Fake Disease to Expose AI Medical Risks

When Almira Osmanovic Thunström wanted to show her medical students how artificial intelligence can go dangerously wrong, she invented a fake disease and fed it to the internet. Within months, popular AI chatbots were confidently diagnosing patients with "bixonimania," a condition that never existed.

The researcher at Sweden's University of Gothenburg created an elaborate setup to test how AI medical advice really works. She built a fake university, invented a fake researcher, and planted mentions of bixonimania across blogs and academic preprints.

The made-up condition supposedly caused red, itchy eyelids from too much screen time. It sounded medical enough to fool both humans and machines.

What happened next surprised even Osmanovic Thunström. The fake disease made it through AI training systems without any filter. ChatGPT and other popular chatbots started suggesting bixonimania as a real diagnosis when people described eye symptoms.

The experiment matters because millions of people worldwide now ask AI chatbots for medical advice every single day. Some use it to supplement doctor visits. Others use it instead of seeing a doctor at all.

Osmanovic Thunström knew that most commercial AI systems pull data from Common Crawl, a nonprofit that scrapes internet content since 2007. Humans review some of this data, but they can't catch everything that looks credible enough.

Her fake condition slipped through because it appeared legitimate. Universities rank high as information sources. Researchers with institutional ties carry weight. Even casual blog mentions get picked up as potential truth.

The researcher didn't run a massive campaign or plant her fake disease everywhere. She just sprinkled mentions in a few places. That tiny effort was enough.

The Bright Side

Osmanovic Thunström's experiment isn't about breaking AI systems. It's about protecting people who depend on them. Her work creates a clear trail showing exactly how misinformation enters medical AI and where the gaps exist.

Her students now understand that AI chatbots predict likely answers based on patterns, not medical knowledge. They see why human doctors remain essential for accurate diagnosis and care.

The research also pushes AI developers to build better filters and verification systems. Some companies have already started improving how they weight medical sources after experiments like this one.

Healthcare experts say the solution isn't abandoning AI but using it responsibly. AI can help doctors work faster and spot patterns humans miss, but it needs better training data and stronger safeguards.

Osmanovic Thunström hopes her fake disease helps build smarter, safer medical AI for everyone who needs it.