AI Fabricates Over 4,000 Citations to Nonexistent Medical Studies, Audit Finds

Artificial intelligence is creating references to medical research that does not exist, recent findings show.

An audit of millions of biomedical papers identified more than 4,000 citations to nonexistent studies, according to an article in The Lancet. Such fake citations threaten clinical guidelines that health care professionals use to deliver care, said Maxim Topaz, associate professor at the Columbia School of Nursing and lead author of the study.

Fabricated citations pose a danger because they shape clinical guidelines drawn from public research, Topaz told CBS News. "When those fake references make it into the literature, they end up in those guidelines, and that's how doctors decide how to provide care for you," he said. "Your doctor could be making decisions around treatment based on studies that never existed."

None of the errors that Topaz and his team found have been corrected or retracted, and they may still affect patient care, he added. The rate of fake references in published medical literature is rising, with the number of erroneous citations increasing 12-fold over the last three years. The fabricated references appeared in nearly 3,000 academic papers.

Topaz began investigating after an AI app he used to edit one of his scientific papers added a bogus citation. The error passed through multiple rounds of peer review until an editor spotted it. "I was mortified, because I've been studying AI for the past 15 years, so if it can happen to me, it can happen to anyone," he said.

These incidents occur when authors state a fact and ask AI for a supporting citation, Topaz explained. "In some cases, AI would slip those in, inadvertently," he said. "You would hope the facts are accurate, but if they are supported by fabricated citations, you don't know if the 'facts' are accurate."

Sometimes AI tools cite a real author but invent the research and attribute it to that person. Other times, the citations are entirely made up, Topaz said. "This is just the tip of the iceberg," he noted, adding that other fields may face similar problems.

AI-generated fake citations can appear perfectly real, Topaz said. He stressed that researchers must rigorously fact-check their work.