AI fabricated – A new audit found thousands of biomedical papers include citations to studies that don’t exist, raising alarms for clinical guidelines.
Artificial intelligence is now raising alarms not only for what it generates, but for what it cites.. Researchers say an audit of biomedical literature uncovered thousands of citations pointing to medical research that never existed. a problem that could ripple into the clinical guidance doctors depend on.
The findings come from a recent review described in a report published in The Lancet.. It examined millions of biomedical papers and identified more than 4. 000 that contained citations to non-existent research. according to the study’s lead author. Maxim Topaz.. Topaz is an associate professor at the Columbia School of Nursing. and he warned that fabricated citations can distort the evidence base used to shape recommendations for patient care.
Those citations matter because clinical guidelines are built from publicly available studies that clinicians rely on when making decisions.. If bogus references end up in the literature and are then used in guideline formation. doctors may be acting on evidence that was never produced.. In other words, the citation trail can become a path to inappropriate confidence.
The report also highlights a second concern: the fabricated items Topaz and his team identified have not been corrected or retracted.. That means the questionable references can remain in circulation. continuing to influence how later researchers interpret prior work and. indirectly. how recommendations are formed.
The problem appears to be escalating.. Topaz said the rate of fake references appearing in published medical literature has grown. noting that the number of erroneous citations increased 12-fold over the last three years.. The audited cases spanned nearly 3. 000 academic papers. indicating that the issue is not confined to a narrow subset of journals or topics.
Topaz’s interest in the problem wasn’t purely theoretical.. He described an experience where an AI app he used to help polish one of his own scientific papers inserted a citation to something that didn’t exist.. The error then made its way through several layers of peer review before a sharp-eyed editor caught the reference.
For Topaz, the episode was especially alarming because it came despite his own long-standing work in AI. He said he was “mortified” by the incident, arguing that if the mistake could occur in his workflow, it could happen to researchers who are less familiar with the ways AI can fail.
According to Topaz. the failures can begin when an author asks an AI tool for a citation to support a statement of fact.. In some cases, he said the tool inadvertently inserts fake citations as if they were verifiable references.. Even if the surrounding wording looks polished. the supporting source may not exist. leaving readers unable to confirm whether the underlying claim is truly backed by evidence.
In other scenarios. the tool may produce citations that appear grounded in real people while still being wrong about the underlying research.. Topaz said some fabricated cases involve attributing invented work to real authors.. In still other instances, the cited studies were entirely made up, with no real research behind the citation.
Topaz cautioned that what has been found so far may represent only a fraction of the risk. He suggested the issue could be wider than biomedical research alone, implying that other fields could be affected by similar citation integrity failures.
He also emphasized how convincing these fabricated citations can look once they enter documents and reference lists. Because they can appear “perfectly real,” researchers may need to go beyond trusting the surface form of a citation and intensify fact-checking habits.
For the scientific and medical communities, the central challenge is tightening the chain of verification.. When incorrect references are difficult to spot and persist without retraction. the cost is not just scholarly embarrassment. but the potential downstream effect on clinical guidance and treatment choices.
AI fabricated citations biomedical research audit clinical guidelines scientific integrity peer review errors citation verification
So basically doctors are trusting fake sources now? Cool.
I saw something about “AI citations” and I’m not gonna lie this makes me worry about the whole medical paper thing. Like if they already don’t exist then why wasn’t it caught way earlier? Also who even checks citations anymore half the time.
Wait, I thought AI was supposed to reduce mistakes, not invent studies lol. If the citations are fake and still not retracted, then it feels like the system is just letting it slide. But maybe it’s just a few journals? I’m reading “4000” and that sounds like a lot but also idk how many total papers.
This is why I don’t trust “clinical guidelines” anyway, feels like everything is built on somebody’s paper trail. The part about it being a 12-fold increase over 3 years is wild though—like that’s basically admitting it’s getting worse. And the fact they “haven’t corrected” anything makes it seem pointless to audit at all, because it’ll just keep spreading. Also I’m confused, doesn’t AI only write the sentences, not choose citations? Unless everyone is using the same software to pull references.