Too Much Junk Science

By Gordon Schnell

We read about them all the time.  The latest major breakthrough in taming one disease or another.  If not an ultimate cure-all, then at least a novel treatment to ease the condition or its associated symptoms.  In some cases, these reported advances really do result in groundbreaking remedies that improve the lives of many.  But in just as many cases it seems, after the initial hype has quieted, we never hear of them again, leaving us to wonder whatever became of the new miracle drugs.

Thanks to some creative sleuthing by Science magazine, we now may have an answer.  Too much junk science.  Too many medical or scientific discoveries that simply do not hold up.  Not because they ultimately fail upon additional testing and review.  But because they were based on faulty science, or even downright fraud, in the first place.  And how do they make it out of the starting gate?  Through a throng of online journals all too happy to put peer review to the side and publish these dubious developments as the next great thing — for a tidy fee of course.

That is what Science’s John Bohannon found when he set out to test the credibility of the more than eight thousand so-called “open access” scientific journals that have sprung up in  recent years.  Unlike the more traditional print journals which rely on subscription fees, these online-only journals make their money from publication fees charged to the authors.  Thus volume, which is boundless on the Internet, is the key to profitability.  Peer review or other screening devices to ensure some measure of quality control, just get in the way.

Bohannon learned this first-hand.  With the help of two independent groups of molecular biologists from Harvard, he wrote a paper on the cancer-fighting properties of a chemical extracted from lichen.  He then submitted it for publication to 304 open-access journals.  More than half accepted the paper.  The only problem is that Bohannon’s paper was a complete fake.  And not one that should have been difficult to deduce.  To the contrary, as Bohannon explains:

Any reviewer with more than a high school knowledge of chemistry and the ability to understand a basic data plot should have spotted the paper’s shortcomings immediately.  Its experiments are so hopelessly flawed that the results are meaningless.  . . .  I created a scientific version of Mad Libs.

His Harvard collaborators were part of the plan.  They helped Bohannon fine-tune the scientific flaws in his spoof submission so they were both “obvious” and “boringly bad.”  The phony name and university affiliation Bohannon used for the paper’s author were equally transparent, or certainly should have been from a few minutes on the Internet.  Neither the purported author, Ocorrafoo Cobange, nor his supposed university, the Wassee Institute of Medicine in Asmara, actually exists.  Bohannon created them out of thin air, drawing upon a well-placed mixture of Swahili words and African names.

It gets even worse.  The 157 journals that accepted the bogus paper were not necessarily fly-by-night operations or of questionable origin.  Many are owned by some of the biggest names in medical publishing such as Elsevier, Wolters Kluwer, and Sage.  Others are put out by legitimate scholarly societies.  Some of the journals that accepted the paper are not even relevant to the paper’s topic such as the Journal of Experimental & Clinical Assisted Reproduction.  Clearly, none of these journals engaged in any kind of meaningful peer review, the touchstone requirement for legitimate scientific publishing.

What is the take-away from Bohannon’s ruse?  “Beyond the headline result,” he writes, “the data from this sting operation reveal the contours of an emerging Wild West in academic publishing.”  The potential repercussions from this scientific free-for-all cannot be understated.  If such a deliberately fabricated paper can so easily get published despite its premeditated failings, think how many legitimate but deeply flawed papers must also get through.  And then think how many legitimate research efforts might be tainted by relying on these faulty underpinnings.

So, the next time we read about the latest wonder cure or magic potion, we would be wise to temper our enthusiasm.  It might be the real deal.  Or it might just be a simple dose of junk science.  Until the scientific community comes up with a better plan to rein in the open-access mischief that Bohannon so skillfully uncovered, it will continue to be a challenge for most of us to really tell the difference.

Tagged in: Healthcare Fraud,