When a scientist submits his or her recent findings in the form of a “paper” to a journal, the manuscript is sent to a panel of reviewers, fellow scientists who have agreed to read the submitted paper and criticize it for flaws in its reasoning, experimental setup, or data processing. This is known as the “peer-review process” and it forms the cornerstone of the scientific enterprise.

An article published in the journal Science today reveals the details of a hoax that was perpetrated over the course of a year to test, we are told, the open-access publishing model. While traditional journals like Science charge often-massive fees for anyone to access their papers, open-access journals charge the publishing authors a fee and allow their paper to be read by anyone. With traditional journals, you pay to read; with open-access, you pay to publish.

The hoax? A paper written by a fictional author at a fictional institute, showing, through fictional data, the anticancer power of a chemical extracted from a lichen. The paper was submitted to 304 open-access journals.

The actual author of this voluntary fraud, John Bohannon, describes the fate which should have befallen this submission: “In fact, it should have been promptly rejected. Any reviewer with more than a high-school knowledge of chemistry and the ability to understand a basic data plot should have spotted the paper’s short-comings immediately. Its experiments are so hopelessly flawed that the results are meaningless.”

The report is shocking: 157 journals accepted to publish the paper; 98 rejected it; problems arose with the remaining 49, as some seemed to be inactive and others stated the paper was still under review. John Bohannon claims that the open-access model is deeply flawed: it invites enterprises to create “journals”, forego peer review, and collect the money from authors hungry to publish. True. The system is not perfect.

However, Curt Rice, writing for The Guardian, makes an equally valid point: the problem need not lie with open-access but with the peer-review process itself. Bohannon’s entire experiment is skewed: he goes after open-access journals and completely bypasses traditional outlets. Indeed, the spoof paper was “concocted by Science“, a traditional journal which, while highly revered, must feel the pressure of living in an age where open-access is gaining in popularity, in much the same way that even the best traditional newspapers feel the heat from alternative, user-generated media content. It’s in Science‘s best interest to discredit the open-access model.

The experience should be repeated, this time including submissions to traditional journals, to test the quality of their peer-review process. It is true that the open-access process encourages greedy companies to publish anything for profit, but the peer-review process as a whole should be carefully examined and fixed if it fails to live up to its reputation.

***

The original news item in the journal Science can be accessed here for free.

The Toronto Star did an excellent job of reporting the facts in their article. And the UK newspaper The Guardian published a very intelligent critique of the reported findings here. Both articles came to my attention through André Picard, who can be followed on Twitter @picardonhealth.