Notwithstanding the “evidence of a lack of evidence for peer review’s efficacy” – as The Lancet editor Richard Horton had noted in an appendix to the 2010 Muir Russell report – publication in a “peer reviewed” journal is still the rallying cry regarding material included in the Intergovernmental Panel on Climate Change (IPCC)’s reports.
Getting a retraction from the IPCC is somewhat akin to getting blood out of a stone. So I wonder what the IPCC’s powers that be will make of the following which appeared in today’s issue of NatureNews:
A surge in withdrawn papers is highlighting weaknesses in the system for handling them
This week, some 27,000 freshly published research articles will pour into the Web of Science, Thomson Reuters’ vast online database of scientific publications. Almost all of these papers will stay there forever, a fixed contribution to the research literature. But 200 or so will eventually be flagged with a note of alteration such as a correction. And a handful — maybe five or six — will one day receive science’s ultimate post-publication punishment: retraction, the official declaration that a paper is so flawed that it must be withdrawn from the literature.
It is reassuring that retractions are so rare, for behind at least half of them lies some shocking tale of scientific misconduct — plagiarism, altered images or faked data — and the other half are admissions of embarrassing mistakes. But retraction notices are increasing rapidly.
“I don’t think there’s any doubt that we’re detecting more fraud, and that systems are more responsive to misconduct. It’s become more acceptable for journals to step in,” says Nicholas Steneck, a research ethicist at the University of Michigan in Ann Arbor. But as retractions become more commonplace, stresses that have always existed in the system are starting to show more vividly.
… frustrations include opaque retraction notices that don’t explain why a paper has been withdrawn, a tendency for authors to keep citing retracted papers long after they’ve been red-flagged (see ‘Withdrawn papers live on’) and the fact that many scientists hear ‘retraction’ and immediately think ‘misconduct’ — a stigma that may keep researchers from coming forward to admit honest errors.
But as more retractions hit the headlines, some researchers are calling for ways to improve their handling. Suggested reforms include better systems for linking papers to their retraction notices or revisions, more responsibility on the part of journal editors and, most of all, greater transparency and clarity about mistakes in research.
The posts on Retraction Watch show how wildly inconsistent retractions practices are from one journal to the next. Notices range from informative and transparent to deeply obscure. A typically unhelpful example of the genre would be: “This article has been withdrawn at the request of the authors in order to eliminate incorrect information.” [New York City-based writer Ivan Oransky, executive editor at Reuters Health] argues that such obscurity leads readers to assume misconduct, as scientists making an honest retraction would, presumably, try to explain what was at fault.
To Drummond Rennie, deputy editor of the Journal of the American Medical Association, there are two obvious reasons for obscure retraction notices: “fear and work.”
The fear factor, says Wager, is because publishers are very frightened of being sued. “They are incredibly twitchy about publishing anything that could be defamatory,” she says.
‘Work’ refers to the phenomenal effort required to sort through authorship disputes, concerns about human or animal subjects, accusations of data fabrication and all the other ways a paper can go wrong. “It takes dozens or hundreds of hours of work to get to the bottom of what’s going on and really understand it,” says Shafer. Because most journal editors are scientists or physicians working on a voluntary basis, he says, that effort comes out of their research and clinical time.
But the effort has to be made, says Steneck. “If you don’t have enough time to do a reasonable job of ensuring the integrity of your journal, do you deserve to be in business as a journal publisher?” he asks.
A better vocabulary for talking about retractions is needed, says Steneck — one acknowledging that retractions are just as often due to mistakes as to misconduct. Also useful would be a database for classifying retractions. “The risk for the research community is that if it doesn’t take these problems more seriously, then the public — journalists, outsiders — will come in and start to poke at them,” he points out.
The only near-term solution comes back to transparency. “If journals told readers why a paper was retracted, it wouldn’t matter if one journal retracted papers for misconduct while another retracted for almost anything,” says Zen Faulkes, a biologist at the University of Texas–Pan American in Edinburg, Texas. [emphases added -hro]
Seems to me that there are a lot of lessons in the above that could be learned by the IPCC, which (we’re constantly told) has a “gold standard” reputation derived from the usage of “peer reviewed” material by its volunteer authors – not to mention (what some would consider its unfounded) claims of “transparency”, “clarity” and “integrity”. And they could certainly use a better vocabulary for discussing (inter alia) uncertainties.
Indeed, one might conclude that it is precisely because the IPCC does not appear to “take its problems more seriously”, that the public (if not journalists) ‘will continue to poke at them’.