During the course of his 2010 testimony to the U.K. parliament’s Science and Technology Committee, UEA’s renowned Director of the Climatic Research Unit (CRU), Phil Jones, responded to a question regarding the peer-review process by indicating that he had never been asked to provide data and code for any of his contributions to the peer-reviewed literature when such papers were under review.
One might reasonably infer from this that when Jones is a reviewer of a paper, he doesn’t ask to see data or code either. I recently came across a Climategate E-mail [2486.txt] from Jones which would confirm that this is a reasonable inference.
Jones was responding to a Jan. 28/2004 observation from Peter Gleick (who has a penchant for reviewing books he’s given no indication of having read, and for failing to substantiate his accusations about others).
In this instance, the thread title was, “MBH Submission (fwd)”, and Gleick’s observation was:
I find Reviewer A’s email a pretty convincing indication of what CC and Mann will face if the code isn’t released.
The timing suggests that “Reviewer A” may well have been Steve McIntyre who, recounted the following, shortly after Schneider died in 2010:
[Schneider] asked me to review a 2004 submission to Climatic Change by Mann et al responding to MM2003 – consistent with his public representations. It seemed to me that there was an inherent conflict of interest in such a review but this was obviously known to Schneider and I attempted to separate out my interests as a disputant from my obligations as a reviewer as much as possible.
[…] my approach was informed by ideas of due diligence that were not then characteristic of academic peer reviewing. In my capacity as a reviewer, I asked to see supporting data for Mann’s supposed rebuttal to MM2003 – the topic of his submission – and to see source code to document his allegations that we’d supposedly made grievous mistakes in implementing his methodology – again an important aspect of his submission. […]
Schneider replied that he had been editor of Climatic Change for 28 years and, during that time, nobody had ever requested supporting data, let alone source code, and he therefore required a policy from his editorial board approving his requesting such information from an author.
But I digress …
Jones’s Jan. 29 reply was addressed to many recipients, including Gleick and Stephen Schneider – founder and editor of the journal Climatic Change [CC] [text reformatted and emphasis added -hro]:
Steve, Peter et al,
I totally agree with Peter on Yuck. The tone of the email from Reviewer A indicates the sorts of issues we would be in. Here are my thoughts:
If you accede to this request the whole peer-review process goes down the tubes.
Reviewers will be able to request the earth from authors. If we all started doing this the number of reviews we could do would dramatically reduce. I currently do about 20-30 reviews a year. If I began asking for this sort of information from journals (AMS, AGU, RMS etc) I would be laughed out of court. I guess it would stop the papers to review coming.
The whole system would grind to a halt. I’ve never requested data/codes to do a review and I don’t think others should either. I do many of my reviews on travel. I have a feel for whether something is wrong – call it intuition. If analyses don’t seem right, look right or feel right, I say so. Some of my reviews for CC could be called into question!
Well, now I understand the intricacies of this much-vaunted “peer-review” process. “Intuition” and Because! I! Said! So! will trump due diligence, every time – otherwise the whole process would go “down the tubes”.
Amazing. Simply amazing.