IPCC and non-peer-reviewed sources: Task group says, ‘let’s disappear the rule’

The Intergovernmental Panel on Climate Change (IPCC) has had a few … hmmm … shall we say, highly publicized credbility problems since November 2009. Readers will recall that one of the recommendations of the InterAcademy Council (IAC)’s “Review of the processes and procedures of the IPCC” was (p. 17):

The IPCC should strengthen and enforce its procedure for the use of unpublished and non-peer-reviewed literature, including providing more specific guidance on how to evaluate such information, adding guidelines on what types of literature are unacceptable, and ensuring that unpublished and non-peer-reviewed literature is appropriately flagged in the report. [emphasis added -hro]

One wouldn’t think it would be too difficult for the IPCC to actually enforce its existing rules on this issue (which it most definitely did not when producting AR4). However – unless my reading comprehension skills have become severely compromised in the last few months – it would appear that the IPCC Task Group (TG) charged with developing some recommendations for implementation in response to the IAC’s recommendation on this particular matter (even though the IPCC had agreed with the IAC’s recommendation) have determined that the IPCC’s mediocre performance in following its own rules have made it necessary to, well, change the rules:

The TG notes that changes to the procedures are warranted to respond to this IAC recommendation.

Fair enough. Changes are warranted, but the TG continues:

The TG, after consulting the WG /TFI TSUs, found that the implementation of this IAC recommendation regarding the appropriate flagging of unpublished and non-peer reviewed literature would not be practical. [emphasis added -hro]

They go on to recommend that Annex 2 (the source of the current “rule”) be amended to “disappear” this “impractical” [and unpracticed more often than not] provision – and to add a few new wrinkles in its place:

Task Group recommendation for decision by the Panel:

Replace the current Annex 2 of the Procedures (‘Procedure for using non-published/non-peer-reviewed sources in IPCC reports’) by a new Annex 2 as described below:

ANNEX 2: PROCEDURE ON THE USE OF LITERATURE IN IPCC REPORTS

This annex is provided to ensure that the IPCC process for the use of literature be open and transparent. Non-journal-based sources can provide crucial information for an IPCC Report, including information about experience and practice with mitigation and adaptation activities (e.g. reports from governments, industry, and other organisations, reports or working papers of research institutions, workshop proceedings). In principle, newspapers and magazines are not valid sources of scientific knowledge. Blogs, social networking sites, and broadcast media are not acceptable sources of information for IPCC Reports.

For the above mentioned sources the following additional procedures are needed.

1. Responsibilities of Coordinating, Lead and Contributing Authors

Authors are requested to critically assess any information they would like to include from a non-journal-based source. Each chapter team should review the quality and validity of each source before incorporating information from the source into an IPCC Report.

Authors who wish to include information from a non-journal based source that is not commercially available are requested to send the full reference and a copy, preferably electronically, to the Working Group/Task Force Bureau Co-Chairs who are coordinating the Report.
[…]

These procedures also apply to those papers undergoing the publication process in peer-reviewed journals at the time of the review.

All sources will be integrated into a reference section of an IPCC Report.
[…]

Look, ma! No blog references allowed … and no more requirement to flag any unpublished and/or non-peer-reviewed literature as such.

Makes one wonder if Phil <we’ll change the definition of peer-reviewed if necessary> Jones was appointed as Chair of this particular task group, eh?!

The mind positively boggles at how they sooooooooooooo don’t get it!

[h/t Arthur Dent whose comment on Climate Audit sent my mouse to the source of this and many other (possibly interesting) docs posted in advance of the May meeting of the IPCC in Abu Dhabi.]

The excerpts posted above are from a 256 page pdf entitled “REVIEW OF THE IPCC PROCESSES AND PROCEDURES Comments from Governments and IPCC Office Holders on the initial draft recommendations prepared by the Task Groups (Submitted by the IPCC Secretariat)”

Perhaps further perusal of this (typically) very poorly compiled .pdf will reveal that Governments and IPCC Office Holders reacted with ‘shock and appall’ to this recommendation of this particular Task Group. But … colour me somewhat skeptical … somehow I doubt it. YMMV, but for some reason I very much doubt that AR5 will demonstrate any significant improvement in processes over AR4. In fact, I’m inclined to suspect that it may well turn out to be … worse than we thought.

P.S. In case anyone’s wondering … I’m still hearing sounds of deafening silence in response to my enquiries to the IAC regarding the missing data in the compilation of responses on which they had purportedly based their recommendations.

Advertisement