A mea culpa. I’ve heard repeated the statement that the IPCC is supposed to only use “peer-reviewed” literature in its reports so many times, I assumed it was true.
ETA: I’ll use this post to record the other mea culpas out there from media and bloggers who repeated the misinformation.
Here, thanks to Climateprogress, are the facts in relation to the claims on the denialosphere.
Here’s the quote on Science News:
The IPCC report was supposed to reflect only peer-reviewed science. Not the speculation of scientists, which the initial source of that 2035 figure (Indian glaciologist Syed Hasnain) recently acknowledged it was. Nor should magazine articles or gray literature reports – like the World Wildlife Fund document that repeated the speculative 2035 figure – become the foundation for IPCC conclusions.Which is why IPCC specifically prohibits reliance on such documents.
Climateprogress posts about Lal’s “admission” that the 2035 claim was used despite knowing it was false in order to sway opinion:
Interestingly, I thought that was true, too, but I decided to check with two top IPCC scientists, and they both confirmed to me that in fact, the IPCC does allow gray literature reports. And the IPCC explains this here (see Annex 2).
Lal told me:
“We were allowed to cite gray literature provided that it looked to us to be good science.” [my emphasis]
So the IPCC is able to use gray literature as long as it looked to be “good science”.
Here is a quote from the IPCC Annex on process:
PROCEDURE FOR USING NON-PUBLISHED/NON-PEER-REVIEWED SOURCES IN IPCC REPORTS
Because it is increasingly apparent that materials relevant to IPCC Reports, in particular, information about the experience and practice of the private sector in mitigation and adaptation activities, are found in sources that have not been published or peer-reviewed (e.g., industry journals, internal organisational publications, non-peer reviewed reports or working papers of research institutions, proceedings of workshops etc) the following additional procedures are provided. These have been designed to make all references used in IPCC Reports easily accessible and to ensure that the IPCC process remains open and transparent.
1. Responsibilities of Coordinating, Lead and Contributing Authors
Authors who wish to include information from a non-published/non-peer-reviewed source are requested to:
a. Critically assess any source that they wish to include. This option may be used for instance to obtain case study materials from private sector sources for assessment of adaptation and mitigation options. Each chapter team should review the quality and validity of each source before incorporating results from the source into an IPCC Report.
b. Send the following materials to the Working Group/Task Force Bureau Co-Chairs who are coordinating the Report:
- One copy of each unpublished source to be used in the IPCC Report
- The following information for each source:
- Name of journal or other publication in which it appears, if applicable
- Information on the availability of underlying data to the public
- English-language executive summary or abstract, if the source is written in a non English
- Names and contact information for 1-2 people who can be contacted for more information about the source.
Responsibilities of the Review Editors
The Review Editors will ensure that these sources are selected and used in a consistent manner across the Report.
3. Responsibilities of the Working Group/Task Force Bureau Co-Chairs
The Working Group/Task Force Bureau Co-Chairs coordinating the Report will (a) collect and index the sources received from authors, as well as the accompanying information received about each source and (b) send copies of unpublished sources to reviewers who request them during the review process.
4. Responsibilities of the IPCC Secretariat
The IPCC Secretariat will (a) store the complete sets of indexed, non-published sources for each IPCC Report not prepared by a working group/the Task Force on National Greenhouse Gas Inventories (b) send copies of non-published sources to reviewers who request them.
5. Treatment in IPCC Reports
Non-peer-reviewed sources will be listed in the reference sections of IPCC Reports. These will be integrated with references for the peer-reviewed sources. These will be integrated with references to the peer reviewed sources stating how the material can be accessed, but will be followed by a statement that they are not published.
Fair enough — there are no doubt documents that would be of use to IPCC reviewers and editors that are not in the peer-reviewed literature that might be included in the reports, so long as they are available for reviewers and as long as their provenance is clear.
So, in other words, this tempest about the IPCC using WWF materials and shouts that they are wrong to do so is on the face of it wrong. Of course, the particular WWF document including the error (2035 vs 2350) should not have been used because it was just plain wrong, but that’s another issue.
Critics were claiming that the IPCC disobeyed its own rules — and I chimed in, assuming wrongly that it was IPCC process to only use peer-reviewed literature.
So mea culpa. I take it back. The IPCC lead authors and reviewers were in their rights to use “gray” literature so long as they made sure to follow the procedures outlined in the IPCC annex for such use. Of course, they should always double check to make sure the non-peer reviewed sources are in fact good science.