March 14, 2010 
I’m trying to think as a neophyte who happens onto the net and comes across a contrarian or denialist website. What claims about climate science do they see on such websites as CA and WUWT and others.
1. Climate scientists do not follow the basic tenets of science, which include openness and sharing of data for the purposes of peer-review and replication and therefore climate science is not ‘science’.
2. Climate scientists have manipulated data to show global warming that does not in fact exist.
What has Steve McIntyre claimed in his inquiry submission?
In my opinion, CRU has manipulated and/or withheld data with an effect on the research record. The manipulation includes (but is not limited to) arbitrary adjustment (“bodging”), cherry picking and deletion of adverse data. The problem is deeply rooted in the sense that some forms of data manipulation and withholding are so embedded that the practitioners and peer reviewers in the specialty seem either to no longer notice or are unoffended by the practices. Specialists have fiercely resisted efforts by outside statisticians questioning these practices – the resistance being evident in the Climategate letters.
So we see posted here quite clearly the accusations of Steve McIntyre against climate science:
1. Data manipulation including bodging, cherry picking and deletion of adverse data — “with an effect on the research record”.
In other words, this manipulation has altered the research record to show something that is not reliable or valid.
2. Data withholding from critics and perversion of peer review.
Let’s look at the claim of data withholding:
I’ve been reading over at CA, searching for info on what Steve McIntyre had and when he had it, and came across this post with the text of his email to Jones dated September 8, 2002:
In Journal of Climate 7 (1994), Prof. Jones references 1088 new stations added to the 1873 stations referred to in Jones 1986. Can you refer me to a listing of these stations and an FTP reference to the underlying data? Thanks, Steve McIntyre
Here is the response from Jones:
Sent: Thursday, September 12, 2002 12:36 PM
Subject: Fwd: Jones 1994 Data Set
Dear Steve,
You are looking into station lists from papers in the early 1990s and 1980s. These are now out of date. There will be a new paper coming out in J. Climate (probably early next year). I’m attaching the station list (5159 stations) from that paper. In this file the first number is the WMO number ( or an approximation to it or just a number – large number of US stations at the end). Official WMO numbers are those here divided by 10. The first station (Jan Mayen has a WMO number of 01001, but in our list it is 10010).
thenLatitude (degrees*10 so 589 is 58.9 N, -ve will be S)
Longutude ( similar to latitude with E -ve)
Height (m , with missing of -999)
Name
Country (this field isn’t always there and doesn’t always take into account changes of the last 20 years. We don’t use this field, so don’t bother keeping up to date with it.Also names are common English names for countries not their official ones that the UN uses).
First year of data
Last year of data (Most of the 2001s also include 2002 but this file hasn’t been altered)
Then some other numbers.
The first file (above description) is what we call station headers. They mean we have temperature data for the years between the first and last year for each station. However there may be lots of missing data or the data may be deemed inhomogeneous (see the papers you have), so a station may not be used in the our analysis for a whole raft of reasons. As we work with station anomalies we also have a file (also 5159 lines) of stations normals (average temps in deg C*10 for 1961-90). If this second file contains -999 (missing values) then the station temperatures will not get used so the station isn’t used.
Once the paper comes out in the Journal of Climate, I will be putting the station temperature and all the gridded databases onto our web site. The gridded files on our web site at the moment are from our current analysis. The new analysis doesn’t change the overall character of the gridded fields, it is just easier for me to send the new lists of stations used from the new analysis.
I hope this helps.
Phil Jones
Here was McIntyre’s reply:
Thanks for this. It seems awkward not to use exact WMO station numbers – do you by any chance have a concordance of your numbers to WMO numbers where they do not correspond? I (think I) noticed that sometimes your numbers are also in use for a nearby but different GHCN station, which seems a bit awkward. I also noticed a few stations in which the lat-long’s do not seem to tie into GHCN data and can forward these possible errata if you like. Wouldn’t it make more sense to convert over to WMO station numbers carrying a concordance to your past numbers?
I’m still interested in the 1994 data set as it has become so standard. Is there a FTP from which the underlying station data and mean temperatures (either as anomalies or absolutes) can be downloaded? I’ve located an FTP for your 1991 version, but have had little success in locating the 1994 version.
When you do publish the 2002 version, I would urge you to make FTP available annualized data for individual station boxes as well as for grid-boxes, so that readers interested in regional studies can carry out verifications. (If this is not currently available for the older data, it would also be nice for it as well.)
It would also be nice if annualized data were also available as I am sure that many of your users are mostly interested in this. The 12-fold reduction in dataset size is fairly important for fitting into Excel spreadsheets, which work nicely on annual data.
Regards, Stephen McIntyre
Jones responds:
Dear Steve,
Attached are the two similar files [normup6190, cruwld.dat] to those I sent before which should be for the 1994 version. This is still the current version until the paper appears for the new one. As before the stations with normal values do not get used.I’ll bear your comments in mind when possibly releasing the station data for the new version (comments wrt annual temperatures as well as the monthly). One problem with this is then deciding how many months are needed to constitute an annual average. With monthly data I can use even one value for a station in a year (for the month concerned), but for annual data I would have to decide on something like 8-11 months being needed for an annual average. With fewer than 12 I then have to decide what to insert for missing data. Problem also applies to the grid box dataset but is slightly less of an issue.
I say possibly releasing above, as I don’t want to run into the issues that GHCN have come across with some European countries objecting to data being freely available. I would like to see more countries make their data freely available (and although these monthly averages should be according to GCOS rules for GAA-operational Met. Service.
Cheers
Phil Jones
So, we see here that Jones did in fact share data with McIntyre all the way back to 2003. Very soon around this time, McI was also requesting data from Michael Mann re: his 1998 and 1999 papers.
Here is an excerpt: You can read the file here:
I have been studying MBH98 and 99. I located datasets for the 13 series used in 99 at ftp://eclogite.geo.umass.edu/pub/mann/ONLINE-PREPRINTS/Millennium/DATA/PROXIES/(the convenience of the ftp: location being excellent) and was intereseted in locating similar information on the 112 proxies referred to in MBH98, as well as listing (the listing at http://www.ngdc.noaa.gov/paleo/ei/data_supp.html is for 390 datasets, and I gather/presume that many of these listed datasets have been condensed into PCs, as mentioned in the paper itself.
Rutherford responded to McI on behalf of Mann and provided him with an Excel file, which was subsequently found to have errors in it.
So all this talk about climate scientists hiding data and denying requests for data seems somewhat suspect. McI asked for a list of stations, and he got it. He got station and temp info. Seems Jones was quite accommodating. McIntyre asked for data used in MBH. He got it — although it wasn’t accurate as was later pointed out. Later Jones indicated that he probably would not share the data due to some issues but in fact, Jones did share data.
ETA: as Eli Rabett points out, the errors were due to improper alignment of the data in columns rather than the wrong data.
One notes that in his submission to the Parliamentary Inquiry, McI doesn’t present evidence that he was supplied with data as requested by Jones and Mann (through Rutherford) very early on. To be completely honest and to provide complete disclosure so necessary to a competent audit, he should have included references to the times he was able to get access to data on the part of Phil Jones and Mann, but he omits this.
I need to know the following:
1. Did Jones give McI the “raw station data” or did he give McI the ‘QC and homogenized’ data used to create the ‘gridded product’ Jones speaks of in the Inquiry?
2. Was this the data that McI used to critique MBH98/99?
Admittedly this is all very complex and it is unclear working from old CA posts and files from the wayback machine what is what, so I would appreciate any help clarifying these questions.
The Policy Lass 
1. Did Jones give McI the “raw station data” or did he give McI the ‘QC and homogenized’ data used to create the ‘gridded product’ Jones speaks of in the Inquiry?
Try this post: http://climateaudit.org/2009/08/06/a-2002-request-to-cru/
There were several types of files and I’m not sure which Jones supplied back in 2002. This request was about Phil Jones 1994 paper on instrumental temperature.
2. Was this the data that McI used to critique MBH98/99?
Hard to say since the data doesn’t appear to have been helpful. MBH did use a reconstruction from Jones as their instrumental record and it seems McIntyre was trying to reconcile it with Jones different publications.
McIntyre probably wouldn’t mention that he did get some data from both Mann and Jones since it has nothing to do with the point that other data were not provided despite many requests. Rutherford provided the wrong data which McIntyre was criticized for using in an early paper. From looking at the climategate E-mails it was Mann who told the group to not respond or give anything to McIntyre so I wonder if the abrupt change from early cooperation was due to this?
But McI is supposed to be all about completeness of disclosure. McIntyre has spread this notion that is now parroted by all the critics that scientist who do climate research have a culture of non-disclosure — he says so in his submission. I am only saying that perhaps McI created that culture of non-disclosure when they did give him data and he, an outsider and friend of deniers, tried to smear them. Of course, he wouldn’t admit to that.
The raw data was over half a million sheets of handwritten weather reports from around the world, which, if stacked up in a pile, would reach 60 metres in height.
That’s what I thought.
There would be the raw raw data directly from the individual stations that gets sent to the National Meterological bureaus. There would be the processed data from the National Meteros that gets sent to CRU. CRU does a quality control and homogenization of the data. It then uses that data to create the gridded poduct that is released on its website. Is that correct?
It seems to me that the ‘septics’ are claiming that they are unable to do an audit on the CRU data because the scientists are withholding it. Yet, Jones did appear to give out the stations and the temp data to McI in 2002 and all the data is available through the NMS if they really want to reproduce the CRUTemp. They can all access the processed gridded product at CRU. It’s the steps in between the raw data and processed gridded product they want to check. This is where they see ‘bodging’ and invalid adjustments and cherry picking going on.
Jones didn’t give the QC/H data to Warwick Hughes a couple of years later when he asked. From what Jones says, he had been told not to give out the data because of some data sharing concerns — and of course, because of all the work etc. — the famous “why should I give it to you?” quote that critics like to cite. Warwick Hughes said he didn’t want to gridded data. He wanted the QC/H data which it appears McIntyre already had.
Sorry to interrupt, but Watts came out of his bunker to have a ding-dong with Ed Darrell, and it gets onto Tamino’s debunking. Get popcorn 😉
http://timpanogos.wordpress.com/2010/03/14/warming-and-science-denialists-stuck-with-political-egg-on-their-predictions/#comment-96746
Choice quotes:
It’s unreal that these people think that ignoring things like time of observation, moves, addition of stephensen screens, etc leads to a more reliable history of temperature.
The data that Rutherford provided was not wrong, it was in a few cases misaligned in the Excel spreadsheet, in a way that a couple of hours work easily untangled. Steve is a very needy guy
Yes, you’re right — there was a collation error rather than any error in the data itself. I should have worded it more clearly in the post. Thanks!
“2) In some recent comments, you are reported as stating that we requested an Excel file and that you instead directed us to an FTP site for the MBH98 data. You are also reported as saying that despite having pointed us to the FTP site, you and your colleague took trouble to prepare an Excel spreadsheet, but inadvertently introduced some collation errors at that time. In fact, as you no doubt recall, we did not request an Excel spreadsheet, but specifically asked for an FTP location, which you were unable or unwilling to provide. Nor was an Excel spreadsheet ever supplied to us; instead we were given a text file, pcproxy.txt. Nor was this file created in April 2003. After we learned on October 29, 2003 that the pertinent data was reported to be located on your FTP site ftp://holocene.evsc.virginia.edu/pub (and that we were being faulted for not getting it from there), we examined this site and found it contains the exact same file (pcproxy.txt) as the one we received, bearing a date of creation of August 8, 2002. On October 29, 2003, your FTP site also contained the file pcproxy.mat, a Matlab file, the header to which read: “MATLAB 5.0 MAT-file, Platform: SOL2, Created on: Thu Aug 8 10:18:19 2002.” Both files contain identical data to the file pcproxy.txt emailed to one of us (McIntyre) in April 2003, including all collation errors, fills and other problems identified in MM. It is therefore clear that the file pcproxy.txt as sent to us was not prepared in April 2003 in response to our requests, nor was it prepared as an Excel spreadsheet, but in fact it was prepared many months earlier with Matlab. It is also clear that, had we gone to your FTP site earlier, we would simply have found the same data collation as we received from Scott Rutherford. Would you please forthwith issue a statement withdrawing and correcting your earlier comments.”
http://www.climateaudit.info/correspondence/mann.031111.htm
OMFG. Sock puppet. Too blatant. Yawn.
“It’s unreal that these people think that ignoring things like time of observation, moves, addition of stephensen screens, etc leads to a more reliable history of temperature.”
So let me get this straight; raw data that CRU did not keep was more reliable than the raw data? I also see no one here has realized that adjustments to said raw data has demonstrated consistent elevated values due to CRU’s “adjustments.”
You certainly seem passionately positive that skeptics are just trivial gadfly s; care to rationalize the statistical upward bias?
Ummm, didn’t Jeff (Air Vent) Id and Roman M verify HadCRUT, but also got a more positive trend?
Care to rationalise the statistical upward bias?
http://noconsensus.wordpress.com/2010/03/24/thermal-hammer/
Or are JI and RM now part of the grand climate conspiracy now?
J Bowers
I see the hammer used GHCN data. To bad the upward bias within that data has already been documented rather thoroughly.
http://wattsupwiththat.com/2010/01/07/cru-3b-%E2%80%93-urban-warm-bias-in-ghcn/
You saying that Urban Heat Island Effect has been “removed?” Inflated trend in, inflated trend out.
While you’re at it, consider the upward trend in temps since the Little Ice Age. The globe is warming, and has been doing so for quite some time.
Ugh, more searching for troof fairies.
Dave, there is no additive effect from UHI, only negative. Science, it can be counter-intuitive, oh my my.
Watts has been debunked into the Stone Age but refuses to acknowledge it, and the dittohead sheeple there just go, “Baa-aahhh!”.
What caused the so-called Little Ice Age? Try; decreased solar activity (look up Maunder Minimum), increased volcanic activity, ocean conveyor slowdown, and then some natural variability.
So, ummmm, how come temperature has been rising right along with CO2 levels, WHILE SOLAR ACTIVITY HAS BEEN UNUSUALLY LOW? Was it magic?
HERE’S A GRAPH of temps with CO2, 10 year linear trends overlapped every 5 years.