January 16, 2010 
Some interesting material over at Climate Audit around the NASA FOI request for emails surrounding McI’s discovery of a mistake in the GISS record post 2000.
As I understand it, Steve McI found an error in the data, which according to Hansen and his colleagues was a result of an error in processing the data. When corrected, the change affected only the US data, not the global data. It had no effect on the ranking of years prior to 2000 and post 2000, it meant a correction of 0.15C.
However, much ado was made of this error.
Here is Steve McIntyre:
Hansen switched from TOBS adjusted versions to raw versions on Jan 1, 2000 without any disclosure of this change to readers and without any scientific justification. It’s nothing to do with throwing out data…That Hansen has made this type of gross programming error in something as extraordinarily simple as a program to merely calculate average temperatures should give pause to anyone assuming that Hansen’s climate model is error-free. The need for a proper arms-length audit of the GISS climate model (or some other equally important model) is obviously a major challenge and, in my opinion, something of very high priority for anyone wanting to rely on these models for policy. [my emphasis]
IT will be interesting to see if GISS apologist, Eli Rabett, comments on this matter.
You can see he suggests this was done on purpose, consciously, and was not an error.
Not only that, but the emails suggest it wasn’t even Hansen who made the error but someone named Reto and that when Reto was going to do a mea culpa, accepting the blame and clearing Hansen, Hansen said no as that would make it look as if Hansen was passing the buck.
Subsequently, Steve McIntyre was lauded as some kind of champion of truth and beauty, and blogospewers claimed that as a result, 1934 and 1998 shifted positions — clearly, according to them, showing that AGW was a hoax or fraud.
In fact, Hansen actually referred to 1934 as the warmest year on record according to GISS in his 2001 paper. So, no. The correction showed no such thing.
What I see happening in this episode is that McI finds a small error, which makes no difference to the ranking of hottest years on US record anyway, and the denialosphere blows it all out of proportion as if this is proof of skullduggery o the part of NASA rather than an innocent error. Then, it gets trumpeted around the blogosphere by people deniers who can’t tell the difference or don’t care to mention the difference between a slight change in the US data post 2000 and a negligible change in global data such that the rankings remain the same, etc.
Judicial Watch itself is propagating this error:
According to multiple press reports, when NASA corrected the error, the new data apparently caused a reshuffling of NASA’s rankings for the hottest years on record in the United States, with 1934 replacing 1998 at the top of the list. [my emphasis]
This is what Hansen et al actually wrote in 2001:
The U.S. annual (January-December) mean temperature is slightly warmer in 1934 than in 1998 in the GISS analysis (Plate 6). This contrasts with the USHCN data, which has 1998 as the warmest year in the century. In both cases the difference between 1934 and 1998 mean temperatures is a few hundredths of a degree. The main reason that 1998 is relatively cooler in the GISS analysis is its larger adjustment for urban warming. In comparing temperatures of years separated by 60 or 70 years the uncertainties in various adjustments (urban warming, station history adjustments, etc.) lead to an uncertainty of at least 0.1°C. Thus it is not possible to declare a record U.S. temperature with confidence until a result is obtained that exceeds the temperature of 1934 by more than 0.1°C.
The temperature anomaly in the United States in the GISS analysis for 2000 through November is about 0.8°C. This is unusually warm, but it is very unlikely that the U.S. temperature in 2000 will exceed the levels of 1934 or 1998.
The global temperature anomaly for 2000 through November is 0.37°C, which is well below the maximum of 1998. Nevertheless, if we consider the fact that 2000 should have felt the maximum influence of the present long-lived La Niña (see below), it is clear that 2000 was an unusually warm year.
The correction to the GISS data would have no effect on temp before 2000 so I have no idea where this idea came about that now, 1934 was shown to be warmer than 1998.
Perhaps someone with a better understanding of this can correct me if I’m wrong.
Regardless, it really is a case study of a moral panic on the part of global warming skeptics and serious bad faith on the part of deniers.
I’m really so glad that McI posted this — it really does clarify many many things for me.
So thanks Steve McIntyre!
The Policy Lass 
My goodness, denialists are still obsessing over this? Desperate times…
In most fields of science, when inconsequential errors are discovered and corrections made, they are done with little fuss. In climate science, any error is proof among childish denialists of massive incompetence and/or fraud.
The disgraceful false assumptions made in McIntyre’s post is also quite revealing.
On a minor related note, as of Nov. 13th,
http://data.giss.nasa.gov/gistemp/updates/
2006 and 1998 are a few hundredths ahead of 1934 in the U.S. data. As before, no significant effect on the global record.
http://data.giss.nasa.gov/gistemp/graphs/Fig.D.txt
Susann,
You are drawing completely wrong conclusions. You are correct that the Y2K error should have no effect on temps prior to 2000. The fact they earlier temps changed is a big problem for me.
Back in 2007 when this was happening, GISS website kept a very prominent link to both US and
global data. Now the only prominent link is to global temperature data. See
http://data.giss.nasa.gov/ However, the US data is still available on the website at
http://data.giss.nasa.gov/gistemp/graphs/Fig.D.txt
You will notice GISS current data has 1998 as the warmest year in the US again. The anomaly
for 1998 and 2006 are tied at 1.29 and 1934 is 1.26.
So 1934 has moved into third place. The
years which should have been corrected (post-2000) are now back up to the range they were in
before the error was found. 2006 and 2001 are actually warmer now than before the error was
found.
Year Pre-error 2007 Post-error 2007 Current
1998 1.24 1.23 1.29
2006 1.23 1.13 1.29
1934 1.23 1.25 1.26
1921 1.12 1.15 1.15
1999 0.94 0.93 1.05
1931 1.08 1.08 0.99
1953 0.91 0.90 0.86
2001 0.90 0.76 0.91
1990 0.88 0.87 0.89
1938 0.85 0.86 0.79
The fact Hansen is cooling the past is reason to doubt Hansen, not reason to claim McIntyre has made an error.
In the FOI’d emails, you can find this one:
Jim Hansen writes [2007-08-10 at 11:59 -500]: “The appropriate response is to show the curves for U.S. and global temperatures before and after (before and after McIntyre’s correction). Makiko doubts that this is possible because the earlier result has been “thrown away”. We will never live this down if we give such a statement. … By the way, I think that we should save the results of the analyses at least once per year …”
Check it out for yourself. Hansen seems not to know they were tossing older analyses. It is a good thing others (including Hansen) have shown that Hansen has had 1934 has the warmest on record, warmer than 1998. However, by 2007 Hansen had 1998 as warmest. The correction made 1934 warmest once again, but then other unexplained adjustments made 1998 warmest once again (tied with 2006). GISTEMP is becoming a laughingstock.
Hansen is right. They will never live it down.
Hansen seems to be the evildoer incarnate to the skeptics and contrarians and deniers despite the fact the error in GISS wasn’t his doing according to the emails. I really don’t understand this, but I expect there has to be a focus for people’s anger and frustration. He put his face out there and has been outspoken and so I guess he is the public face of AGW theory.
It’s always interested me, this science advocacy role he’s taken on. He must feel very certain of his position to feel such a need to step outside the normal role of a scientist. People argue he is a catastrophist, an alarmist, and yes, I have read some of his papers that do predict dire consequences for unabated GHG emissions. I’ve never had any reason to doubt his sincerity or suspect that his work has been adjusted in a fraudulent manner. The error that McI pointed out seems rather inconsequential. Any changes that have been made to the GISS record after 2001 I would expect have to be explained. I will look into that and see what I can find.
Hansen makes a point of discussing in his paper how the difference between 1934 and 1998 are so minimal as to make it impossible to determine which was hottest because the margin of error is so big. 0.1C from what I read. So for any year to beat 1934, the anomaly has to be more than 0.1C over 1934 otherwise it is really not possible to argue that either was ahead of the other.
In the end, I don’t think it really matters much — it’s very possible that there were factors in play in the 1930s that led to the high temperatures experienced in the US — natural variations, the fact that aerosols had not yet had much influence to mask AGHG warming, etc.
The thing is, if you do not trust Hansen or the GISS, if that’s your starting point, then you won’t accept anything that comes out of his shop. I’m not at that point. I don’t see any evidence that he shouldn’t be trusted. It’s possible that despite his advocacy he is entirely sincere and his work is valid.
Show me evidence that his work has been fraudulently altered or else it’s all just speculation and is no better than gossip.
Perhaps you can explain why this apparent cooling of the past is in error? Is there a paper somewhere that explains this such that we can discuss the methodology and decide if it is legitimate or erroneous or fraudulent?
Susann,
Did you look at the numbers I presented above? Perhaps the format made it difficult to understand. There are three columns. The first is what GISS showed prior to the discovery of the error in 2007. The second column is the numbers after the error was corrected. The third column is the numbers as they appeared about two weeks ago. They have may changed since then, but I hope not.
In Hansen’s 2001 paper he admitted 1934 was the warmest year in the US. By 2007, 1934 had cooled and 1998 was the warmest. After the error was corrected, 1934 was the warmest again. If you go on his website, 1998 is the warmest again. Why should this be? As far as I can see, it should not be. It should not have affected temps prior to 2000. These unwarranted adjustments to the temp history is one of the big reasons people criticize Hansen.
You should be criticizing Hansen, not McIntyre.
Ron, I did look at them.
Indeed, some data has changed since the 2001 paper. From NASA’s website, it appears that there were a number of coding errors or some such reason that led to changes. Among others, 1934 went up, 1921 went up, 1931 went down, 1953 went down.
Do you have any evidence that these changes are fraudulent? Or are you just assuming they are?
Ron, are you sure those are GISS data? I ask, because as far as I know GISS still has anomalies below +1.0 for the 1951-1980 base period but several of the years in your table show anomalies over +1.0.
BTW, Hansen explained why temps in earlier years change slightly in the emails: more station data becomes available as it is entered and this affects — very slightly — the anomaly data for earlier years.
The anomolies are US data, not global.
I do not understand your explanation of why the earlier years should be affected. If the base period was changing, that could change the anomaly. But why should the base period change? It still does not make sense to me.
More data allow for greater accuracy as spatial coverage is increased. That is all, the base period is not changing it is just that more data is used to calculate the anomaly. Notice that the changes are only a few hundredths of a degree and that the differences are well within the margin of error.
Susann,
I’m sorry. I see I was answering a different question than the one you asked. You are asking why I believe cooling the past is an error.
My answer is that the changes in the temp record are not supported by any methodology published in any paper. Once the number is determined for a year in history, then that is the number until a new methodology is published. Hansen has made so many changes to the temp record, he cannot even remember. Look at the email. He is telling his staff they need to archive their numbers at least once a year. This certainly looks like a change in policy.
If a published methodology existed for cooling the past, then we could examine it and discuss it. But these adjustments are simply unwarranted and undocumented. To use Trenberth’s word, it is travesty.
“The fact Hansen is cooling the past”
The funny thing is you’re making the same poor assumptions McIntyre makes.
From GISS:
“November 13, 2009: NOAA is no longer updating the original version of the USHCN data; it ended in May 2007. The new version 2 currently extends to July 2009. Starting today, these newer data will be used in our analysis. Documentation and programs will be updated correspondingly.”
So the current analysis for U.S. data is from USHCN.v2. Your amusing “outrage” is thus misplaced. See the 2009 Menne et al. AMS study.
http://www.ncdc.noaa.gov/oa/climate/research/ushcn/#references
Carry on.
MarkB,
Can you provide a link to the quote? It is an interesting, but it does not answer all the criticisms.
The problem is GISS began warming the years after 2000 within just days or weeks of correcting the error. 2006 was warmer than 1934 within weeks. The change did not happen all at once in Nov. 2009 as you seem to imagine. This gives the appearance that the adjustments were goal oriented. A skeptic can easily imagine the directive going out – “Team, we have to find adjustments to make sure the 2000s are the warmest year in the decade. Get to it!”
The answer from RC back in 2007 was “It’s no big deal. Adjustments happen all the time.” But that does not really explain anything, does it? Why should adjustments happen all the time? Once an analysis is completed, shouldn’t it stay the same until a new analysis is done?
There is a fundamental problem with any analysis that allows adjustments “all the time.”
“Can you provide a link to the quote?”
It’s the 2nd link in the first comment.
“The problem is GISS began warming the years after 2000 within just days or weeks of correcting the error. 2006 was warmer than 1934 within weeks. The change did not happen all at once in Nov. 2009”
You’ll need to provide evidence for this assertion that the USHCN.v2 correction was not a single step.
“This gives the appearance that the adjustments were goal oriented. A skeptic can easily imagine the directive going out – “Team, we have to find adjustments to make sure the 2000s are the warmest year in the decade. Get to it!””
Contrarians certainly “easily imagine” things. They fill in gaps in knowledge with various fantasies. To the contrarian crowd, any adjustment is seen that way. In this case, you’ve incorrectly asserted that the adjustment originated with GISS and that the adjustments are undocumented and not described. Knowledge removes ignorance.
“Once an analysis is completed, shouldn’t it stay the same until a new analysis is done? ”
The recent adjustments were the result of the new analysis (USHCN.v2) being done.
MarkB,
The adjustments had nothing to do with the 2009 Menne paper as the adjustments happened soon after the Y2K correction in 2007. To review the history, see these posts on ClimateAudit:
http://climateaudit.org/2007/08/08/a-new-leaderboard-at-the-us-open/
http://climateaudit.org/2007/09/11/crossword-puzzle-4-re-visiting-y2k/
http://climateaudit.org/2007/09/17/hansen-says-no-thanks/
One problem with the ad hominens made by McIntyre (as he quibbles over negligible changes) which involve charges that Hansen didn’t like 1934 being slightly warmer than 1998 (3rd link…and a rather childish and unsupported assertion) is that USHCN.v2, used in the GISS analysis, places both 2006 and 1998 warmer than 1934. You will see that the changes from 2007 through today are from this step. Thus, his ad hominens are once again misplaced.
ClimateAudit is not a reliable source of information, as we’ve seen.
Susann,
By the way, did you ever view the Special Report by John Coleman, co-founder of The Weather Channel on data manipulation at GISS?
You can see the video at http://www.kusi.com/weather/colemanscorner/81559212.html
“Temperature readings have been manipulated at the primary climate data centers in the United States in order to support the global warming agenda.” – John Coleman of KUSI TV and founder of The Weather Channel.
Susann,
Here is a paper written by Joseph D’Aleo regarding the examination by E. Michael Smith. D’Aleo and Smith werer the two people Coleman interviewed for the Special Report.
Click to access NOAAroleinclimategate.pdf
If this is the best you’ve got, you guys are sunk. I’m pretty sure these two guys didn’t know what they were talking about. For example I checked their claim that GISS used only, what was the number? 6, stations in CA — 3 in the SF area and a further 3 in So. Cal. They specifically pointed out that no stations in the valley or in the Sierras were used in the analysis.
After making this somewhat unbelievable claim they showed a chart for Davis (in the central valley) which showed homogenized vs. unhomogenized data. After watching the show I went to GISS and looked up the data for Davis. Yep, there it was, with a record from the late 19th C right up to 2009 and a continuous record from about 1905. Right there in the central valley of California, a place where they claimed GISS was not using any data.
If it was this easy to catch them in an error — and it took me about five minutes to check their statement out — how many other problems with their argument might there be?
Well, for one the warm vs. cool locations argument wasn’t very good. The reason for this is that the SAT analyses all deal in anomalies and not absolute temps and anomalies have large spatial coherence and don’t care whether a station is in a relatively warm or cool location.
Also, I just checked the stations in the vicinity of Davis (click on the * next to the station name) and there were at least a dozen stations within less than 100 miles of Davis which were reporting in 2009. That is considerably more than the number 6 or whatever it was for the whole state.
Rattus,
I have not confirmed what Smith is reporting he found. I am simply passing it on.
It is possible the stations are reporting data but the data is not being used in the GISS calculations. At least, that is what I understood Smith to say. Can you find out if data from the stations you found are actually used in the calculations? If it is, then KUSI should broadcast a retraction.
Yes, it is real easy — just go to the GISS station data page and type in the name of your favorite station. They are all used in the analysis, the question is “were they still reporting in 2009?”. If the answer is yes, then they are being used in the computing of the current global anomalies. An easy and fun thing to do is to click on the “*” next to the station name to get a list of stations which are nearby, where nearby seems to mean up to 1200km away, which is the maximum distance used for the intepolation of anomalies, not absolute temperatures. This list also seems to be limited to about 35 stations, so if an area is heavily monitored (the US, for example) the maximum distance away will be less.
BTW, data from stations not in the GHCN database may still be used in the CRU record since they get data directly (and under NDA) from national MET services. For example there are at least two METAR stations reporting in Bolivia currently, but they do not appear in the GHCN database. Why not? You would have to ask the Bolivian met service. GISS uses only the publically available data from GHCN, supplemented with data from USHCN for the US.
Rattus,
I’m sorry but there is a tremendous leap of faith in your post. Based on the report by KUSI, there is reason to believe not all of the stations are being used in the calculations. You did not provide me with any information to convince me otherwise.
Please visit Chiefio, E. Michael Smith. Smith is the computer programmer you saw on KUSI explaining what he had found. His blog on the topic is at http://chiefio.wordpress.com/gistemp/
If I understand his findings correctly, cold temperature stations were dropped from the “active list” even though they are still reporting. Maybe I misuderstand. But if you have a problem with his report, take it to him. I will be interested to hear his reply.
The only way he could tell would be by reading the code which is posted online and to my knowledge does nothing of the sort. Since it has been out there for a couple of years, and people at climateaudit and clear climate code have been going over it, if something like this were happening it would have been all over the place by now.
As far as “cold” stations being dropped: there were a lot of stations in the old Soviet Union which went off line when the USSR collapsed in the early nineties. That much is true. But taking this and generalizing it is just bunkum. Read the documentation at GISS. It explains the basic procedure quite clearly. If you want more detail, read the papers cited therein. The only stations which are dropped from the base data sets are urban stations for which a correction factor cannot be derived.
I just don’t understand how you cannot accept the provable fact that stations which are are clearly included in the record are included in the record. If you are so suspicious, you code hop on over to Clear Climate Code and download their Python implementation of GISTemp and run it yourself. Guess what? It produces exactly the same results as GISTemp itself.
BTW, it looks like CCC have been at work on their version of GISTemp for about 6 months or so, although they did appear to take a about a year hiatus. If you download and run the program you can see the station list in input/v2.inv. Knock your socks off.
I have read so many articles or reviews about the blogger lovers however this
paragraph is in fact a nice piece of writing,
keep it up.