Of Climategate, constabularies and Copenhagen: Gavin Schmidt’s ever-changing story

As I have mentioned in previous posts and comments on the posts/comments of others in the past few years, when Climategate burst forth into the blogosphere circa Nov. 19, 2009 I was very much a neophyte. Consequently, there’s an awful lot I did not see then – and did not know then – that I see and know now.

I’ve been trying to complete Of Climategate, constabularies and Copenhagen: coincidences worth considering Part 2 for the last 5 days, week but I keep stumbling across stuff that diverts my attention and/or changes my perspective!

The most recent “detour” on my journey through postings past was precipitated by someone whose opinion I greatly respect: Andrew Montford, aka Bishop Hill [Aug 20, 2011 at 8:23 AM] – and brilliant author of The Hockey Stick Illusion (now available on Kindle) and of the GWPF commissioned September 2010 The Climategate Inquiries.

In my comment (challenged by Andrew) I had remarked [Aug 20, 2011 at 3:19 AM] (inter alia):

Quite possibly [Gavin’s] Nov. 23 “reconstruction” was an attempt to cover his “embellishments” when spinning to [NYT’s Andy] Revkin. But it would be a very convenient comment to highlight for the Norfolk Constabulary (and/or Scotland Yard’s “eCrime Unit” who were “assisting”), wouldn’t it?! Nah … must be pure coincidence that Steve’s CA and Jeff’s TAV were the only 2 “beneficiaries” mentioned in Gavin’s Nov. 23 “reconstruction” whose owners were initially contacted by the police. But I digress …

Gavin was aware of this on Nov. 17. Three days later (Nov. 20) he claims that there was a “failed attempt to upload”. Yet by Nov. 23, this “failed attempt to upload” had magically been transformed into 4 downloads (for which there is absolutely no corroboration)! In short, it appears that Gavin has been telling different stories to different people at different times.

YMMV, but in my books this is not a particularly good indicator of veracity. Not to mention that “Honesty is the best policy” is not a motto or maxim that immediately springs to mind when one thinks of The Team and its close associates

Andrew, who is perhaps more willing than I to give the benefit of the doubt to a member of The Team. had responded,

In Gavin’s comment on the “CRU context” thread, he says:

somebody hacked into the RC server from an IP address associated with a computer somewhere in Turkey, disabled access from the legitimate users, and uploaded a file FOIA.zip to our server.

This suggests they were successful. It’s possible to read more into this than is actually there. The “attempt” angle might be simply that the whole plan came to nothing

This sent me off on my latest “detour” which I began at Steve McIntyre’s Nov. 16/09 Miracles and Strip Bark Standardization” which he had whimsically concluded with:

In this same thread was the very un-noticed (and in hindsight somewhat cryptical but, considering the context mischieviously witty) Nov 17, 2009 at 5:24 AM “A miracle just happened.” comment.

[Please Note: All bolds in any quotations below have been added by me -hro]

This particular thread appears to have ended with a comment on Nov 17, 2009 at 9:24 PM – but with no mention of the “A miracle just happened” comment. I leave it as an exercise to the reader to evaluate the validity of some of the 13 “trackbacks” to this particular post. Those of Dec. 9 and 10 from certain quarters are … curious. But I digress …

On November 19 at 9:20 p.m., following a number of posts which contained samples from the Climategate E-mail archive, in a thread she had named “Real files or fake?” Lucia indicated she had sent a link to Gavin – although she did not specifiy whether it was to her thread or to the post at JeffID’s containing the link to the server from which the archive could be downloaded. But an hour later, Lucia explained:

Gavin emailed me out of the blue. He told me the link was down at JeffId’s. I’d taken a screen shot so I sent the screen shot to Gavin. I don’t know if Gavin’s efforts led to getting the link down, or if that .ru server is down due to the link going viral. I mean … even though the link is not posted, do you have any idea how many people must be slamming that server?[emphasis added -hro]

The first MSM article I recall reading at the time was Andy Revkin’s in the NYT’s Environment section: a piece datelined Nov. 20:

[Gavin Schmidt] said the breach at the University of East Anglia was discovered after hackers who had gained access to the correspondence sought Tuesday to hack into a different server supporting realclimate.org, a blog unrelated to NASA that he runs with several other scientists pressing the case that global warming is true.

The intruders sought to create a mock blog post there and to upload the full batch of files from Britain. That effort was thwarted, Dr. Schmidt said, and scientists immediately notified colleagues at the University of East Anglia’s Climatic Research Unit.

Revkin also did a Nov. 20 post on the dot.earth blog:

Before they propagated, the purloined documents, nearly 200 megabytes in all, were uploaded surreptitiously on Tuesday to a server supporting the global warming Web site realclimate.org, along with a draft mock post, said Gavin Schmidt, a NASA climate scientist managing that blog. He pulled the plug before the fake post was published.

A slightly different story. And in a Nov. 22 update to this blogpost, Revkin wrote:

It remains interesting that before they were placed on an ftp site and dispersed across the Internet, someone tried to plant them on Realclimate.org and publish a mock post linking to them.

But the one thing all three of these excerpts have in common is: Look, Ma! No downloads. Not on the 20th, and not on the 22nd.

Nor was there any hint of any alleged “downloads” in Gavin’s own post of Nov. 20, that he had dubbed “The CRU Hack“. It begins:

As many of you will be aware, a large number of emails from the Climatic Research Unit (CRU) at the University of East Anglia webmail server were hacked recently (Despite some confusion generated by Anthony Watts, this has absolutely nothing to do with the Hadley Centre which is a completely separate institution). As people are also no doubt aware the breaking into of computers and releasing private information is illegal, and regardless of how they were obtained, posting private correspondence without permission is unethical. We therefore aren’t going to post any of the emails here. We were made aware of the existence of this archive last Tuesday morning when the hackers attempted to upload it to RealClimate, and we notified CRU of their possible security breach later that day. [pdf of page as of Aug. 19 2011]

There’s that alleged “attempted upload” again, but no “downloads”. The perceptive reader will, however, notice Gavin’s artful attempt at misdirection. This carefully crafted exercise in damage control and spin ends with the undated, unlinked and unsourced:

Update: The official UEA statement is as follows:

“We are aware that information from a server used for research information
in one area of the university has been made available on public websites,”
the spokesman stated.

“Because of the volume of this information we cannot currently confirm
that all of this material is genuine.”

“This information has been obtained and published without our permission
and we took immediate action to remove the server in question from
operation.”

“We are undertaking a thorough internal investigation and we have involved
the police in this enquiry.”

There are 1000+ comments in this thread which Gavin closed on 23 Nov 2009 @ 12:37 AM, redirecting readers to a new thread which he dubbed: “The CRU Hack: Context”

The first comment in this “context” thread has a timestamp of 23 Nov 2009 @ 12:33 AM

Here’s a screen capture of the full text (as of Aug. 22, 2011) of Gavin’s CRU context comment:

There are some discrepancies between the above and that which can be found in the Nov. 23 post in which Steve McIntyre documented Gavin’s context comment in its entirety (or at least its entirety at the time he’d grabbed it – the edited time was not included, nor the link back to CA)

For the record, here’s Gavin’s 156. in its entirety as captured by Steve (and bolded in parts by me):

There seems to be some doubt about the timeline of events that led to the emails hack. For clarification and to save me going through this again, this is a summary of my knowledge of the topic. At around 6.20am (EST) Nov 17th, somebody hacked into the RC server from an IP address associated with a computer somewhere in Turkey, disabled access from the legitimate users, and uploaded a file FOIA.zip to our server. They then created a draft post that would have been posted announcing the data to the world that was identical in content of the comment posted on The Air Vent later that day. They were intercepted before this could be posted on the blog. This archive appears to be identical to the one posted on the Russian server except for the name change. Curiously, and unnoticed by anyone else so far, the first comment posted on this subject was not at the Air Vent, but actually at ClimateAudit (comment 49 on a thread related to stripbark trees, dated Nov 17 5.24am (Central Time I think)). The username of the commenter was linked to the FOIA.zip file at realclimate.org. Four downloads occurred from that link while the file was still there (it no longer is).

The use of a turkish computer would seem to imply that this upload and hack was not solely a whistleblower act, but one that involved more sophisticated knowledge. If SM or JeffID want to share the IPs associated with the comments on their sites, I’ll be happy to post the IP address that was used to compromise RC.

Wow … look at that, folks! There they are in all their shining glory … 4 alleged downloads! And we’re supposed to accept this claim because, well, because Gavin said so, I guess!

Jeff posted the following comment:

Indeed, Gavins RC post was odd. I wouldn’t have known about it except that someone copied it to tAV. RC locked their thread to his post for so long that the posts which had backed up pushed his email from 86 to 140 something. It sat for hours.

What is really odd is that I offered the number to them days before they publicly asked. He emailed me personally to ask again – really strangely!!??!!

But let’s take a closer look at how Gavin chose to introduce this apparent “standalone” Nov. 23 “context” comment:

There seems to be some doubt about the timeline of events that led to the emails hack. For clarification and to save me going through this again,

This suggested to me that Gavin must have been getting lots of questions about the “timeline of events” … so I went off in search of these doubting posters. I found a grand total of one (but I must confess I only looked in the thread in which he’d posted the comment):

142.Was there an actual attempt to “hack” RealClimate or was there just someone attempting to post (as was done at other blogs)? If an actual hack attempt, has it been reported to the police? Details?

[Response: Yes. No. Here. – gavin]

Comment by TCO — 23 Nov 2009 @ 3:21 PM

The “Here” linked to 156. above. Considering the questions, his answers were not exactly illuminating, were they?! However, it was certainly a convenient comment from which to hang his latest “reconstruction”. But I shall attempt to translate from Gavinesque (not something I’ve had occasion to do before) …

There were 3 questions and an equal number of one-word answers. Assuming that the order of Gavin’s answers matched the questions, I guess one could take one’s pick as to whether there was “an actual attempt” to ‘hack’or it was just someone posting a comment (“as was done at other blogs”). Gavin’s “No.” (in response to “has it been reported to the police”) well, depending on what you picked for the “Yes”, this could be problematic: if it was an actual hack surely it “would have been reported to the police”, and he wouldn’t need to report it to the police if someone was merely attempting to post a comment, “as was done at other blogs”. Oh, well … perhaps he just wasn’t thinking … and the devilment was in his “Details”

But there’s something really odd about these “Details” of this Nov. 23 reconstruction. First of all, remember the “out of the blue” E-mail Gavin had sent to Lucia on Nov. 19 within 20 minutes of Mosher’s posting of the first E-mails from the archive? As Lucia explained:

He told me the link was down at JeffId’s. I’d taken a screen shot so I sent the screen shot to Gavin

As it turns out, there’s actually more to this than meets the eye. According to The Mosher Timeline the full context of Gavin’s “concern” (for want of a better word) about the link being down at Jeff’s, is as follows:

Date: Thu, 19 Nov 2009 15:48:21 -0500
From: Gavin Schmidt
To: lucia liljegren
Subject: a word to the wise

Lucia, As I am certain you are aware, hacking into private emails is very illegal. If legitimate, your scoop was therefore almost certainly obtained illegally (since how would you get 1000 emails otherwise). I don’t see any link on Jeff-id’s site, and so I’m not sure where mosher got this from, but you and he might end up being questioned as part of any investigation that might end up happening. I don’t think that bloggers are shielded under any press shield laws and so, if I were you, I would not post any content, nor allow anyone else to do so. Just my twopenny’s worth

Gavin

But according to his Nov. 23 “reconstruction” of the Details, Gavin already had the link in the “draft/mock post” (with contents “identical” to that on Jeff’s blog) that he’d “thwarted”. And he allegedly had FOIA.zip (with contents “identical” to FOIA2009.zip – which he’d been sitting on since Nov. 17; so he must have known and been able to verify the emails Steve Mosher was posting, notwithstanding the (temporarily) invisible link. Right?! But whatever his reason for this “out of the blue” E-mail might have been, once Lucia kindly sent him the screen capture he need have no further doubts.

But wait, there’s more … and, sorry folks… but … it’s worse than we thought.

You see, 3 days before Gavin’s Nov. 23 “reconstruction” of the Details, (on Nov. 20 sometime before 9:20 P.M. New Zealand time, which unless I’m mistaken, would have been sometime before 10:20 A.M. U.K. time, on the same day), Phil Jones (who had declined to comment either to Revkin or to the Guardian – although I don’t know whether this was before or after he spoke to Ian Wishart of TGIF … my guess is after) gave an “exclusive interview” to Wishart. Here’s what he had to say:

The director of Britain’s leading Climate Research Unit, Phil Jones, has told Investigate magazine’s TGIF Edition tonight that his organization has been hacked, and the data flying all over the internet appears to be genuine.

In an exclusive interview, Jones told TGIF, “It was a hacker. We were aware of this about three or four days ago that someone had hacked into our system and taken and copied loads of data files and emails.”

“Have you alerted police”

“Not yet. We were not aware of what had been taken.”

Jones says he was first tipped off to the security breach by colleagues at the website RealClimate.

“Real Climate were given information, but took it down off their site and told me they would send it across to me. They didn’t do that. I only found out it had been released five minutes ago.”

TGIF asked Jones about the controversial email discussing “hiding the decline”, and Jones explained what he was trying to say….

Excuse me?! It is now Nov. 20, and according to Gavin’s Nov. 23 “reconstruction” of the Details, he notified UEA (either “immediately” or “later that day”) on Nov. 17, but he hasn’t told them what was in this alleged “uploaded” and (alleged 4 times downloaded) FOIA.zip.

And he didn’t “send it across”?! Poor Phil! With friends like Gavin, who needs enemies, eh?! Perhaps Gavin sent this alleged FOIA.zip via slow-boat to China, but since he also had the link included in this “thwarted” “mock/draft” post “that was “identical” to the one on Jeff’s blog, surely he could have copied it into an E-mail and sent it to poor Phil so he could get the archive and figure out “what had been taken”.

Fast forward to February 4, 2010

First a h/t to Duke C. at Bishop Hill for pointing me in the direction of yet another embellishment on the Details. Although it’s somewhat difficult to determine who the source might be. The Guardian had an article that got off to a reasonably good start:

Climate emails: were they really hacked or just sitting in cyberspace?

Slack security or subversion at the university may have led to ‘unintentional sharing’, making the police investigation pointless

More than two months after the moment that thousands of confidential emails, documents and computer code from the University of East Anglia (UEA) was released online it remains a mystery who was behind the hack.

Even Sir David King, the government’s former chief scientist, remains confused. This week, he sought to blame the leak on a foreign intelligence agency, only to admit later he had no evidence.

The university called in police last November, insisting they were victims of a criminal “theft” of data. Under Superintendent Julian Gregory, a group was pulled together from the counter-terrorism squad and Scotland Yard’s electronic crimes unit, which also included two officers from the national domestic extremism team who have expertise in pursuing “climate extremists”.

So far, the police investigation has got nowhere. It is not even clear whether the crime of computer data interception has actually occurred. What if the hacker was given a legitimate password? What if the data was accidentally open to public access?

The known facts are these. Over the weekend starting Friday 13 November, someone copied files from a backup server at the UEA’s Climate Research Unit (CRU) in Norwich. They were then posted anonymously on the internet and various bloggers were alerted

[…]

UAE has confirmed that all of this material was simply sitting in an archive on a single backup CRU server, available to be copied.

The Guardian has carried out a detailed analysis of the emails and documents.

And it took four reporters to do the job: Lead by the fearless, and infamous David Leigh, this team also included Charles Arthur, Rob Evans and Fred Pearce.

This article is quite interesting in its own right (or more to the point its own wrongs!)

On Tuesday 17 November, the leaked data was passed anonymously to the small group who, for some time, had been targeting CRU and its director Phil Jones. The technique involved hacking into the server of climate science blog RealClimate, and then extruding the material via a series of exotic foreign “proxy” servers.

Oh, dear! Poor Phil. But that aside, could someone please tell me what an “exotic” foreign proxy server is, and how is it different from the non-exotic kind?! But I think what’s being suggested here is a new twist on Gavin’s “reconstruction” Details: the alleged upload to RealClimate was for the purpose of renaming the file and then uploading it to a Russian server.

The article continues:

The very first release was a sort of prank. Nasa scientist Gavin Schmidt in New York, an opponent of the sceptics, says that at 6.20am his time, someone tried to upload the files onto his own RealClimate website via a Turkish server.

The hacker seems to have used a technique called “privilege escalation vulnerability” to become an administrator, rather than an ordinary user of the site.

Oh, look at that … now this alleged “upload” has morphed into “sort of a prank” done by those who had sufficiently “sophisticated knowledge” to use a “Turkish computer” (according to Gavin’s original “reconstruction” Details). Hmmm … I wonder if a “Turkish computer” would be considered as an exotic or non-exotic proxy server.

OMG, I almost overlooked the “privilege escalation vulnerability” … now that is scary stuff. But there are precautions one can take: On the WordPress platform, the remote publishing functionality is disabled by default – thereby significantly reducing the impact. And it only seems to be a problem if the user already has Author or Contributor access level. But here’s the really sad thought: if these events had happened two weeks later – i.e. circa Dec. 9 or shortly thereafter – WordPress would have had a fix. And this would never have happened. But then again, I see that:

“The hacking of the RealClimate blog exploited the fact that its wordpress flatform (sic) has security holes well known to hackers.”

Hmmm … maybe not. Oh, well perhaps RealClimate needs to upgrade to the WordPress platform, and put some basic security measures in place.

Here’s something that jumped out at me:

Schmidt says the hacker “disabled access from the legitimate users, and uploaded a file FOIA.zip to our server. They then created a draft post”.

“disable access from the legitimate users” struck me as a rather unusual construction when I read it the first time. And the first time I read it was in Gavin’s Nov. 23 “reconstruction” of the Details. So those two sentences were lifted verbatim from Gavin’s very convenient comment [in which he only mentioned two “beneficiaries”: Steve and Jeff … and, of course, he had very helpfully added a link pointing to the “A miracle just happened” comment at CA]. Can’t imagine how this quartet would have found it, there were so many comments there. Oh, wait a minute, Gavin must have sent them the link.

Leigh et al continued that part of their story with a note of authenticity:

[The draft] read as follows: “We feel that climate science is, [etc]”

with the hyperlink to the Russian server (and the correct filename FOI2009.zip)

Now, here’s the great mystery: Did the one of the quartet grab this from Jeff’s site, or did Gavin figure out how to paste this quarantined “draft/mock” post into an E-mail (as he was unable to do for poor Phil). Or did he simply forward the screen capture that Lucia had sent him on the 19th?

The article continues:

Schmidt swiftly spotted the hack and took it down. He also alerted CRU in Norwich. But even as he did that, a cryptic comment appeared on McIntyre’s site. “A miracle has happened,” it said, providing a link via the RealClimate website which immediately led to four unidentified downloads

Oh, there they are again: the four alleged downloads. Perhaps one of these four “analysts” could explain to me how on Gaia’s green earth anyone could “download” anything from something that has already been “taken down”?

But on Feb. 9, 2010, Fred Pearce went solo and the alleged “downloads” had disappeared:

Search for hacker may lead police back to East Anglia’s climate research unit

On 17 November at 6.20am EST, someone tried to upload the zip file containing the CRU emails onto the RealClimate website via a Turkish server. They then created a draft post that read: “We feel the climate science is, in the current situation, too important to be kept under wraps. We hereby release a random selection of correspondence, codes and documents. Hopefully, it will give some insight into the science and the people behind it.” It gave 20 samples from the emails and a link to download more.

Gavin Schmidt, the Nasa scientist running the site, swiftly spotted it and took it down. Having read the files he alerted CRU. But even as he did that, a cryptic comment appeared on McIntyre’s ClimateAudit site at 7.24am. “A miracle has happened,” it said, providing a link via the RealCimate website. Nobody noticed this initially or tried to use the link, which in any case would not have worked.

Oh, and btw if you’re going to quote verbatim, please do some fact-checking, first. The message left on the ClimateAudit blog, in a post entitled, “Miracles and Strip Bark Standardization” by the anonymous poster “RC” was (and still is):

A miracle just happened

Forget about Gavin, and his ever-changing story.

UPDATE 08/24/2011 01:22 AM See also: What the world needs now … addendum to Gavin Schmidt’s ever-changing story

UPDATE 08/28/2011 02:14 PM Corrected links to Guardian articles of Feb. 4 and Feb. 9

UPDATE 12/6/2011 12:27 PM See also Climategate: Of thumbnails, big pictures and timing