The leading climate change protagonists are, wait for it, in denial! How delicious. Gavin Schmidt practically runs the blogsite RealClimate. He is a co-conspirator with Phil Jones, Michael Mann, Kevin Trenberth, Jim Salinger and others who are now identified as part of the cabal which have foisted one of the biggest frauds ever in human history. Schmidt has used RealClimate as a propaganda organ, defaming opponents, claiming scientific superiority, blocking critical posts, exercising highly partial and self-serving censorship, etc.
Since the CRU e-mail scandal he has tried to minimize the whole affair. "It's just a group of highly motivated scientists using a few infelicitous expressions" he suggests. But underneath it all, the global warming consensus remains as strong as ever. Either Schmidt knows, and is deliberately obfuscating the case, or he is willingly suspending disbelief. For what has emerged is that the whole data set "proving" that global warming is taking place is at best unreliable, at worst, corrupted.
Now this is such a basic issue, few people are bold enough to ask about it. Everyone just assumes that if scientists assert that the earth is warming there must be good evidence upon which the assertion rests. But the most embarrassing question is, How do we know the earth's temperature is rising? Reliable satellite temperature data has only been in existence for a very short period of time. Surface temperature data goes back farther. Where does the most "authoritative" global surface temperature record reside? Yes, you guessed it: at the Climate Research Unit of the University of East Anglia, Hadley, UK--the very institution from which the e-mails have been released.
Further study of the leaked material shows that the entire data series purporting to show that the earth's temperatures are rising is nothing more than a cheap parlour trick. Even the mainstream media are now picking this up--obviously analysts and reporters have been put to work. (In other words the shock and horror is spilling out into those media organs who until this time have been stentorian cheerleaders for the messianic cause of preventing humanities destruction.) Here is the latest from CBS.
In addition to e-mail messages, the roughly 3,600 leaked documents posted on sites including Wikileaks.org and EastAngliaEmails.com include computer code and a description of how an unfortunate programmer named "Harry" -- possibly the CRU's Ian "Harry" Harris -- was tasked with resuscitating and updating a key temperature database that proved to be problematic. Some excerpts from what appear to be his notes, emphasis added:We repeat: the whole case ultimately rests on the world temperature data series which the e-mails reveal to be corrupted and unreliable. Upon this festering pile of contaminated garbage the UN and member governments (including New Zealand, by means far more foul than fair) are trying to build a destructive edifice of regulation and tax and vast extensions of powers of government.
"I am seriously worried that our flagship gridded data product is produced by Delaunay triangulation - apparently linear as well. As far as I can see, this renders the station counts totally meaningless. It also means that we cannot say exactly how the gridded data is arrived at from a statistical perspective - since we're using an off-the-shelf product that isn't documented sufficiently to say that. Why this wasn't coded up in Fortran I don't know - time pressures perhaps? Was too much effort expended on homogenisation, that there wasn't enough time to write a gridding procedure? Of course, it's too late for me to fix it too. Meh.
"I am very sorry to report that the rest of the databases seem to be in nearly as poor a state as Australia was. There are hundreds if not thousands of pairs of dummy stations, one with no WMO and one with, usually overlapping and with the same station name and very similar coordinates. I know it could be old and new stations, but why such large overlaps if that's the case? Aarrggghhh! There truly is no end in sight... So, we can have a proper result, but only by including a load of garbage!
"One thing that's unsettling is that many of the assigned WMo codes for Canadian stations do not return any hits with a web search. Usually the country's met office, or at least the Weather Underground, show up – but for these stations, nothing at all. Makes me wonder if these are long-discontinued, or were even invented somewhere other than Canada!
"Knowing how long it takes to debug this suite - the experiment endeth here. The option (like all the anomdtb options) is totally undocumented so we'll never know what we lost. 22. Right, time to stop pussyfooting around the niceties of Tim's labyrinthine software suites - let's have a go at producing CRU TS 3.0! since failing to do that will be the definitive failure of the entire project.
"Ulp! I am seriously close to giving up, again. The history of this is so complex that I can't get far enough into it before by head hurts and I have to stop. Each parameter has a tortuous history of manual and semi-automated interventions that I simply cannot just go back to early versions and run the update prog. I could be throwing away all kinds of corrections - to lat/lons, to WMOs (yes!), and more. So what the hell can I do about all these duplicate stations?..."
As the leaked messages, and especially the HARRY_READ_ME.txt file, found their way around technical circles, two things happened: first, programmers unaffiliated with East Anglia started taking a close look at the quality of the CRU's code, and second, they began to feel sympathetic for anyone who had to spend three years (including working weekends) trying to make sense of code that appeared to be undocumented and buggy, while representing the core of CRU's climate model.
One programmer highlighted the error of relying on computer code that, if it generates an error message, continues as if nothing untoward ever occurred. Another debugged the code by pointing out why the output of a calculation that should always generate a positive number was incorrectly generating a negative one. A third concluded: "I feel for this guy. He's obviously spent years trying to get data from undocumented and completely messy sources."
Programmer-written comments inserted into CRU's Fortran code have drawn fire as well. The file briffa_sep98_d.pro says: "Apply a VERY ARTIFICAL correction for decline!!" and "APPLY ARTIFICIAL CORRECTION." Another, quantify_tsdcal.pro, says: "Low pass filtering at century and longer time scales never gets rid of the trend - so eventually I start to scale down the 120-yr low pass time series to mimic the effect of removing/adding longer time scales!"
Now we know, of course, why Dr Phil Jones, head of the CRU, together with his cabal have worked strenuously to resist making their base temperature data available. They must have known all along what skeletons were in the closet--or at least suspected, but probably preferred to turn a blind eye. This is where the warmists Achilles' Heel lies exposed.
At the end of this post, for those who wish to take the time, we have copied over a guest post which has appeared on Anthony Watts's site. It is an extensive record of an attempt to get access to the temperature data of CRU upon which the whole case for global warming rests. The requests and e-mail exchange is interspersed with the (then) behind-the-scenes conspiring not to comply. This conspiracy and orchestrated litany of obfuscation to hide data, strikes at the very root of scientific credibility.
In the meantime, here is a summary of some of the most fundamental issues which are emerging from the e-mails and their fallout from Andrew Bolt:
And now, the blow-by-blow account of actual obfuscation and deception on the part of CRU as it faces legitimate Freedom of Information Act requests--information which should have been disclosed years ago--and, we believe, would have been if the data had been genuine in the first place. It is only those who fear the truth who try to hide it.
Wednesday, November 25, 2009 at 11:05am
THREE weeks ago Prime Minister Kevin Rudd named me as part of an international conspiracy to spread lies about global warming.
How I laughed. But I’m not laughing now. Emails leaked at the weekend show there is indeed a conspiracy to deceive the world - and Rudd has fallen for it.
This conspiracy comprises a group of warming scientists who have been central in spreading the false claim that the world has never been hotter and man’s gases are to blame.
It’s come to light after nearly 4000 emails and documents were stolen from the Climatic Research Unit of the University of East Anglia and dumped on the internet by what is almost certainly a whistleblower.
What they reveal is perhaps the greatest scientific scandal of our time - a conspiracy by warmist scientists to fudge statistics, sack sceptical scientists, block the release of data to prevent checking, illegally destroy data, deceive reporters, censor sceptical papers, and hide errors in their work.
Most extraordinary are the emails in which these scientists admit to each other what they’ve never confessed to the world - that the world is not warming as their theories predicted.
In fact, it’s been cooling since 2001.
Cried one, IPCC co-author Kevin Trenberth, in an email to other members of this conspiracy: “The fact is that we cannot account for the lack of warming at the moment and it’s a travesty that we can’t.”
These are not some obscure scientists. Rather, they include co-authors of the reports of the United Nations’ Intergovernmental Panel on Climate Change - which Rudd cites as his proof that warming is “happening and it’s caused by human activity”.
They include Phil Jones, head of the CRU unit from which the emails were taken - a unit that Britain’s former chief scientific adviser, Sir David King, said “set the agenda for the major research effort” in climate science.
They include Pennsylvania State University’s Michael Mann and CRU deputy director Keith Briffa, both IPCC co-authors, who also produced the two studies that most convinced journalists of the false claim that it’s now hotter than the Medieval Warm Period just 800 years ago.
They also include scientists responsible for the HadCRU data - one of the four main measurements of the world’s temperature today.
All this may sound too James Bond-like to be true. Yet only three years ago we were warned that many of these same people had indeed created a network that was distorting science.
In 2006 the United States House of Representatives Energy and Commerce Committee commissioned eminent statisticians to check Mann’s famous “hockey stick” - a graph used by the IPCC to claim today’s temperatures were the highest for thousands of years.
Their report not only found that Mann’s work was too flawed to be relied upon, just as a retired petroleum engineer, Steve McIntyre, had first said, but that there was now a “clique” of 43 climate scientists stifling true debate, with Mann, Jones and Briffa all named.
Few heeded the warning. And so that clique morphed into a conspiracy that has helped to panic the world - including Rudd - into spending billions on a scare that may not in fact exist.
That’s the background. Now here are just some of the emails.
FIDDLING DATA: “Hide the decline”
Phil Jones tells Michael Mann and others how he made his data show warming:
I’ve just completed Mike’s Nature trick of adding in the real temps to each series for the last 20 years (ie from 1981 onwards) and from 1961 for Keith’s to hide the decline.
Mick Kelly, Professor of Climate Change at Jones’ university, on hiding recent cooling:
Anyway, I’ll maybe cut the last few points off the filtered curve before I give the talk again as that’s trending down as a result of the end effects and the recent cold-ish years.
A CRU programming code for dealing with tree-ring data:
Uses corrected MXD but shouldn’t usually plot past 1960 because these will be artificially adjusted to look closer to the real temperatures.
From green entrepreneur Adam Markham to the CRU, ordering better propaganda:
(WWW Australia) are worried that this may present a slightly more conservative approach to the risks than they are hearing from CSIRO. In particular, they would like to see the section on variability and extreme events beefed up if possible.
FAITH, NOT SCIENCE: “This is all gut feeling”
Phil Jones shows his faith:
I would like to see the climate change happen, so the science could be proved right, regardless of the consequences. This isn’t being political, it is being selfish.
Phil Jones to his CRU staff:
I hope you’re not right about the lack of warming lasting till about 2020. I’d rather hoped to see the earlier Met Office press release with Doug’s paper that said something like - half the years to 2014 would exceed the warmest year currently on record, 1998!
From Phil Jones to Adelaide-born Tom Wigley, now of the US University Corporation for Atmospheric Research, saying he cannot believe the Medieval Warm Period really was warmer:
Bottom line - there is no way the MWP (whenever it was) was as warm globally as the last 20 years ... this is all gut feeling, no science, but years of experience of dealing with global scales and variability.
From Phil Jones to Michael Mann, on the death of Australian sceptic John Daly:
In an odd way this is cheering news!
COVERING UP: “Destroy the emails”
Phil Jones warns Michael Mann that Steve McIntyre and Prof Ross McKittrick, two sceptics who first debunked Mann’s “hockey stick”, are now wanting to check CRU data:
If they ever hear there is a Freedom of Information Act now in the UK, I think I’ll delete the file rather than send to anyone ... We also have a data protection act, which I will hide behind.
From Phil Jones to Michael Mann and others:
PS I’m getting hassled by a couple of people to release the CRU station temperature data. Don’t any of you three tell anybody that the UK has a Freedom of Information Act!
Phil Jones to American climatologist and IPCC lead author Benjamin Santer:
I did get an email from the FoI person here early yesterday to tell me I shouldn’t be deleting emails - unless this was normal deleting to keep emails manageable!
Phil Jones to Michael Mann, just three weeks after an FOI request from sceptic David Holland:
Can you delete any emails you may have had with Keith re AR4?
Keith will do likewise ... Can you also email Gene and get him to do the same?
SILENCING SCEPTICS: “Beat the crap out of him”
Ben Santer on a sceptical climatologist :
Next time I see Pat Michaels at a scientific meeting, I’ll be tempted to beat the crap out of him.
Phil Jones on cheating on deadlines to sneak warmist material into the IPCC:
Ammann/Wahl - try and change the Received date! Don’t give those sceptic something to amuse themselves with.
Tom Wigley on ousting the editor of Geophysical Research Letters (achieved):
If you think that Saiers is in the greenhouse sceptics camp, then, if we can find documentary evidence of this, we could go through official AGU channels to get him ousted.
Phil Jones to Michael Mann on keeping two sceptics’ papers from the IPCC:
I can’t see either of these papers being in the next IPCC report. K and I will keep them out somehow - even if we have to redefine what the peer-review literature is!
Michael Mann on removing the editor of Climate Science (achieved):
How to deal with this is unclear, since there are a number of individuals with bona fide scientific credentials who could be used by an unscrupulous editor to ensure that anti-greenhouse science can get through the peer review process (Legates, Balling, Lindzen, Baliunas, Soon, and so on).
Michael Mann to the CRU’s Tim Osborn and Keith Briffa, on blocking sceptics’ comments on his RealClimate website:
We can hold comments up in the queue and contact you about whether or not you think they should be screened through or not ...
OUR THEORY ISN’T WORKING! “Where the heck is global warming?”
IPCC lead author Kevin Trenberth privately tells Mann, Santer, Wigley, Jones and leading alarmists such as Stephen H. Schneider and James Hansen that the data doesn’t show what their climate models predicted:
… where the heck is global warming? We are asking that here in Boulder where we have broken records the past two days for the coldest days on record. ... The fact is that we can’t account for the lack of warming at the moment and it is a travesty that we can’t.
WHAT DOES THIS ALL MEAN?
It doesn’t mean all global warming science is bogus. But it does mean the process used to produce it is corrupted.
It means “peer review” is too often “mates’ review”. It means sceptical scientists have not had the hearing they deserve, and leading warmist scientists have not been honest or frank.
It means the “consensus” of scientists we hear of may not actually exist and the IPCC reports cannot be trusted to be balanced.
It means that claims we’ve never been hotter are false or unproven, and much research now needs to be rechecked.
And at heart it means global warming theory is too weak to accept, being contradicted by a decade of climate.
People seem to be missing the real issue in the CRU emails. Gavin over at realclimate keeps distracting people by saying the issue is the scientists being nasty to each other, and what Trenberth said, and the Nature “trick”, and the like. Those are side trails. To me, the main issue is the frontal attack on the heart of science, which is transparency.
Science works by one person making a claim, and backing it up with the data and methods that they used to make the claim. Other scientists then attack the claim by (among other things) trying to replicate the first scientist’s work. If they can’t replicate it, it doesn’t stand. So blocking the FOIA [Freedom of Information Act]allowed Phil Jones to claim that his temperature record (HadCRUT3) was valid science.
This is not just trivial gamesmanship, this is central to the very idea of scientific inquiry. This is an attack on the heart of science, by keeping people who disagree with you from ever checking your work and seeing if your math is correct.
As far as I know, I am the person who made the original Freedom Of Information Act to CRU that started getting all this stirred up. I was trying to get access to the taxpayer funded raw data out of which they built the global temperature record. I was not representing anybody, or trying to prove a point. I am not funded by Mobil, I’m an amateur scientist with a lifelong interest in the weather and climate. I’m not “directed” by anyone, I’m not a member of a right-wing conspiracy. I’m just a guy trying to move science forwards.
The recent release of the hacked emails from CRU has provided me with an amazing insight into the attempt by myself, Steve McIntyre, and others from CA [ClimateAudit] and elsewhere to obtain the raw station data from Phil Jones at the CRU. We wanted the data that was used to make the global temperature record that is relied on to claim “unprecedented” global warming. I want to give a chronological account of the interactions. While we don’t know if all of these emails are valid, the researchers involved such as Gavin Schmidt and Michael Mann that clearly indicate that they think they are authentic They certainly fit with my experience. I have only included the relevant parts of emails, and indicated where I have snipped by an ellipsis (…).
The story actually starts with Warwick Hughes, a climate researcher who had previously been in cordial contact with Phil Jones, the lead researcher of the CRU. I find only one email in the archive (0969308954) where Phil emails Warwick, from 2000. This is in response to some inconsistencies that Warwick had found in Phil’s work:
Warwick Hughes to Phil Jones, September ‘04:
Dear Phillip and Chris Folland (with your IPCC hat on),
Some days ago Chris I emailed to Tom Karl and you replied re the grid cells in north Siberia with no stations, yet carrying red circle grid point anomalies in the TAR Fig 2.9 global maps. I even sent a gif file map showing the grid cells barren of stations greyed out. You said this was due to interpolation and referred me to Phillip and procedures described in a submitted paper. In the last couple of days I have put up a page detailing shortcomings in your TAR Fig 2.9 maps in the north Siberian region, everything is specified there with diagrams and numbered grid points.
[1] One issue is that two of the interpolated grid cells have larger anomalies than the parent cells !!!!?????
This must be explained.
[2] Another serious issue is that obvious non-homogenous warming in Olenek and Verhojansk is being interpolated through to adjoining grid cells with no stations, like cancer.
[3] The third serious issue is that the urbanization affected trend from the Irkutsk grid cell neare Lake Baikal, looks to be interpolated into its western neighbour.
I am sure there are many other cases of this, 2 and 3 happening.
Best regards,
Warwick Hughes (I have sent this to CKF)
Phil to Warwick, same email:
Warwick,
I did not think I would get a chance today to look at the web page. I see what boxes you are referring to. The interpolation procedure cannot produce larger anomalies than neighbours (larger values in a single month). If you have found any of these I will investigate. If you are talking about larger trends then that is a different matter. Trends say in Fig 2.9 for the 1976-99 period require 16 years to have data and at least 10 months in each year. It is conceivable that at there are 24 years in this period that missing values in some boxes influence trend calculation. I would expect this to be random across the globe.
Warwick,
Been away. Just checked my program and the interpolation shouldn’t produce larger anomalies than the neighbouring cells. So can you send me the cells, months and year of the two cells you’ve found ? If I have this I can check to see what has happened and answer (1). As for (2) and (3) we compared all stations with neighbours and these two stations did not have problems when the work was done (around 1985/6). I am not around much for the next 3 weeks but will be here most of this week and will try to answer (1) if I get more details. If you have the names of stations that you’ve compared Olenek and Verhojansk with I would appreciate that.
Cheers
Phil
OK, so far we have a couple of scientists discussing issues in a scientific work, no problem. But as he found more inconsistencies, in order to understand what was going on, in 2005 Warwick asked Phil for the dataset that was used to create the CRU temperature record. Phil Jones famously replied:
Subject: Re: WMO non respondo
… Even if WMO agrees, I will still not pass on the data. We have 25 or so years invested in the work. Why should I make the data available to you, when your aim is to try and find something wrong with it. …
Cheers Phil
Hmmm … not good. Or as they say in “1984″, double-plus ungood. Science can only progress if there is a free exchange of scientific data The scientific model works like this:
* A scientist makes claims, and reveals the data and methods he used to come to his conclusions.
* Other scientists who don’t agree attack the claim by (inter alia) seeing if they can replicate the result, using the first scientist’s data and methods.
* If the claims cannot be replicated, the claim is adjudged to be false.
Obviously, if the data or the methods are kept secret, the claims cannot be verified. Attacking other scientist’s claims is what what scientists do. This adversarial system is the heart of science. Refusing scientific data because someone will attack it is an oxymoron, of course they will attack it. That’s what scientists do.
When I found out about this, I couldn’t believe it. I thought, a scientist can’t do that, can they? This is science, not hide and seek. So I wrote to the University of East Anglia (of which the CRU is a Department) on September 8, 2006, saying:
I would like to obtain a list of the meteorological stations used in the preparation of the HadCRUT3 global temperature average, and the raw data for those stations. I cannot find it anywhere on the web. The lead author for the temperature average is Dr. Phil Jones of the Climate Research Unit.
Many thanks, Willis Eschenbach
I got no response from Phil Jones or anyone at CRU or UEA. So I filed a Freedom of Information act request for the data.
Now at this point, let me diverge to what was happening at CRU during this time. The first reference to Freedom of Information in their emails is from 2005, before they had received a single request. Immediately, they start to plan how to evade requests should some come in:
Tom Wigley, Former Director CRU, to Phil Jones, 21/01/2005
Phil,
…
I got a brochure on the FOI Act from UEA. Does this mean that, if someone asks for a computer program we have to give it out?? Can you check this for me (and Sarah). …
Thanks,
Tom.
Phil replies to Tom:
Tom,
…
On the FOI Act there is a little leaflet we have all been sent. It doesn’t really clarify what we might have to do re programs or data. Like all things in Britain we will only find out when the first person or organization asks. I wouldn’t tell anybody about the FOI Act in Britain. I don’t think UEA really knows what’s involved.
As you’re no longer an employee I would use this argument if anything comes along. I think it is supposed to mainly apply to issues of personal information – references for jobs etc.
..
Cheers
Phil
So the coverup starts immediately, even before the first request. “I wouldn’t tell anyone about the FOI act in Britain”.
Tom to Phil
Phil,
Thanks for the quick reply. The leaflet appeared so general, but it was prepared by UEA so they may have simplified things. From their wording, computer code would be covered by the FOIA. My concern was if Sarah is/was still employed by UEA. I guess she could claim that she had only written one tenth of the code and release every tenth line.
…
Tom
You can see how they plan to observe the spirit of the FOI Act. Claim a temporary employee isn’t really and employee so they are not covered.
Phil to Tom
Tom,
…
As for FOIA Sarah isn’t technically employed by UEA and she will likely be paid by Manchester Metropolitan University. I wouldn’t worry about the code. If FOIA does ever get used by anyone, there is also IPR to consider as well. Data is covered by all the agreements we sign with people, so I will be hiding behind them. I’ll be passing any requests onto the person at UEA who has been given a post to deal with them.
Cheers
Phil
Phil Jones has just gotten the news that FOI will apply, and immediately he starts to plan how he is going to hide from an FOI request. Cite technicalities, claim IPR rights, those are good hiding places.
The next email (1109021312) is later in 2005:
At 09:41 AM 2/2/2005, Phil Jones wrote to Michael Mann :
Mike,
…
Just sent loads of station data to Scott. Make sure he documents everything better this time ! And don’t leave stuff lying around on ftp sites – you never know who is trawling them. The two MMs have been after the CRU station data for years. If they ever hear there is a Freedom of Information Act now in the UK, I think I’ll delete the file rather than send to anyone. Does your similar act in the US force you to respond to enquiries within 20 days? – our does ! The UK works on precedents, so the first request will test it.
We also have a data protection act, which I will hide behind. Tom Wigley has sent me a worried email when he heard about it – thought people could ask him for his model code. He has retired officially from UEA so he can hide behind that. IPR should be relevant here, but I can see me getting into an argument with someone at UEA who’ll say we must adhere to it !
….
Phil
So now we have two more ways for Phil to hide from the FOI Act … along with a threat to delete the data rather than release it. Astounding. And this is before they’ve even received a single FOI request.
Mann replies to Jones:
Thanks Phil,
Yes, we’ve learned out lesson about FTP. We’re going to be very careful in the future what gets put there. Scott really screwed up big time when he established that directory so that Tim could access the data.
Yeah, there is a freedom of information act in the U.S., and the contrarians are going to try to use it for all its worth. But there are also intellectual property rights issues, so it isn’t clear how these sorts of things will play out ultimately in the U.S….
mike
Next, from February 05. Jones to Mann, cc to Hughes and Bradley (co-authors of the “hockeystick” study)
From: Phil Jones:
To: mann
Subject: Fwd: CCNet: PRESSURE GROWING ON CONTROVERSIAL RESEARCHER TO DISCLOSE SECRET DATA
Date: Mon Feb 21 16:28:32 2005
Cc: “raymond s. bradley”, “Malcolm Hughes”
Mike, Ray and Malcolm,
…
Leave it to you to delete as appropriate !
Cheers
Phil
PS I’m getting hassled by a couple of people to release the CRU station temperature data. Don’t any of you three tell anybody that the UK has a Freedom of Information Act !
The first rule of the Freedom of Information act … nobody talks about the Freedom of Information Act.
With that as a prologue, let me return to my FOI request. On February 10, 2007, I received my reply from Mr. Dave Palmer of CRU:
Dear Mr. Eschenbach
FREEDOM OF INFORMATION ACT 2000 – INFORMATION REQUEST (FOI_07-04)
Your request for information received on 28 September now been considered and I can report that the information requested is available on non-UEA websites as detailed below.
The Global Historical Climatology Network (GHCN-Monthly) page within US National Climate Data Centre website provides one of the two US versions of the global dataset and includes raw station data. This site is at: http://www.ncdc. noaa.gov/ oa/climate/ ghcn-monthly/ index.php
This page is where you can get one of the two US versions of the global dataset, and it appears that the raw station data can be obtained from this site.
Datasets named ds564.0 and ds570.0 can be found at The Climate & Global Dynamics Division (CGD) page of the Earth and Sun Systems Laboratory (ESSL) at the National Center for Atmospheric Research (NCAR) site at: http://www.cgd. ucar.edu/ cas/tn404/
Between them, these two datasets have the data which the UEA Climate Research Unit (CRU) uses to derive the HadCRUT3 analysis. The latter, NCAR site holds the raw station data (including temperature, but other variables as well). The GHCN would give their set of station data (with adjustments for all the numerous problems).
They both have a lot more data than the CRU have (in simple station number counts), but the extra are almost entirely within the USA. We have sent all our data to GHCN, so they do, in fact, possess all our data.
In accordance with S. 17 of the Freedom of Information Act 2000 this letter acts as a Refusal Notice, and the reasons for exemption are as stated below
Exemption Reason
s. 21, Information accessible to applicant via other means Some information is publicly available on external websites
I was outraged. So the next day, I made a second request:
Dear Mr. Palmer:
Thank you for your reply (attached below). However, I fear that it is totally unresponsive. I had asked for a list of the sites actually used. While it may (or may not) be true that “it appears that the raw station data can be obtained from [GHCN]“, this is meaningless without an actual list of the sites that Dr. Jones and his team used.
The debate about changes in the climate is quite important. Dr. Jones’ work is one of the most frequently cited statistics in the field. Dr. Jones has refused to provide a list of the sites used for his work, and as such, it cannot be replicated. Replication is central to science. I find Dr. Jones attitude quite difficult to understand, and I find your refusal to provide the data requested quite baffling.
You are making the rather curious claim that because the data “appears” to be out on the web somewhere, there is no need for Dr. Jones to reveal which stations were actually used. The claim is even more baffling since you say that the original data used by CRU is available at the GHCN web site, and then follow that with the statement that some of the GHCN data originally came from CRU. Which is the case? Did CRU get the data from GHCN, or did GHCN get the data from CRU?
Rather than immediately appealing this ruling (with the consequent negative publicity that would inevitably accrue to CRU from such an action), I am again requesting that you provide:
1) A list of the actual sites used by Dr. Jones in the preparation of the HadCRUT3 dataset, and
2) A clear indication of where the data for each site is available. This is quite important, as there are significant differences between the versions of each site’s data at e.g. GHCN and NCAR.
I find it somewhat disquieting that an FOI request is necessary to force a scientist to reveal the data used in his publicly funded research … is this truly the standard that the CRU is promulgating?
Thank you for your cooperation in this matter.
Willis Eschenbach
Hey, I was trying to be a nice guy, not make a public scene. On April 12, 2007, I got my second reply:
In regards the “gridded network” stations, I have been informed that the Climate Research Unit’s (CRU) monthly mean surface temperature dataset has been constructed principally from data available on the two websites identified in my letter of 12 March 2007. Our estimate is that more than 98% of the CRU data are on these sites.
The remaining 2% of data that is not in the websites consists of data CRU has collected from National Met Services (NMSs) in many countries of the world. In gaining access to these NMS data, we have signed agreements with many NMSs not to pass on the raw station data, but the NMSs concerned are happy for us to use the data in our gridding, and these station data are included in our gridded products, which are available from the CRU web site. These NMS-supplied data may only form a very small percentage of the database, but we have to respect their wishes and therefore this information would be exempt from disclosure under FOIA pursuant to s.41. The World Meteorological Organization has a list of all NMSs.
That didn’t help one bit. Without knowing which data was used, it was meaningless. They’ve tried s.21, they’ve tried s.41, neither exemption applies. So the next day, I replied:
While it is good to know that the data is available at those two web sites, that information is useless without a list of stations used by Jones et al. to prepare the HadCRUT3 dataset. As I said in my request, I am asking for:
1) A list of the actual sites used by Dr. Jones in the preparation of the HadCRUT3 dataset, and
2) A clear indication of where the data for each site is available. This is quite important, as there are significant differences between the versions of each site’s data at e.g. GHCN and NCAR.
Without knowing the name and WMO number of each site and the location of the source data (NCAR, GHCN, or National Met Service), it is not possible to access the information. Thus, Exemption 21 does not apply – I still cannot access the data.
I don’t understand why this is so hard. All I am asking for is a simple list of the sites and where each site’s data is located. Pointing at two huge piles of data and saying, in effect, “The data is in there somewhere” does not help at all.
To clarify what I am requesting, I am only asking for a list of the stations used in HadCRUT3, a list that would look like this:
WMO# Name Source
58457 HangZhou NCAR
58659 WenZhou NCAR
59316 ShanTou GHCN
57516 ChongQing NMS
etc. for all of the stations used to prepare the HadCRUT3 temperature data.
That is the information requested, and it is not available “on non-UEA websites”, or anywhere else that I have been able to find.
I appreciate all of your assistance in this matter, and I trust we can get it resolved satisfactorily.
Best regards,
I received another letter, saying that they could not identify the locations of the requested information. I wrote back again, saying:
Dear Mr. Palmer:
It appears we have gone full circle here, and ended up back where we started.
I had originally asked for the raw station data used to produce the HadCRUT3 dataset to be posted up on the UEA website, or made available in some other form.
You refused, saying that the information was available elsewhere on non-UEA websites, which is a valid reason for FOI refusals.
I can report that the information requested is not available on non-UEA websites as detailed below.
Your most recent letter (Further _information_ letter_final_ 070418_rev01. doc), however, says that you are unable to identify the locations of the requested information. Thus, the original reason for refusing to provide station data for HadCRUT3 was invalid.
Therefore, since the information requested is not available on non-UEA websites, I wish to re-instate my original request, that the information itself be made available on your website or in some other form. I understand that a small amount of this data (about 2%, according to your letter) is not available due to privacy requests from the countries involved. In that case, a listing of which stations this applies to will suffice.
The HadCRUT3 dataset is one of the fundamental datasets in the current climate discussion. As such, it is vitally important that it can be peer reviewed and examined to verify its accuracy. The only way this can be done is for the data to be made available to other researchers in the field.
Once again, thank you for your assistance in all of this. It is truly not a difficult request, and is fully in line with both standard scientific practice and your “CODE OF PRACTICE FOR RESPONDING TO REQUESTS FOR INFORMATION UNDER THE FREEDOM OF INFORMATION ACT 2000″. I am sure that we can bring this to a satisfactory resolution without involving appeals or unfavorable publicity.
My best regards to you,
w.
Here is the response from 27 April:
Dear Mr. Eschenbach FREEDOM OF INFORMATION ACT 2000 – INFORMATION REQUEST (FOI_07-04)
Further to your email of 14 April 2007 in which you re-stated your request to see
“a list of stations used by Jones et al. to prepare the HadCRUT3 dataset” I am asking for: 1) A list of the actual sites used by Dr. Jones in the preparation of the HadCRUT3 dataset, and 2) A clear indication of where the data for each site is available. This is quite important, as there are significant differences between the versions of each site’s data at e.g. GHCN and NCAR.”
In your note you also requested “the name and WMO number of each site and the location of the source data (NCAR, GHCN, or National Met Service)”,
I have contacted Dr. Jones and can update you on our efforts to resolve this matter.
We cannot produce a simple list with this format and with the information you described in your note of 14 April. Firstly, we do not have a list consisting solely of the sites we currently use. Our list is larger, as it includes data not used due to incomplete reference periods, for example. Additionally, even if we were able to create such a list we would not be able to link the sites with sources of data. The station database has evolved over time and the Climate Research Unit was not able to keep multiple versions of it as stations were added, amended and deleted. This was a consequence of a lack of data storage in the 1980s and early 1990s compared to what we have at our disposal currently. It is also likely that quite a few stations consist of a mixture of sources.
I have also been informed that, as the GHCN and NCAR are merely databases, the ultimate source of all data is the respective NMS in the country where the station is located. Even GHCN and NCAR can’t say with precision where they got their data from as the data comes not only from each NMS, but also comes from scientists in each reporting country.
In short, we simply don’t have what you are requesting. The only true source would be the NMS for each reporting country. We can, however, send a list of all stations used, but without sources. This would include locations, names and lengths of record, although the latter are no guide as to the completeness of the series.
This is, in effect, our final attempt to resolve this matter informally. If this response is not to your satisfaction, I will initiate the second stage of our internal complaint process and will advise you of progress and outcome as appropriate. For your information, the complaint process is within our Code of Practice and can be found at: http://www1. uea.ac.uk/ polopoly_ fs/1.2750! uea_manual_ draft_04b. pdf
Yours sincerely David Palmer Information Policy Officer University of East Anglia
I loved the story line in this one “we do not have a list consisting solely of the sites we currently use”. Say what? How do they produce updates that change the temperature all the way back to 1870 if they don’t have the data or a list of the sites? But I digress …
So I advised him that I was appealing. His letter was passed to a Ms. Kitty Inglis, who replied
May 21, 2007, Decision of Information Commissoners’ Office
FREEDOM OF INFORMATION ACT 2000 – INFORMATION REQUEST (FOI_07-04)
Dear Mr Eschenbach
Following David Palmer’s letter of 27th April 2007 to you regarding your dissatisfaction with our response to your FOI request of 25th January 2007, I have undertaken a thorough review of the contents of our file and have spoken with both Mr. Palmer and Professor Jones.
As a result of this investigation, I am satisfied that we have done all we can to fulfil [sic] your request and to provide you with the information you require where it is possible for us to do so.
I confirm that we are able to make available on the Climatic Research Unit website a list of stations, including name, latitude, longitude, elevation and WMO number (where available).
We are unable to provide a simple list of sources for these stations as we do not hold this information. Nor do we hold the raw (i.e. unadjusted) station data, as you describe it, at UEA. As stated in prior letters to you, raw station data are available on the NCAR and GHCN websites and gridded data are available on the Climatic Research Unit website. If these data are insufficient for your requirements, you will need to contact the NMS for the country in which the station is located to obtain the information you require.
I hope you are able to accept this response. We have contacted the Information Commissioner’s Office in relation to this matter and their advice is that if you are still dissatisfied with this response, you can, at this time, exercise your right of appeal to the
Information Commissioner by contacting them at:
Information Commissioner’ s Office
Wycliffe House
At that point, I let it go. I had a small victory, we got a list of the stations. Of course, it took me a couple more letters to actually get them to post the list. But I got nothing else of what I had requested, and the list was full of all kinds of errors.
Meanwhile, behind the scenes at CRU, I now find out that they were circling the wagons … what follows are their internal discussions about a series of FOI requests from myself, Steve McIntyre, Doug Keenan and others to CRU for various data. Phil Jones to Tom Keenan and Wei-Chyung Wang, 6/19/2007:
Wei-Chyung and Tom,
…
1. Think I’ve managed to persuade UEA to ignore all further FOIA requests if the people have anything to do with Climate Audit.
2. Had an email from David Jones of BMRC, Melbourne. [EMAIL NOT FOUND IN CRU EMAILS – Willis] He said they are ignoring anybody who has dealings with CA, as there are threads on it about Australian sites.
3. CA is in dispute with IPCC (Susan Solomon and Martin Manning) about the availability of the responses to reviewer’s at the various stages of the AR4 drafts. They are most interested here re Ch 6 on paleo.
Cheers
Phil
Well, that explains a few things … they’ve managed to “persuade UEA to ignore all further FOIA requests if the people have anything to do with Climate Audit.” I hadn’t noticed that exemption in the FOI documentation I’d seen. Call me crazy, but I don’t think that’s in FOI Exemptions, I doubt if it’s legal, and it definitely isn’t ethical. I note that they are circling the wagons in Australia as well … this is followed by:
Phil Jones to Thomas Peterson of NOAA, 6/20/2007 AM (1182342470) :
Tom P.
Just for interest. Don’t pass on.
Might be a precedent for your paper to J. Climate when it comes out. There are a few interesting comments on the CA web site. One says it is up to me to prove the paper from 1990 was correct, not for Keenan to prove we’re wrong. Interesting logic.
Cheers
Phil
Wei-Chyung, Tom,
I won’t be replying to either of the emails below [FROM STEVE MCINTYRE AND DOUG KEENAN], nor to any
of the accusations on the Climate Audit website. I’ve sent them on to someone here at UEA to see if we
should be discussing anything with our legal staff. The second letter seems an attempt to be nice to me,
and somehow split up the original author team. I do now wish I’d never sent them the data after their FOIA
request!
Cheers
Phil
He obviously views sending data in response to an FOIA request as optional.
Thomas Peterson to Jones, same email:
Fascinating. Thanks for keeping me in the loop, Phil. I won’t pass it on but I will keep it in the back of my mind when/if Russ asks about appropriate responses to CA requests. Russ’ view is that you can never satisfy them so why bother to try?
Again, responding to an FOIA request is viewed as optional.
Phil Jones :
…
PS to Gavin – been following (sporadically) the CA stuff about the GISS data and release of the code etc by Jim. May take some of the pressure off you soon, by releasing a list of the stations we use – just a list, no code and no data. Have agreed to under the FOIA here in the UK.
Oh Happy days!
So I see … that’s why I only got the station list and not the data, just to ” take some of the pressure off “. Thanks, Phil.
Jones to Bradley and Amman, 5/9/08 (1210341221):
Mike, Ray, Caspar,
A couple of things – don’t pass on either.
…
2. You can delete this attachment if you want. Keep this quiet also, but this is the person [DAVID HOLLAND – Willis] who is putting in FOI requests for all emails Keith and Tim have written and received re Ch 6 of AR4. We think we’ve found a way around this.
Finding ways around FOI requests seems to be a popular sport at CRU. This is in reference to Steve trying to get the review comments to Chapter 6 of the UN IPCC Fourth Assessment Report.
Next, here’s the brilliant way that they had found around the FOIA, a bombshell of an idea, Jones to Michael Mann, 29 May 2008 (1212063122):
Mike,
Can you delete any emails you may have had with Keith re AR4? Keith will do likewise. He’s not in at the moment – minor family crisis.
Can you also email Gene and get him to do the same? I don’t have his new email address.
We will be getting Caspar to do likewise.
…
Cheers
Phil
Again, call me crazy, but deleting evidence in the face of an FOI request must be illegal. Gene is Eugene Wahl. Of course, what these guys don’t realize is that there are multiple copies of most emails floating around. In some ways, I hope they deleted them, so that it can be proven. The story continues …
Tim Osborne to Jones, Briffa, and Mann, 23 Jun 2008 (1214229243) :
Subject: Re: CA
Hi Phil, Keith and “Confidential Agent Ammann”,
At 17:00 21/06/2008, P.Jones wrote:
This is a confidential email
So is this.
Have a look at Climate Audit. Holland has put all the responses and letters up. There are three threads – two beginning with Fortress and a third later one. Worth saving the comments on a Jim Edwards – can you do this Tim?
I’ve saved all three threads as they now stand. No time to read all the comments, but I did note in “Fortress Met Office” that someone has provided a link to a website that helps you to submit FOI requests to UK public institutions, and subsequently someone has made a further FOI request to Met Office and someone else made one to DEFRA. If it turns into an organised campaign designed more to inconvenience us than to obtain useful information, then we may be able to decline all related requests without spending ages on considering
them. Worth looking out for evidence of such an organised campaign.
Tim
Another thing to hide behind, a false claim of an “organised campaign”. I loved the “Confidential Agent Amman” …
Phil Jones:
To: Gavin Schmidt
Subject: Re: Revised version the Wengen paper
Date: Wed Aug 20 09:32:52 2008
Cc: Michael Mann
Gavin,
…
Keith/Tim still getting FOI requests as well as MOHC and Reading. All our FOI officers have been in discussions and are now using the same exceptions not to respond – advice they got from the Information Commissioner. As an aside and just between us, it seems that Brian Hoskins has withdrawn himself from the WG1 Lead nominations. It seems he doesn’t want to have to deal with this hassle.
The FOI line we’re all using is this. IPCC is exempt from any countries FOI – the skeptics have been told this. Even though we (MOHC, CRU/UEA) possibly hold relevant info the IPCC is not part our remit (mission statement, aims etc) therefore we don’t have an obligation to pass it on.
Cheers
Phil
So now the Information Commissioner is in on the deal, s/he’s advising them to use the same exceptions not to respond. No need to think about it, all of the wheels have been greased.
Next, Ben Santer chimes in:
Ben Santer to Thomas Karl, Karen Owen, Sharon Leduc , “Thorne, Peter”, Leopold Haimberger , Karl Taylor, Tom Wigley, John Lanzante, Susan Solomon, Melissa Free, peter gleckler , “‘Philip D. Jones’”, Thomas R Karl, Steve Klein, carl mears, Doug Nychka, Gavin Schmidt, Steven Sherwood, Frank Wentz, “David C. Bader”, Professor Glenn McGregor, “Bamzai, Anjuli”
Dear Tom,
…
My personal opinion is that both FOI requests (1) and (2) are intrusive and unreasonable. Steven McIntyre provides absolutely no scientific justification or explanation for such requests. I believe that McIntyre is pursuing a calculated strategy to divert my attention and focus away from research. As the recent experiences of Mike Mann and Phil Jones have shown, this request is the thin edge of wedge. It will be followed by further requests for computer programs, additional material and explanations, etc., etc.
Quite frankly, Tom, having spent nearly 10 months of my life addressing the serious scientific flaws in the Douglass et al. IJoC paper, I am unwilling to waste more of my time fulfilling the intrusive and frivolous requests of Steven McIntyre. The supreme irony is that Mr. McIntyre has focused his attention on our IJoC paper rather than the Douglass et al. IJoC paper which we criticized. As you know, Douglass et al. relied on a seriously flawed statistical test, and reached incorrect conclusions on the basis of that flawed test.
I believe that our community should no longer tolerate the behavior of Mr. McIntyre and his cronies. McIntyre has no interest in improving our scientific understanding of the nature and causes of climate change. He has no interest in rational scientific discourse. He deals in the currency of threats and intimidation. We should be able to conduct our scientific research without constant fear of an “audit” by Steven McIntyre; without having to weigh every word we write in every email we send to our scientific colleagues.
In my opinion, Steven McIntyre is the self-appointed Joe McCarthy of climate science. I am unwilling to submit to this McCarthy-style investigation of my scientific research. As you know, I have refused to send McIntyre the “derived” model data he requests, since all of the primary model data necessary to replicate our results are freely available to him. I will continue to refuse such data requests in the future. Nor will I provide McIntyre with computer programs, email correspondence, etc. I feel very strongly about these issues. We should not be coerced by the scientific equivalent of a playground bully.
I will be consulting LLNL’s Legal Affairs Office in order to determine how the DOE and LLNL should respond to any FOI requests that we receive from McIntyre. I assume that such requests will be forthcoming.
I am copying this email to all co-authors of our 2008 IJoC paper, to my immediate superior at PCMDI (Dave Bader), to Anjuli Bamzai at DOE headquarters, and to Professor Glenn McGregor (the editor who was in
charge of our paper at IJoC).
I’d be very happy to discuss these issues with you tomorrow. I’m sorry that the tone of this letter is so formal, Tom. Unfortunately, after today’s events, I must assume that any email I write to you may be subject to FOI requests, and could ultimately appear on McIntyre’s “ClimateAudit” website.
With best personal wishes,
Ben
Well, he got the last paragraph right, at least. He also thinks that an FOIA request must serve some “scientific justification”, with the justification determined by … well … by the person receiving the request, of course. Another previously unknown part of the FOI Exemptions comes to light.
Ben Santer to Tom Wigly, 12 Dec 07 (1228330629):
At 01:17 03/12/2008, Ben Santer wrote:
Dear Tom,
…
One of the problems is that I’m caught in a real Catch-22 situation. At present, I’m damned and publicly vilified because I refused to provide McIntyre with the data he requested. But had I acceded to McIntyre’s initial request for climate model data, I’m convinced (based on the past experiences of Mike Mann, Phil, and Gavin) that I would have spent years of my scientific career dealing with demands for further explanations, additional data, Fortran code, etc. (Phil has been complying with FOIA requests from McIntyre and his cronies for over two years). And if I ever denied a single request for further information, McIntyre would have rubbed his hands gleefully and written: “You see – he’s guilty as charged!” on his website.
You and I have spent over a decade of our scientific careers on the MSU issue, Tom. During much of that time, we’ve had to do science in “reactive mode”, responding to the latest outrageous claims and inept science by John Christy, David Douglass, or S. Fred Singer. For the remainder of my scientific career, I’d like to dictate my own research agenda. I don’t want that agenda driven by the constant need to respond to Christy, Douglass, and Singer. And I certainly don’t want to spend years of my life interacting with the likes of Steven McIntyre.
I hope LLNL management will provide me with their full support. If they do not, I’m fully prepared to seek employment elsewhere.
With best regards,
Ben
Dr. Santer, here’s a novel idea. Put enough information out when you publish the work so that your work can be replicated. Put on the web whatever is necessary in the way of code, data, and methods to allow your work to be checked by someone else. If you do that, not only will you not be bothered, but you will be following the scientific method. None of us at ClimateAudit are doing this to harass anyone, as you claim. We’re doing this because we cannot replicate your work, and thus your work is purely anecdotal rather than scientific.
Phil responds (same email):
Cc: mann , Gavin Schmidt, Karl Taylor, peter gleckler
Ben,
When the FOI requests began here, the FOI person said we had to abide by the requests. It took a couple of half hour sessions – one at a screen, to convince them otherwise showing them what CA was all about. Once they became aware of the types of people we were dealing with, everyone at UEA (in the registry and in the Environmental Sciences school – the head of school and a few others) became very supportive. I’ve got to know the FOI person quite well and the Chief Librarian – who deals with appeals. The VC is also aware of what is going on – at least for one of the requests, but probably doesn’t know the number we’re dealing with. We are in double figures.
One issue is that these requests aren’t that widely known within the School. So I don’t know who else at UEA may be getting them. CRU is moving up the ladder of requests at UEA though – we’re way behind computing though. We’re away [aware?]of requests going to others in the UK – MOHC, Reading, DEFRA and Imperial College.
So spelling out all the detail to the LLNL management should be the first thing you do. I hope that Dave is being supportive at PCMDI. The inadvertent email I sent last month has led to a Data Protection Act request sent by a certain Canadian, saying that the email maligned his scientific credibility with his peers!
If he pays 10 pounds (which he hasn’t yet) I am supposed to go through my emails and he can get anything I’ve written about him. About 2 months ago I deleted loads of emails, so have very little – if anything at all. This legislation is different from the FOI – it is supposed to be used to find put why you might have a poor credit rating !
In response to FOI and EIR requests, we’ve put up some data – mainly paleo data. Each request generally leads to more – to explain what we’ve put up. Every time, so far, that hasn’t led to anything being added – instead just statements saying read what is in the papers and what is on the web site! Tim Osborn sent one such response (via the FOI person) earlier this week. We’ve never sent programs, any codes and manuals.
In the UK, the Research Assessment Exercise results will be out in 2 weeks time. These are expensive to produce and take too much time, so from next year we’ll be moving onto a metric based system. The metrics will be # and amounts of grants, papers and citations etc. I did flippantly suggest that the # of FOI requests you get should be another.
When you look at CA, they only look papers from a handful of people. They will start on another coming out in The Holocene early next year. Gavin and Mike are on this with loads of others. I’ve told both exactly what will appear on CA once they get access to it!
Cheers
Phil
Well, that explains why David Palmer and Ms. Kitty Inglis, the Chief Librarian, were so unsupportive. Took a couple of half hour sessions, but at the end of that, rather than being a representative of the FOI process, they were functioning as the personal representatives of Phil Jones. We have a new reason I hadn’t noticed in the FOI law for refusing a request, because the requester posts at CA.
And since they have the FOI person, and the FOI Appeals person, and the Information Commissioner in there pockets, and they have the standard terms of refusal figured out … just how difficult can it be to deny an FOI Request?
Jones to Ben Santer again, 10 Dec 2008:
Ben,
Haven’t got a reply from the FOI person here at UEA. So I’m not entirely confident the numbers are correct. One way of checking would be to look on CA, but I’m not doing that. I did get an email from the FOI person here early yesterday to tell me I shouldn’t be deleting emails – unless this was ‘normal’ deleting to keep emails manageable! McIntyre hasn’t paid his £10, so nothing looks likely to happen re his Data Protection Act email.
Anyway requests have been of three types – observational data, paleo data and who made IPCC changes and why. Keith has got all the latter – and there have been at least 4. We made Susan aware of these – all came from David Holland. According to the FOI Commissioner’ s Office, IPCC is an international organization, so is above any national FOI. Even if UEA holds anything about IPCC, we are not obliged to pass it on, unless it has anything to do with our core business – and it doesn’t! I’m sounding like Sir Humphrey here! McIntyre often gets others to do the requesting, but requests and responses all get posted up on CA regardless of who sends them.
On observational data, there have been at least 5 including a couple from McIntyre. Others here came from Eschenbach and also Douglas Keenan. The latter relate to Wei-Chyung Wang, and despite his being exonerated by SUNY, Keenan has not changed his web site since being told the result by SUNY!
The paleo data requests have all been to Keith, and here Tim and Keith reply. The recent couple have come from McIntyre but there have been at least two others from Holland. So since Feb 2007, CRU is in double figures. We never get any thanks for putting things up – only abuse and threats. The latest lot is up in the last 3-4 threads on CA.
I got this email over the weekend – see end of this email. This relates to what Tim sent back late last week. There was another one as well – a chatty one saying why didn’t I respond to keep these people on CA quiet. I’ve ignored both. Finally, I know that DEFRA receive Parliamentary Questions from MPs to answer. One of these 2 months ago was from a Tory MP asking how much money DEFRA has given to CRU over the last 5 years. DEFRA replied that they don’t give money – they award grants based on open competition. DEFRA’s system also told them there were no awards to CRU, as when we do get something it is down as UEA!
I’ve occasionally checked DEFRA responses to FOI requests – all from Holland.
Cheers
Phil
Since he and Mann and the others have already deleted their emails, looks like David Palmer (the “FOI person”) was a bit too late with his excellent advice … however, I did get a “Mentioned In Dispatches” from Phil, at least …
I also like the sly way he tells Ben how to illegally delete emails, just do it as part of ‘normal’ deleting to keep emails manageable! Yeah, right, that’s the ticket.
Phil Jones to Raymond Pierrehumbert, 16 Jan 09 (1200493432):
Cc: Michael Mann , Gavin Schmidt
Ray,
…
I have had a couple of exchanges with Courtillot. This is the last of them from March 26, 2007. I sent him a number of papers to read. He seems incapable of grasping the concept of spatial degrees of freedom, and how this number can change according to timescale. I also told him where he can get station data at NCDC and GISS (as I took a decision ages ago not to release our station data, mainly because of McIntyre). I told him all this as well when we met at a meeting of the French Academy in early March.
…
Cheers, Phil
This is a very clear statement of what he has done. He has refused to release the data, not because there is any logical reason to do so, but “because of McIntyre”. This is shameful, and the fact that the FOI people, Dave Peters and Kitty Inglis and the Information Commissioner, went along with this is dereliction of duty.
Finally, Steven Schneider chimes in to write Ben from Stanford University, 6 Jan 09 (1231257056):
Cc: “David C. Bader”, Bill Goldstein, Pat Berge, Cherry Murray, George Miller, Anjuli Bamzai, Tomas Diaz De La Rubia, Doug Rotman, Peter Thorne, Leopold Haimberger, Karl Taylor, Tom Wigley John Lanzante, Susan Solomon, Melissa Free, peter gleckler, “Philip D. Jones”, Thomas R Karl, Steve Klein, carl mears, Doug Nychka, Gavin Schmidt, Steven Sherwood, Frank Wentz
“Thanks” Ben for this, hi all and happy new year. I had a similar experience– but not FOIA since we at Climatic Change are a private institution- -with Stephen McIntyre demanding that I have the Mann et al cohort publish all their computer codes for papers published in Climatic Change I put the question to the editorial board who debated it for weeks. The vast majority opinion was that scientists should give enough information on their data sources and methods so others who are scientifically capable can do their own brand of replication work, but that this does not extend to personal computer codes with all their undocumented sub routines etc. It would be odious requirement to have scientists document every line of code so outsiders could then just apply them instantly. Not only is this an intellectual property issue, but it would dramatically reduce our productivity since we are not in the business of producing software products for general consumption and have no resources to do so. The NSF, which funded the studies I published, concurred–so that ended that issue with Climatic Change at the time a few years ago.
This continuing pattern of harassment, as Ben rightly puts it in my opinion, in the name of due diligence is in my view an attempt to create a fishing expedition to find minor glitches or unexplained bits of code–which exist in nearly all our kinds of complex work–and then assert that the entire result is thus suspect. Our best way to deal with this issue of replication is to have multiple independent author teams, with their own codes and data sets, publishing independent work on the same topics–like has been done on the “hockey stick”. That is how credible scientific replication should proceed.
Let the lawyers figure this out, but be sure that, like Ben is doing now, you disclose the maximum reasonable amount of information so competent scientists can do replication work, but short of publishing undocumented personalized codes etc. The end of the email Ben attached shows their intent–to discredit papers so they have no “evidentiary value in public policy”–what you resort to when you can’t win the intellectual battle scientifically at IPCC or NAS.
Good luck with this, and expect more of it as we get closer to international climate policy actions, We are witnessing the “contrarian battle of the bulge” now, and expect that all weapons will be used.
Cheers, Steve
PS Please do not copy or forward this email.
Now, why would Dr. Schneider not want his email copied or forwarded … perhaps because he is saying don’t follow the spirit of the Freedom of Information Act, don’t release the code that shows the math and reveals how you got your results? He foolishly thinks that studies can be “replicated” by using different data and different codes … but that says absolutely nothing about the original study and whether it contains any mistakes. The only way to determine whether a study like a historical temperature reconstruction contains errors is to examine the scientists actual work You can’t just pick another different bunch of proxies, analyze them, and say “I’ve found mathematical errors in your reconstruction”. You can only find those errors if you examine the actual math the researcher used, and to do that you need access to “their” codes.
I put “their” codes, in quotes because, under the policies of the University of East Anglia (and many other Universities), the codes do not belong to Phil Jones. They were developed as a part of his employment, and as such they belong to the University, and not to Phil.
The researchers complain in various places that they do not want to reveal their “primary data” because it is available on the web. While this is often true, as I saw in my FOIA requests to CRU, it is not sufficient Just saying “I got the information from Website X” as CRU did is often totally inadequate to locate the data in question. Santer makes this charge, that anyone could go the CMIP website and get the data themselves … but unless he says exactly which data from which run of which model, the website address is meaningless.
The main impression that I get from the emails is that the various scientists think that I and other requesters are simply doing this to harass them. Nothing could be further from the truth. I respect actual scientists, I’m short of time myself so I understand time pressures, so I have no desire to put any scientist to any extra effort beyond providing what science requires – a full accounting of the data, the methods, and in some cases the computer code used to do the research. Anything more is harassment … but anything less is scientific obstruction. And if they would provide those things when they publish their results, they’d never hear from me. And if Nature Magazine and Science Magazine and the National Science Foundation and all of the journals and funders would just enforce their own existing rules on archiving and transparency, the problem would be solved. But noooo, for the select friends of Phil these bothersome transparency regulations are ignored and overlooked.
As I said, the issue is not Trenberth or the Nature “trick” or scientists talking smack about each other. It is the illegal evasion of legitimate scientific requests for data needed to replicate a scientific study. Without replication, science cannot move forwards. And when you only give data to friends of yours, and not to people who actually might take a critical look at it, care to guess what you might end up with?
A “consensus” …
My best to everyone,
w.
No comments:
Post a Comment