Posts Tagged ‘IoP’

Dear Institute of Physics… (Part II)

June 18, 2010

Following my statement in March this year that I would leave the IoP if they didn’t withdraw their evidence statement from the House of Commons Science and Technology Committee investigation into the CRU, here’s the result…

Dear Institute of Physics

Thank you for my recent reminder to renew my membership.  Unfortunately I no longer wish to be associated with the IoP.

My concern goes back to March this year when I contacted the Institute regarding their evidence submission to a House of Commons Science and Technology Committee investigation into the work of the Climatic Research Unit at the University of East Anglia.

The IoP evidence submission was particularly one sided in its analysis and engaged in wild speculation.  It appeared to have an agenda to undermine the work of the CRU without supplying any evidence to substantiate its claims.  This is clearly irresponsible and inappropriate behaviour from a professional society.  [My quick analysis of it is here.]

The response from the IoP to my concern about this evidence submission was also well below what I expected.  The only specific response [which I can’t find on the IoP website now but is reproduced at the bottom of my letter from March 2010 here] was an anonymous note from a member of the board that issued the submission saying that we should be “relaxed” about the process by which the statement was written (despite the fact that “working scientists just don’t have time to [produce the first draft of the statement]”.  If any other responses were issued I suspect I missed them as I had no further communication from the Institute.

The anonymous statement, and refusal to disclose who had actually written the submission, was particularly ironic as the evidence statement was calling for greater openness from the scientists it was criticising!  Furthermore, IoP journals do not even require the level of data “openness” that the submission criticised CRU for not following.

In the end, the HoC S&TC report was very supportive of the CRU so at least your evidence submission had no impact in that respect.  However, the IoP still set a precedent that it is willing to openly criticise the work of scientists without providing any evidence to warrant such an attack.

I am sad to leave the IoP as I have been inspired by some of your work and recognise what you do to promote physics.  However, following this episode, I can no longer support the Institute.

Your faithfully,

Andrew Russell

[In the IoP’s defence, I did have some private conversations with a few people from the Institute and they were aware that there was a problem here to be dealt with.  I have no idea if anything was done as they were obviously keen to deal with it in private.  However, I still feel that the IoP should have more open about their efforts to get things in order and that a mistake had been made.]

The beginning of the end of climategate?

April 3, 2010

The UK House of Commons Science and Technology Committee (S&TC) published their report on “The disclosure of climate data from the Climatic Research Unit at the University of East Anglia” last week. This follows their oral evidence session and requests for written evidence concerning this matter.

What did they find?

The key conclusions from the report were:

“The focus on Professor Jones and CRU has been largely misplaced.”

“We have suggested that the community consider becoming more transparent by publishing raw data and detailed methodologies.”

and

“Within our limited inquiry and the evidence we took, the scientific reputation of Professor Jones and CRU remains intact.” *

I am very happy with the first and third points here. The excellent work of Phil Jones has been instrumental in bringing attention to climate change and its great that this inquiry has acknowledged that.

Data policy

As for the second point above, I have no problem with this as long as any rules imposed are carefully considered and not “knee jerk” measures designed just to show that something has been done.

Indeed, James Annan has pointed out on his blog that this criticism of “standard practice” regarding data policy must surely be aimed at the Research Councils and not individual scientists or universities. Chris Rowan neatly tweeted that this stance boils down to the “Government condemning government-funded scientists for following government IP policy”. Hmm.

It is also worth considering this point from the S&TC report:

“Even if the data that CRU used were not publicly available—which they mostly are—or the methods not published—which they have been—its published results would still be credible: the results from CRU agree with those drawn from other international data sets; in other words, the analyses have been repeated and the conclusions have been verified.”

This seems to be saying that, whilst desireable, even ideal data sharing rules are not fundamental in verifying CRU’s work.

(To clarify my own position, I’ve never said that I don’t think that data should be shared. However, I did think that it was unfair to criticise CRU for not following ideal data sharing standards that never existed. It seems that the S&TC largely agree with this. Rather, the S&TC criticise UEA for not supporting and advising researchers appropriately.)

Evidence

The S&TC came to these conclusions despite receiving a rather skewed view of climate research from their evidence.

For example, there was the flawed IoP evidence submission and there were a high number of submissions from prominent “skeptics” (e.g. McKitrick, McIntyre, Global Warming Policy Foundation).

This input, though, does not seem to have shaped the findings of the report in any significant way, other than some of these people being the source of many of the Freedom of Information requests that CRU received and dealt with poorly.

Looking at the minutes of the report, it seems that Graham Stringer MP attempted to amend some elements of the document to bring it more in line with the “skeptical” evidence. He was unsuccessful. However, as Deep Climate points out, even his proposed amendments would not have changed the conclusions much.

Why did this situation arise in this inquiry? Some of the comment on a recent post from Stoat suggests that those working in climate science should be more active in contributing to these inquiries, which sounds like a good idea to me.

Where next?

Of course, this is only the first of three inquiries to investigate this episode with the other two being the Muir Russell headed Independent Climate Change Email Review and the Scientific Appraisal Panel, which includes some very big names.

Hopefully, though, this report heralds the beginning of the end of “climategate”.

* Graham Stringer MP voted against the inclusion of this point.

Add to FacebookAdd to DiggAdd to Del.icio.usAdd to StumbleuponAdd to RedditAdd to BlinklistAdd to TwitterAdd to TechnoratiAdd to Yahoo BuzzAdd to Newsvine

Institute of Physics S&TC evidence submission – what’s actually wrong with it?

March 6, 2010

There’s clearly some interest in the IoP evidence submission and my original letter to the IoP didn’t really go into my objections in great detail. I thought I should go through the evidence submission in one place instead of explaining my views in response to blog comments.

Overall, I’m not objecting to the statement because I disagree with it (although I do). I object to it because, for an evidence submission, it contains no evidence and it is judgemental. The IoP should be embarrassed to have its name associated with it and, in my opinion, should retract this evidence statement. Here are my thoughts on why the first 8 points from the submission are inadequate:

“1. The Institute is concerned that, unless the disclosed e-mails are proved to be forgeries or adaptations, worrying implications arise for the integrity of scientific research in this field and for the credibility of the scientific method as practised in this context.”

This point contradicts the IoP’s assertion in their recent press releases that they accept the current understanding of climate science as presented by the IPCC. More worryingly, despite the fact that CRU’s research findings have not been undermined by the email leak, this point implies that the integrity and credibility of the whole of climate science is now in doubt. There is no justification for this extrapolation even if CRU’s research were compromised.

“2. The CRU e-mails as published on the internet provide prima facie evidence of determined and co-ordinated refusals to comply with honourable scientific traditions and freedom of information law. The principle that scientists should be willing to expose their ideas and results to independent testing and replication by others, which requires the open exchange of data, procedures and materials, is vital. The lack of compliance has been confirmed by the findings of the Information Commissioner. This extends well beyond the CRU itself – most of the e-mails were exchanged with researchers in a number of other international institutions who are also involved in the formulation of the IPCC’s conclusions on climate change.”

This is a rather one sided analysis of the unique issues relating to the data that CRU use – there is what I consider to be a more balanced and better informed overview here.

Further, UEA present a different interpretation of the Freedom of Information issues informed by their correspondence with the Information Commissioner’s Office.

“3. It is important to recognise that there are two completely different categories of data set that are involved in the CRU e-mail exchanges:

• those compiled from direct instrumental measurements of land and ocean surface temperatures such as the CRU, GISS and NOAA data sets; and

• historic temperature reconstructions from measurements of ‘proxies’, for example, tree-rings.”

There’s not much to take issue with here other than the very final part – “for example, tree rings” – which doesn’t really give an indication of the full range of proxy measures that have been used in the reconstructions.

“4. The second category relating to proxy reconstructions are the basis for the conclusion that 20th century warming is unprecedented. Published reconstructions may represent only a part of the raw data available and may be sensitive to the choices made and the statistical techniques used. Different choices, omissions or statistical processes may lead to different conclusions. This possibility was evidently the reason behind some of the (rejected) requests for further information.”

“This possibility was evidently the reason…” I read this as wild speculation. Is there any evidence to back this theory up? If not, it has no place in an evidence submission. More generally, though, why have the studies that have investigated the details of the proxy compilation in the IPCC report (The “Wegman” report, 2006 and The National Research Council Report, 2006) not been mentioned and critiqued?

“5. The e-mails reveal doubts as to the reliability of some of the reconstructions and raise questions as to the way in which they have been represented; for example, the apparent suppression, in graphics widely used by the IPCC, of proxy results for recent decades that do not agree with contemporary instrumental temperature measurements.”

For me, this is the low point of the evidence submission. Surely the reason for asking organisations like the IoP to provide evidence is that they have the knowledge to provide context to the issues at hand. This point has not done that – it is merely an ill informed judgement on a quote from an old email (the “…hide the decline…” one).

If this point were to be taken seriously, they would need to have referred to the figures in specific papers or documents where they believe that the proxy records were suppressed.

I suspect that they have not done this because there is no evidence to back up this point. For example, Briffa et al. 1999 (“Reduced sensitivity of recent tree-growth to temperature at high northern latitudes” in Nature) specifically discusses the issues of dendroclimatology divergence. Chapter 6.6 of the IPCC AR4 WG1 report also looks at the details of the proxy reconstructions of the climate of the last 2000 years, including divergence. There are other examples. I cannot see how the IoP can interpret this as suppression.

”6. There is also reason for concern at the intolerance to challenge displayed in the e-mails. This impedes the process of scientific ‘self correction’, which is vital to the integrity of the scientific process as a whole, and not just to the research itself. In that context, those CRU e-mails relating to the peer-review process suggest a need for a review of its adequacy and objectivity as practised in this field and its potential vulnerability to bias or manipulation.”

This point again seems one sided and gives no recognition to the efforts of the scientists in question to engage with their critics before they were subjected to unfounded attacks on their work and integrity.

In relation to the second issue in this point, peer review, the IoP should surely have looked beyond an account of what people were saying in an incomplete archive of private emails. If we look at what actually happened in the public arena then, as an example, certain papers that were discussed in those emails were not excuded from the IPCC report as was suggested.

“7. Fundamentally, we consider it should be inappropriate for the verification of the integrity of the scientific process to depend on appeals to Freedom of Information legislation. Nevertheless, the right to such appeals has been shown to be necessary. The e-mails illustrate the possibility of networks of like-minded researchers effectively excluding newcomers. Requiring data to be electronically accessible to all, at the time of publication, would remove this possibility.”

Again, there are particular issues relating to a small percentage of the data used by the CRU. Newcomers to the field were not excluded from contributing (if anything, the work of CRU in collecting all this data and producing usable girded products promotes the inclusion of newcomers) but the issues relating to the data mean that, at presnt, to replicate the work they would have to request the data from third parties themselves.

This point again seems rather one sided. For example, was the aim of the FoI requests really to replicate the CRU temperature product? This blog post and some of the comments indicate that it probably wasn’t. But I agree that making the data available is still important as a matter of principle.

“8. As a step towards restoring confidence in the scientific process and to provide greater transparency in future, the editorial boards of scientific journals should work towards setting down requirements for open electronic data archiving by authors, to coincide with publication. Expert input (from journal boards) would be needed to determine the category of data that would be archived. Much ‘raw’ data requires calibration and processing through interpretive codes at various levels.”

Similar to the final sentence of point 7, this is a nice recommendation but it does not really address the issues at hand. Incidentally, this proposed “standard” is not even employed by the IoP’s own journals – for example, ERL allows data to be uploaded with papers but it is not required. To imply that CRU have acted badly by not complying with this non-existent standard makes no sense.

Reference:

Anonymous members of the Institute of Physics Science Board (2010). The disclosure of climate data from the Climatic Research Unit at the University of East Anglia House of Commons Science and Technology Committee Memorandum

Dear Institute of Physics…

March 3, 2010

After publishing my post on the CRU and S&TC yesterday, I looked into the IoP’s evidence submission in a bit more detail and was quite surprised. Here is an open letter to the IoP about that evidence:

Dear Institute of Physics

As a member of the IoP I am very concerned about the recent memorandum submitted by the IoP to the House of Commons Science and Technology Committee.

In my view, it is unfair to criticise the CRU on the basis that they did not comply with data sharing standards that, at present, don’t exist. There is clearly a need for rules regarding openness in relation to data and methods but it is foolish to retrospectively admonish people for not following them! Do the journals currently published by the IoP employ the data policies suggested in your statement?

There also seems to be some misunderstanding in the statement of the particular issues relating to the data that the CRU use in their research and how some of the issues discussed in the private and incomplete email records were resolved in the peer reviewed literature and other open arenas. In particular, point 5 of your evidence is incorrect and irresponsible given that it casts unwarranted doubt over the findings of the IPCC.

Furthermore, your statement gives no recognition to the efforts of the scientists in question to engage with their critics before they were subjected to unfounded attacks on their work and integrity and orchestrated FoI request campaigns.

Finally, I am confused as to why the Energy group was tasked with preparing the statement and not the Environmental Physics group, who would have been more aware of the particular issues in this case.

I realise that a small clarification has been issued but if the IoP continues to stand by this statement then I will have no other option but to reconsider my membership of your organisation.

Yours faithfully,

Andy Russell MInstP

Some similar posts:

Stoat: The IoP fiasco
Some beans: A letter to the Institute of Physics

CORRECTION: The statement was written by the “Energy Sub-group of the IoP’s Science Board” not the “IoP’s Energy Group” as I previously thought. Either way, the IoP has still not been open about who wrote the statement.

UPDATE 5/3/2010: In response to my letter I was sent an email from the IoP standing by the submission and an anonymous statement from a member of the Science Board about the process of writing the evidence submission. Here is that statement:

“The IOP contribution has been widely understood and welcomed – not universally, apparently, but then there is a debate going on. Scientists are sometimes criticised for not engaging, and I hope we can look forward to hearing from professional bodies representing other branches of science.

“The Institute should feel equally relaxed about the process by which it generated what is, anyway, a statement of the obvious. The standard process for policy submissions by IOP – it makes dozens per year – was followed. Typically a call for evidence is spotted or received by IOP HQ. Usually the timescale is tight. A first draft is put together at Portland Place – working scientists just don’t have time to do this. The draft is then emailed round to all members of IOP advisory bodies that might want to contribute, which is where working scientists come in. Science Board is one of several such bodies. When we do respond – my personal strike rate is definitely under 50 per cent – we copy round our responses, so other committee members get multiple opportunities to comment if they wish to. During this phase people can and do say “Sorry, I disagree”. Remember that there is a tight deadline, and by the time it is reached, everyone who seriously wants to comment will have done so. Collective responsibility then kicks in, which is why I am not revealing my identity.”

UPDATE 6/3/2010: apparently not realising that it kind of proves my point about the sort of behaviour Phil Jones has had to deal with for years, I got an email today in response to my perfectly reasonable assessment of the IoP’s evidence submission:

For a physicist you are a bit of a cretin.

However your views on the release of scientific data for verification are positively neolithic.

What kind of blind, bigoted AGW Alarmist tosser are you?

Pull your head out of your arse and smell the coffee!

Have a nice day!