While I have written on this blog about the IPCC’s need to be more transparent in its workings, there comes a point where the constant barrage of reports and studies of previous IPCC findings becomes a serious problem. Yes, another report, this time about the issue of glaciers in the Himalayas in the previous IPCC assessment report (AR4). Despite fairly intense efforts to discredit AR4 – coming under the heading of “climategate” (people saying dumb things, and sometimes not so dumb things that were willfully misconstrued, on e-mails that were stolen from a server at a major climate research center) and Himalayagate (where the working group I recently joined relied on a bit of non-peer reviewed literature from an otherwise reputable source – called grey literature in scientific jargon – that turned out to be wrong), the reports on AR4 (yep, reports on the report) have yet to question the overall findings of the IPCC to this point. Some see this as evidence of a conspiracy, where the inquiries into the IPCC and its findings are already corrupt and unable to come to independent conclusions. Personally, I have a bit of difficulty believing in such a wide-ranging, well-coordinated conspiracy. Maybe, just maybe, the findings are as close to valid as we can verify under current knowledge. Maybe?
Anyway, my complaint here is not really with yet another inquiry into the IPCC – I’d bet my house that the inquiry will not challenge the larger findings of AR4. However, when all that ever reaches the news is a constant barrage of reports on the findings of inquiries into supposed misdeeds on the part of the IPCC, it is hard to blame the general public for doubting the validity of its findings.
All of this goes back to a much earlier post that got a lot of attention, especially considering how remarkably simple its central point was: we need to be very transparent in what we do going forward. That, and as Bob Watson, former head of the IPCC and a colleague from the Millennium Assessment and GEO-4, noted about the “Himalayagate issue”:
“To me the fundamental problem was that when the error was found it was handled in a totally and utterly atrocious manner.”
Yep. The IPCC, and indeed most major environmental assessments, really need to get comfortable with the idea that sometimes we will be wrong – that is the nature of knowledge, let alone science – and we need to start engaging the professionals when it comes to PR. I have it firsthand from Pachauri that the IPCC is fully engaged in such an effort (PR, that is), and has been for some time. Good. We recruit scientific experts for their special abilities, but somehow we resist recruiting PR people? To quote the physics Nobelist Erwin Schroedinger, “If you cannot – in the long run – tell everyone what you have been doing, your doing has been worthless”. To take it another step, if we cannot honestly and clearly communicate what we do and don’t know, and when we get things wrong, than whatever we have done right will be lost in the maelstrom.
Oh, and can we get a new tag for scandals? Himalayagate? Really? Could we find a more odd combination of geography to cram into a single word?
Good post. And truthfully, you have my sympathy. This sort of process is never easy for anyone caught up in it.
‘Himalayagate’ is more commonly referred to as ‘glaciergate’ by those who originally dug it up. The IPCC apparently quoted from an interview given by a Dr Syed Hasnain to the little-known Indian environmental magazine Down to Earth, but perhaps because that didn’t look very authoritative, they instead chose to cite a WWF report that quoted a New Scientist article containing another interview that Dr Hasnain gave in which he said some very similar things.
I don’t know about you, but I don’t regard WWF as an unbiased source; reputable or not.
But of course the main issue is not that it was quoted from grey literature – the sceptics would quite like you to quote some of their stuff, too! – but that what is quite frankly an utterly incredible statement got past all the layers of peer review and source checking. I’m not a glaciologist, and even I know better. I would even know that the corrected statement – that if glaciers continued to decline at the current rate they could be gone by 2350 – is an unreasonable extrapolation, since the rate varies constantly. Somebody must have been asleep when they typed that.
New Scientist was once very good, but nowadays prints serious stories about anti-gravity machines, and the WWF are of course trying to persuade people to donate. Nobody on our side of the fence cares much what they say. And apart from the Indians, for who the glaciers were a big deal, we don’t much care about the glacier error itself either. Its importance is only that it is clear and unambiguous evidence that the IPCC reports have not been checked as thoroughly as has been claimed, and so the big question is: what else got through?
Oh, and nobody knows yet whether the Climategate emails were stolen or leaked. The police enquiries are continuing.
I think the reason that this report has taken a somewhat different tone is that they have finally realised that it is not a matter of the barrage of reports and enquiries eroding the public’s belief in the validity of it’s findings, but that the public has seen the problems and
already doubts the findings and it is now a question of how the IPCC can recover. Pretending that nothing is wrong simply won’t work.
They now have to make a show of cleaning house, doing things differently, making the changes obvious, so they can say things are different – so that now when they come to exactly the same conclusions, they can say we all ought to believe it.
I honestly don’t know if that will work. If they’d done it a few years ago, I’m sure it would have. (The IPCC in many ways made the political contrarians’ jobs easier.) If they’d done it six months ago, immediately after Climategate, it probably would have. You could have shown how seriously you take such matters, drawn a line firmly under all the weaknesses (that are the contrarians’ strengths), and proceeded with the IPCC’s strengths and integrity intact. But now? After repeated attempts to say nothing’s wrong? I don’t know.
I fear the plan may be for current leadership to be made a scapegoat, as a way to separate the IPCC from the past. And that would be unjust.
Yeah, I share your feelings about how that got into the report. I am the review editor for a chapter of AR5, and this sort of thing keeps me up at night, because it will be my fault (to some extent) if something like this crops up in my chapter. Here is what I find interesting – a huge number of people look at/peer review these reports (including governments, NGOs and scientists) and offer comments. I am not sure of the number in the previous AR for IPCC, but my chapter of GEO-4 had hundreds of comments that had to be addressed. So, did all the reviewers miss this error too? If so . . . wow. Because not all reviewers are sympathetic to AGW, or at least how we talk about it now – so you would think a critic would have hammered this point in the review process. Or did this get added after review, thus ducking peer review (sort of a no-no, if you ask me), and if so, who did it? Or were critical comments about this data point ignored by the review editor and CLAs of the chapter (also a huge no-no)? I really don’t know, and I have not seen a clear statement on what the hell happened here. This does bother me, because accountability does matter – if I screw up, it is my failure and I will own it. Why have we not seen a clear statement here of who screwed this up, and why? It would certainly help to clarify the process, and improve transparency . . .
I agree with your point about the leadership – people are casting around for someone to throw under the bus, but this is a more historical and institutional problem than something that should (or can) be pinned on one or two “top people”.
Some of the official comments on glaciergate are discussed in the IAC report. There’s also the comment reported here by Georg Kaser – although this has been denied elsewhere.
As to why they kept it in, I only know of the comment reported here from Murari Lal. Don’t use that one – it’s not a good excuse.
Getting a clear statement on how it all happened would be nice, but I suspect that any statement will have to be filtered through the trained media management people first, so I’m not hopeful.
As for worrying about something like that getting through on your watch, I’d suggest you document your efforts to make sure it doesn’t – things like checking that every section has been reviewed by at least two people, asking any known sceptics on your team to act as devil’s advocate for you and pick out the most important issues, letting any critical reviewers know how the team plans to respond with a quick email and getting their sign-off that they’re satisfied before moving on, for remaining unresolved disputes, seek a third opinion, or consider adding a footnote that a dispute exists, list the things you didn’t have time to check, etc.
Then make sure that’s all archived and publicly available, as it should be, so everybody can see you did your best. And if anything does get through, you will have more shoulders to share the burden of blame across.
You might also find it worthwhile reading what the sceptics complained about last time with regard to review editors not doing their jobs properly, and then trying not to repeat the same mistakes.
The story related to Lal is pretty interesting – though it does not, to me, suggest anything nefarious. Instead, what it sounds like is the review editor and the chapter authors were completely overwhelmed and simply let this slip . . . and then were unlucky enough to highlight the very thing they’d let slip. I know this is a generous read, but I have firsthand experience with the hundreds of comments that come in on this sort of an assessment, and they do bury you. I also don’t believe that the CLAs would be stupid enough to think they’d get away with intentionally falsifying part of the report – and they would know that any such effort would, in the long run, do tremendous damage to the credibility of the larger report. I just have difficultly believing that anyone was so stupid as to put a short term media blurb above the long term integrity of the report. I know a lot of the people in this community, and the vast majority are good scientists.
The story from Pielke’s blog is a bit more ambiguous – the scientist critiquing the glacier finding was in WG I, and therefore would have had little to no contact with the authors of WG II – so, we can all agree that the report was way out of whack, but it doesn’t really seem to me that there were a hell of a lot of voices screaming about it until it was published – some review comments, mostly.
I appreciate the link to previous complaints about review editors – this is quite useful. Sadly, a lot of your recommendations are a bit out of my hands, in that I serve under CLAs. Yes, I run the review process, but what the chapter CLAs choose to address is not really under my control – ditto for how they choose to resolve disputes, etc. So you are correct, I will have to keep an email trail of all my suggestions/comments/etc. in case things go bad. I genuinely hope they will not – and given this furore, I suspect we might eventually get a better process here.