“Bias and the IPCC reportAccentuate the negative” heads the article from the Economist on line:
FOR everyone else it was the glaciers: for the Dutch it was the flooding. Last January errors in the work of the Intergovernmental Panel on Climate Change (IPCC) hit the headlines. The chapter on Asia in the report by the IPCC’s second working group, charged with looking at the impact of climate change and adapting to it, mistakenly claimed that the Himalayan glaciers would be gone by 2035. This contradicted some reasonably basic physics, had not been predicted by the glacier specialists in the first working group (which deals with the natural science of past and future climate change) and was unsupported by any evidence. There was a report from the 1990s which said something similar about all the world’s non-polar glaciers, but it gave the date as 2350. Then there was a crucial typo and some shoddy referencing. Nevertheless the IPCC’s chair, Rajendra Pachauri, had lashed out at people bringing the criticism up, accusing them of “voodoo science”. He then had to eat his words, and set up, with Ban Ki-moon, a panel to look into ways the IPCC might be improved.
Inspired by this to look for other errors, a journalist for a Dutch newspaper spotted that the chapter on Europe gave a figure for the area of the Netherlands below sea level that was much too large. The area at risk of flooding by the sea had been conflated with that at risk of flooding by the Rhine and the Meuse rivers. That the careful Dutch should have provided faulty information and not spotted it in the review process was an embarrassment to the then environment minister, Jacqueline Cramer; following a debate in parliament she called on the Netherlands Environmental Assessment Agency (PBL), an independent body, to look at all the regional chapters in the working group II report and make sure they were up to snuff. This the PBL has now done; its report was published on July 5th.
The authors try hard to make clear that their findings do not undermine the IPCC’s conclusions on climate change. And there is nothing in their report as egregious as the glaciers or as embarrassing as the Dutch sea level. But they did find a number of things to take issue with, most of which they thought minor but eight of which they classed as major; and their work seems to bring out a systemic tendency to stress negative effects over positive ones. This tendency can be defended. But a reading of the report suggests there may also be broader and potentialy more misleading bias. The PBL report chose as its main focus a table in the “Summary for Policy Makers” of the IPCC’s 2007 “Synthesis Report”, which brings together the results of working groups one, two and three (which deals with responses to climate change). Where did these bullet points actually come from, the PBL team asked, and how well supported were they?
The auditors found one new error which they deemed major: a statement about the frequency of turbulence in South African fishing waters which had been translated directly into a statement about the productivity of the fisheries. The IPCC has indicated it will produce an erratum for this, and for a number of other errors all concerned deemed minor. But the PBL also identified seven statements, which, while not errors, it thought were deserving of comment (for which read criticism).
Perhaps the most striking relates to Africa. The table in the summary for policy makers reads: “By 2020, in some countries, yields from rain-fed agriculture could be reduced by up to 50%.” The evidence on which this is based says only that yields during years in which there are droughts could be reduced by 50%. Furthermore, the relevant reference applies only for Morocco—and it cites as its source an earlier paper that the PBL says no.
Other criticisms turn on a tendency to generalise. Research showing decreased yields of millet, groundnuts and cowpeas in Niger becomes a claim that crop yields are decreasing in the Sahel, the strip that separates the Sahara from the savannah in Africa, rather than that the yields of some crops are decreasing in some parts of the Sahel. The results of research on cattle in Argentina are applied to livestock (which would include pigs, chickens, llamas and the rest) throughout South America. The expert authors do not provide a compelling reason for their claim that fresh water availability will decline overall in south, east and southeast Asia, or that the balance of climate-related effects on the health of Europeans will be negative.With the exception of the South African fisheries it is not clear that any of this is wrong, which is why, on these matters, the PBL does not speak of error. Martin Parry, a specialist in agriculture who was the co-chair of the second working group’s report, defends his colleagues’ work. Agriculture in other parts of North Africa is very like that in Morocco, and during droughts the crop yields there already drop by more than 50%. To say that yields decline in the Sahel does not mean all crop yields in all of the Sahel. Cattle make up most of Latin America’s livestock, and much of the rest of it can be expected to do worse. The IPCC does not just assemble evidence, Parry stresses: it assesses it. When its expert authors weigh their words on things like water in Asia and health effects in Europe they do so in the context of a wide range of knowledge. And they do so in ways that cannot be reduced to ticks in the boxes of Dutch assessors going through things line by line four years later.
The authors might better document the extra insight brought to bear, and be more transparent about the application of their judgment. But at 1000 pages the Working Group II report alone is already a challenge to the book-binder’s art. Does it really need to be longer?
Another problem identified by the PBL analysis is that, in general, negative impacts are stressed over positive ones. The table in the summary for policymakers is almost unremittingly bad news; the conclusions in the chapters that fed into it, while far from cheery, were more mixed. In a similar way, when there is a range of possible impacts, the top end of the range tends to get more play in the summaries for policy makers than the bottom end does. The PBL says that this is a reasonable way to proceed in a document that is explicitly aimed at policy makers thinking about adaptation, but it is not clear how transparent this approach is to readers.
This may reflect a larger issue. Work on the impacts of climate change—the literature Working Group II assesses—tends to focus on vulnerabilities and damage for much the same reason the IPCC authors do. They seem more important, more urgent and quite possibly more fundable. The UN Framework Convention on Climate Change requires countries to assess their vulnerabilities, and these assessments are fodder for Working Group II (one of them was the source for the 50% drop in rain fed agriculture yields). Thus the evidence base from which an assessment of impacts has to start is to some extent skewed.
Perhaps the most worrying thing about the PBL report, though, is a rather obvious one about which its authors say little. In all ten of the issues that the PBL categorised as major (the original errors on glaciers and Dutch sea level, and the eight others identified in the report), the impression that the reader gets from the IPCC is more strikingly negative than the impression which would have been received if the underlying evidence base had been reflected as the PBL would have wished, with more precise referencing, more narrow interpretation and less authorial judgment. A large rise in heat related deaths in Australia is mentioned without noting that most of the effect is due to population rather than climate change. A claim about forest fires in northern Asia seems to go further than the evidence referred to—in this case a speech by a politician—would warrant.
The Netherlands look more floodable, Asian glaciers more fragile. A suspicion thus gains ground that the way in which the IPCC sythesises, generalises snd checks its findings may systematically favour adverse outcomes in a way that goes beyond just serving the needs of policy makers. Anecdotally, authors bemoan fights to keep caveats in place as chapters are edited, refined and summarised. The PBL report does not prove or indeed suggest systematic bias, and it stresses that it has found nothing that should lead the parliament of the Netherlands, or anyone else, to reject the IPCC’s findings. But the panel set up to look at the IPCC’s workings by Dr Pachauri and Mr Ban should ask some hard questions about systematic tendencies to accentuate the negative.
