15 January 2013

Extreme Misrepresentation: USGCRP and the Case of Floods

The US Global Change Research Program has released a draft national assessment on climate change (here in PDF) and its impacts in the United States, as required by The US Global Change Research Act of 1990 (which incidentally was the subject of my 1994 PhD dissertation). There has been much excitement and froth in the media.

Here I explain that in an area where I have expertise on, extremes and their impacts, the report is well out of step with the scientific literature, including the very literature it cites and conclusions of the IPCC. Questions should (but probably won't) be asked about how a major scientific assessment has apparently became captured as a tool of advocacy via misrepresentation of the scientific literature -- a phenomena that occurs repeatedly in the area of extreme events. Yes, it is a draft and could be corrected, but a four-year effort by the nation’s top scientists should be expected to produce a public draft report of much higher quality than this.

Since these are strong allegations, let me illustrate my concerns with a specific example from the draft report, and here I will focus on the example of floods, but the problems in the report are more systemic than just this one case.

What the USGCRP report says:
Infrastructure across the U.S. is being adversely affected by phenomena associated with climate change, including sea level rise, storm surge, heavy downpours, and extreme heat… Floods along the nation’s rivers, inside cities, and on lakes following heavy downpours, prolonged rains, and rapid melting of snowpack are damaging infrastructure in towns and cities, farmlands, and a variety of other places across the nation.
The report clearly associates damage from floods with climate change driven by human activities. This is how the draft was read and amplified by The New York Times:
[T]he document minces no words.

“Climate change is already affecting the American people,” declares the opening paragraph of the report, issued under the auspices of the Global Change Research Program, which coordinates federally sponsored climate research. “Certain types of weather events have become more frequent and/or intense, including heat waves, heavy downpours, and, in some regions, floods and droughts."
To underscore its conclusion, the draft report includes the figure at the top of this post (from Hirsch and Ryberg 2011), which shows flood trends in different regions of the US. In a remarkable contrast to the draft USGCRP report, here is what Hirsch and Ryberg (2011) actually says:
The coterminous US is divided into four large regions and stationary bootstrapping is used to evaluate if the patterns of these statistical associations are significantly different from what would be expected under the null hypothesis that flood magnitudes are independent of GM [global mean] CO2. In none of the four regions defined in this study is there strong statistical evidence for flood magnitudes increasing with increasing GMCO2.
Got that? In no US region is there strong statistical evidence for flood magnitudes increasing with increasing CO2. This is precisely the opposite of the conclusion expressed in the draft report, which relies on Hirsch and Ryberg (2011) to express the opposite conclusion.

Want more? Here is what IPCC SREX, the recent assessment of extreme events, says (here in PDF):
There is limited to medium evidence available to assess climate-driven observed changes in the magnitude and frequency of floods at regional scales because the available instrumental records of floods at gauge stations are limited in space and time, and because of confounding effects of changes in land use and engineering. Furthermore, there is low agreement in this evidence, and thus overall low confidence at the global scale regarding even the sign of these changes.
The SREX is consistent with the scientific literature -- neither detection (of trends) nor attribution (of trends to human forcing of the climate system) has been achieved at the global -- much less regional or subregional -- levels. Yet, USGCRP concludes otherwise.

The leaked IPCC AR5 SOD reaffirms the SREX report and says (here in PDF), in addition to documenting a signal of earlier snowmelt in streamflows, no such signal of increasing floods has been found:
There continues to be a lack of evidence regarding the sign of trend in the magnitude and/or frequency of floods on a global scale
The IPCC has accurately characterized the underlying literature:
Observations to date provide no conclusive and general proof as to how climate change affects flood behaviour
Given the strength of the science on this subject, the USGCRP must have gone to some effort to mischaracterize it by 180 degrees. In areas where I have expertise, the flood example presented here is not unique in the report (e.g., Hurricane Sandy is mentioned 31 times).

Do note that just because the report is erroroneous in areas where I have expertise does not mean that it is incorrect in other conclusions. However, given the problematic and well-documented treatment of extremes in earlier IPCC and US government reports, I'd think that the science community would have its act together by now and stop playing such games.

So while many advocates in science and the media shout "Alarm" and celebrate its depiction of extremes, another question we should be asking is, how is it that it got things so wrong? Either the IPCC and the scientific literature is in error, or the draft USGCRP assessment is -- But don't take my word for it, check it out for yourself.

22 comments:

  1. I was waiting for this. Now who will defend the report? I read the ExSumm and part of the Agriculture chapter and thought it could have been WWF or Greenpeace, in both tone and conclusions. As well as being, imho, erroneous, but what do I know, it read as if its task was really to prove that things are in fact as bad as they can be.

    So my smell test alarm was blaring.

    ReplyDelete
  2. Living in Massachusetts, I chose to look at the Northeast regional section of the report. Not only is it biased (as in less than perfectly even handed), it is one-sided advocacy, pure and simple. After one passing mention of some improved agricultural conditions in the introduction, it goes on to entirely negative details.

    No mention of the benefits of warmer winters - like we had last year. Less snow on the ground? Fewer deaths due to heart attacks from seniors shoveling. Fewer road accidents on slippery roads. Fewer power outages due to downed wires. And less spending on heating, both domestic and commercial. And since we in New England tend to use oil to heat our homes, less air pollution and less CO2 released as well.

    With the rise of interest in locally grown food, longer growing seasons means more produce grown by local farmers - and more people attracted to farming, with land use benefits that come with agriculture. More farms means more jobs, as well as more food dollars staying within the state, to recycle within the local economy.

    And warmer weather means more companies willing to stay in or come to Massachusetts. Given a choice of warm versus cold, many entrepreneurs and executives prefer warm - see California. The report focuses on the rare +95 degree days, while ignoring those 60s that become 70s and 30s become 40s.

    The discussion of sea level mentions non-climate reasons for increased sea level in passing (to cover themselves, I assume) and then talks as if all harm from sea level increases comes from AGW. No mention of the fact that sea level at The Battery in New York has been going up at a constant rate since 1850(!).

    If this report was written by Greenpeace, I'd understand. I don't expect an honest and balanced discussion from advocates. From a government-sponsored project, it's grotesque, and fundamentally dishonest. This isn't just a shaded report, where someone tweaked the language around the edges - these people went all in, as if it was meant to be seen only be the echo chamber of green advocacy groups.

    ReplyDelete
  3. Very brave Roger, to swim into a hurricane.

    That report is the fantasy, this is the reality of American energy policy.

    The banks and oil companies have given up on carbon trading are are investing in oil, gas and fracking. The little green pixies are surplus to requirements.


    New York Times dismantles environment desk

    http://www.guardian.co.uk/environment/2013/jan/11/new-york-times-dismantles-environment-desk



    Chicago Climate Exchange. The hub of carbon trading in the USA, closes.

    http://news.investors.com/ibd-editorials/110910-553236-the-crash-of-the-climate-exchange.htm?p=full


    US may soon become world’s top oil producer

    NEW YORK—US oil output is surging so fast that the United States could soon overtake Saudi Arabia as the world’s biggest producer.

    http://www.guardian.co.tt/business/2012-10-24/us-may-soon-become-world%E2%80%99s-top-oil-producer


    Guess Who's Fueling the Fracking Boom?

    One deal that opened up the floodgate of big money interest in fracking was KKR’s $312 million investment in June 2009 for 33% of East Resources, a Pennsylvania oil and gas exploration firm founded by Terrence Pegula.

    http://www.forbes.com/sites/halahtouryalai/2012/10/03/guess-whos-fueling-the-fracking-boom/

    ReplyDelete
  4. Here are some claims from the section on Alaska and the Arctic in the Assessment Report.

    Alaska will warm rapidly in the rest of this century 6°F to 8°F in the north(p. 761)

    Models project 15 to 25 day increase in length of the snow-free and frost-free seasons (p. 761)

    Summer sea ice is receding rapidly and is projected to disappear by mid-century (p. 762) (References are to post 1979 exclusively.)


    Polar bear population estimated to be in decline (p. 764)

    Some models project that near-surface permafrost will be lost entirely from large parts of Alaska by the end of the century (p. 768) (Projections are for 1 meter depth.)

    Thawing permafrost causes . . . the release of heat-trapping gases that increase climate warming (p. 767)

    Ocean waters globally have become 30% more acidic due to absorption of large amounts of human-produced carbon dioxide (CO2) from the atmosphere. (772)

    All of these projections are from models and are expressed with "High Confidence"

    Exaggeration abounds in the above statements:

    1. Alaska is cooling, not warming.

    2. An Arctic ice record low for the satellite era lasted for a month or two, and was caused by a rare autumn cyclone.

    3. Polar bears are increasing according to observers on the ground.

    4. The top meter of permafrost contains little gas, and in Siberia, permafrost areas are growing.

    5. Marine life, including corals, thrive with more CO2, and the pH of ocean water varies annually by +/- 0.3 around an average of 8.1. It hasn't been neutral (7.0), let alone acidic, in 600 million years.

    ReplyDelete
  5. This is simply another example of a major scientific organisation manipulating data/reports to suit a predetermined agenda. I would have thought that the recent negative exposure of other reports from organisations like IPCC and NOAA would have made the scientific community very wary of reproducing anything similar. However, its apparent that the arrogant disregard for the truth is as strong as ever and I’m just glad that this report has been seen for what it is at an early stage.
    I have to wander when the people responsible for manipulating the findings of this report and others like it will be held accountable. Remember that reports like this provide the inputs in to major policy decisions that affect billions of people. Misrepresentation of findings to gain future funding and adversely influence major policy decisions is criminal fraud.

    ReplyDelete
  6. That is not all. I have not finished with this report but their Figure 2.2 is both fraudulent and fraudulently presented. Horizontal (temperature) scale is squashed together to create that menacing red triangle of rising temperature on the right. I have demonstrated that between 1979 and 1997 global mean temperature stayed the same (see "What Warming?" Figure 15). But they show it as a part of that imaginary "late twentieth century warming." In the twenty-first century 2005 and 2010 are shown jointly taking first place as the two highest temperature peaks in history. That is a complete fraud. In Actual fact the highest peak in instrumental temperature record is the super El Nino of 1998. 2010 is an El Nino peak but 2005 is part of the seven-year twenty-first century high platform and has nothing to distinguish itself. To top it off, the scale for the carbon dioxide is so chosen that it looks like it follows the temperature rise. That is achieved by cutting off the bottom of the scale and stretching the units to bring it in line with the rising red triangle. Another PR trick for the unwary. When such fraudulent data appear at the beginning of a thousand-page report it is a sign that this report is a propaganda piece and cannot be trusted as real science.

    ReplyDelete
  7. I attended a couple of meetings of the federal advisory committee used by the global change research program. I wrote about it here:

    http://tinyurl.com/3z8wd88

    The committee consisted of about 40 academics, federal employees, people from environmental organizations, etc. . All were approved by the White House (i.e. John Holdren). There was no dissent and every vote was unanimous. Every indication was that the participants knew why they were there and the outcomes were preordained. In other words the committee was window dressing or a pretend advisory committee.

    ReplyDelete
  8. I agree that the text takes H&R out of context, but the H&R findings are rubbish.

    They neglect to account for the integration between CO2 and temperature, fail to control for forcings other than CO2, and ignore spatial correlation and fingerprinting against model predictions.

    So, the H&R trend map may be OK, but the conclusions they draw about trend significance are completely spurious.

    More details at the link. I should have turned this post into a comment to the journal, but was too busy with my day job:

    http://blog.metasd.com/2011/10/linear-regression-bathtub-fail/

    ReplyDelete
  9. I read the report. It read like a 'worst case scenario' planning document.

    To be fair, there is nothing wrong with looking at worst case scenario's from a planning perspective. The DOD does it all the time.

    ReplyDelete
  10. Strangely, the figure at top is actually NOT from Hirsch & Ryberg, unless there's more than one version floating around. The fig in the Hydrological Sciences Journal article (2011 online, print 2012) shows CO2/streamflow correlations, and this fig shows flood trends. Also, the original article is not in color. Perhaps this is a replot/reanalysis of the data, courtesy of one of the authors, incorrectly cited?

    ReplyDelete
  11. Interesting quote there from Hirsch and Ryberg. Now some, not Eli to be sure might wonder why you left off the next sentence:

    ------------------

    In none of the four regions defined in this study is there strong statistical evidence for flood magnitudes increasing with increasing GMCO2. One region, the southwest, showed a statistically significant negative relationship between GMCO2 and flood magnitudes.
    ------------------

    ReplyDelete
  12. Roger, you also may want to take a look at Tom Fiddaman's comments on the paper. In short, it looks like this it is not very useful, and the data pretty much agrees with the AR4 predictions for precipitation in North America

    ReplyDelete
  13. #11 Eli,
    The US is only a couple of percent of global land area and one region of the US is much less. Many (I don't know if it included you) keep complaining when skeptics comment on a cold winter in some region, or some other piece of evidence that unusual warming (or drought or flooding, etc.) is not happening, and say this is only a small area of the globe, and thus not relevant. In fact there is some relatively small global region (say a small country sized) that has had very unusual weather (record wet, dry, hot, cold, stormy, etc.) almost every year. Your point on the southwest being the only region with some variation of interest, is not very meaningful considering the poor quality of data available more than a few years back.

    ReplyDelete
  14. I would like to add my agreement and more substance to what Tom Fiddaman said about the H&R study being rubbish. From what I can tell, they completely oversimplified the causal pathway between GMCO2 and flooding. Unless someone can explain to me why we would expect for GMCO2 to be directly related to magnitude and frequency of flooding (key work DIRECTLY), I would assert that the hypothesis was flawed from the outset.

    It makes much more sense that a potential causal pathway (and there's likely more than one pathway) would look something more like this: GMCO2 --> increased average temperature --> polar ice caps melting --> sea level rise --> annual peak stream flow. Thus, we would expect the correlation to be distorted, even negligible, as it is acting through multiple mediating factors.

    I'm also concerned about their measurement of the variables. Let's start by looking at GMCO2, since that was their preferred independent variable. GMCO2 is probably an inaccurate proxy for CO2 measurements in specific regions. For one, it varies with seasonality. For two, it varies spatially across the globe. Is it fair to assume that the global mean CO2 for a given year is at all similar to the CO2 level in each of their defined regions?

    Adding to that is the fact that our dependent variable, streamflow, is not dependent on just one variable (ie sea level rise or the more distant connection to GMCO2). Factors such as soil saturation, rainfall amount, rainfall intensity, change in vegetation or soil type, topography can influence peak streamflow and can vary from year to year. All of these will have different relationships with the proposed independent variable too. This is what we call the potential for confounding.

    To further complicate matters, increase in average temperature doesn't have the same effects everywhere. While sea level may rise, we also experience periods of drought. Obviously, when there is drought (like in the Southwest), we would not expect more flooding, and would thus construct a different hypothesis that is more consistent with our knowledge and common sense.

    Do I dare mention that there are also hypotheses that increased air temperature may also precede increase CO2 levels? This would also change the hypothesized causal model I drew out earlier.

    All of this is just a long winded way of saying that we should think about the studies that we are citing, how we are citing them, and whether or not they are based on good hypotheses and assumptions. Every good scientist knows that correlation does not mean causation and that poor measurements can significantly alter your results. No matter which side of the argument you are on, it is essential to understand the complexity of interactions we are investigating and the directly related complexity of the mathematical models we must run to appropriately test these hypotheses.

    ReplyDelete
  15. In other news

    Coca Cola does not make people fat.

    Extremely violent cinema does not lead to violence in individuals or society, it's art. Quentin Tarantino is a liberal.

    International holidays and flights advertised by liberal newspapers for well off readers do not contribute to global warming.

    Large, high powered cars advertised by liberal newspapers for well off readers do not contribute to global warming.

    Expensive wine from France arrives by sailing ship. It always has.

    Barack Obama's full speed ahead extraction of fossil fuel in the USA does not contribute to global warming because he isn't an idiot like GW Bush.

    The global financial crash had nothing to with AIG or Lehman Brothers. It was caused by stupid, poor people buying houses.

    Scientists never stray from the truth because they have the purist intellectual motivation. Aristotle never owned a car.

    ReplyDelete
  16. Roger,

    Like Eli I want to know how is it that you got it so wrong copy and pasting from a research paper? Surely someone as competent as you could not make such a simple, yet critical error. Some would rightly argue that you deliberately did so to mislead your readers and to help force the authors' work to fit with your narrative and objective.

    This would ordinarily not necessarily be a big issue, but when the person who is quote mining is also quick to make accusations of wrong doing or incompetence against others, it starts to smack of hypocrisy by said quote miner.

    Thanks!

    ReplyDelete
  17. I agree that "we should indeed think about the studies we are citing and how we are citing them." Unfortunately, it seems that the USGCRP authors failed to do this.

    ReplyDelete
  18. It seems to be after being around for 73 years that it is not uncommon to be lied to by those in positions of responsibility.
    I remember in school the teachers telling us how wonderful the Swiss were, yet in later years I learned that a lot of dirty work went on between the Nazis and the Swiss (hiding of Jewish treasure)
    We were taught in school that we descended from lower life (Monkeys etc)
    I learned that is not true.
    I have recently learned that Big Banks are really dishonest with the poor man out in the neighborhoods.
    I have come to know that the religious clergy Of Catholic and Protestant Fundamentalist churches ( with the exception Jehovah's Witnesses) have lied to the people telling them poor folks burn in Hell, there future is determined by fate, that God is a Three headed Mystery, and don't forget the sex scandals that are often reported.
    Like the the Bible writers long ago said "every man is a Liar

    ReplyDelete
  19. Eli, Tom and FoodforThought,

    The H&R 2011 paper contains the exact same graph as that shown at the top of this post, albeit Roger's is in color and the journal's is in B&W since the journal likely charge authors to print color figures. However, Tom is confusing the meaning of the figure. He claims the paper shows it as 'CO2/streamflow correlations' instead of flood trends. The caption of the figure in the actual article clearly states that the figure displays beta_1 for the model presented earlier in the article which correlates global mean CO2 concentrations and PEAK streamflow. Earlier in the paper the authors clear state the purpose of the study by saying,

    'This study attempts to quantify the changes taking place in the flood behaviour in the coterminous United States as a function of greenhouse gas concentrations'

    Therefore, it seems rather odd for anyone to make a claim that there proposed model is not testing for this relationship or that the results from said model show CO2/streamflow correlations.

    As for physics, the baseline physical understanding of the greenhouse effect is that is a global energy budget effect. Therefore, any correlations between changes in greenhouse gas concentrations and climatic parameters should take into account globally averaged mean amounts of CO2. The connection between flooding and GMCO2 would be that average temperatures have increased enough to find noticeable changes in the occurrence of floods across the US due to increased snow melt at higher elevations, where most of the fresh water in north America comes from. For this change in temperature to occur from the greenhouse effect, as a globally averaged energy budget effect, one must account for the globally averaged amount of CO2.

    This line of reasoning is straightforward enough that at least one, if not three reviewers allowed the article to be published...

    ReplyDelete
  20. Adding a bit to Maxwell's comment above in #19, Hirsch & Ryberg's paper uses "peak streamflow" (the regressand) synonymously with "flood magnitudes." As Tom says in #10, H&R's original map shows a coefficient of correlation, that of ln(peak streamflow) to pCO2 in ppmv (β1). However, as pCO2 has had a marked trend over the period studied, it's not a stretch to understand β1 as representing a trend in peak streamflow as well. I note that the USGCRP hasn't completed that transformation: while the figure caption indicates "trends", and the legend states "absolute percent change per decade", the legend retains the β1 numerical values. At the current pCO2 rate of ~20 ppmv/decade, β1 of 0.01 corresponds to about a 20%/decade change in peak streamflow.

    ReplyDelete
  21. Eli and Albatross I am at a bit of a loss on your comments about how the quote was edited. The part left out says that in one region there is a NEGATIVE correlation between increased CO2 and floods. This is the complete opposite of the government report. What is your point?

    ReplyDelete
  22. Nicolas: as you say, if Roger did selectively quote mine, he left out a sentence that supports his point.

    (btw Eli doesn't have a point. He never does. His aim, always, is to obscure. Shame on Albatross for falling for it.)

    ReplyDelete