Update to ‘IPCC systematically low-balling climate estimates?’

Four months ago I wrote a post about how several speakers at the AGU 2012 Fall Meeting suggested that the IPCC may be systematically underestimating several key climate change-related parameters (total anthropogenic GHG emissions, Arctic ice melt rates, sea level rise projections).   Sure enough, one of those speakers is co-author on a new paper that proposes there is a systematic trend towards ‘erring on the side of least drama’, i.e., of backing off from scientifically-rigorous predictions that could be interpreted as alarming:

Brysse, K.; Oreskes, N.; O’Reilly, J.; Oppenheimer, M. Climate change prediction: Erring on the side of least drama? Global Environmental Change 2013, 23, 327–337.

At about the same time this was published, the Economist came out with a very different article suggesting just the opposite, that global climate models are currently over-predicting temperature increases because estimates of climate sensitivity to CO2 emissions are too high.  Check out DotEarth blog or Yale 360 for additional technical discussions of the issue.  For the record, while the article in the Economist is extremely interesting, I find its dismissive tone troubling- even if climate change is coming in on the low end of the prediction range, it’s still serious enough to warrant a much greater effort towards mitigation and adaptation than humanity is currently making.

Observed vs. simulated surface temperatures, from http://www.climate-lab-book.ac.uk/2013/updated-comparison-of-simulations-and-observations/

That being said, for a lively rebuttal of the Brysse et al. 2013 article, check out http://rogerpielkejr.blogspot.com/2013/02/science-is-shortcut.html (Disclaimer- this blog post is what led me to the paper in the first place).  After having read the article in its entirety, I have to say that I think Dr. Pielke’s harsh critique is on-point, and that Brysse et al. 2013 is characterized by cursory analysis, anecdotal reasoning, and tendencies toward proof by assertion and appeal to authority (Disclaimer 2- I’m not a social scientist, so my standards of what constitutes legitimate reasoning or convincing arguments might be fundamentally different than that of the authors).  The hypothesis presented, that scientists have more incentive to bias their work towards under-estimation of the severity of environmental impacts than towards over-estimation, is entirely reasonable and worthy of analysis.  However, the article unfortunately offers little actual analysis:  it cites only a small number (5) of previous assessments (none of which is more current than 4 years old, even though more recent critical reports are available), makes no systematic attempt at quantitative analysis of the results, has no mention of statistics (perhaps unsurprising given the small sample size), and offers not even a single figure or table to help organize their thoughts or help the reader follow their critique of those multiple studies.  The article goes on to present some anecdotal case studies of past instances where scientists have been taken to task for over-predicting certain environmental impacts, and ends with a far-ranging discussion (which I found genuinely interesting) of how skepticism of new theories is inherent in modern science, with references to Darwin, Lyell, and Kuhn.  Interesting stuff to be sure, though I remain unmoved by their overall conclusions.

 

PS- One of the authors of the Brysse 2013 study will be lecturing at CSU on Thursday.  I’m very curious what she will have to say!

Quality control in Excel spreadsheets- a serious and universal issue!

Via Paul Krugman’s blog, I’ve been following a fascinating online discussion about the importance of spreadsheet error-checking and independent replication of modeling results in the economic research sector:

• Holy Coding Error, Batman
• Researchers Finally Replicated Reinhart-Rogoff, and there are Serious Problems
• How Much Unemployment Was Caused by Reinhart and Rogoff’s Arithmetic Mistake?
• Harvard duo defend case for austerity

In a nutshell, when an independent research team finally succeeded in reproducing the results of a controversial recent paper that had big implications for policy response to the  recession, they found, in addition to a series of questionable assumptions and weighting schemes, a basic error in the Excel spreadsheet that many of us users are familiar with:

I just hate it when I add extra rows rows of data and forget to update my equation arguments accordingly- if only Excel would warn me about that!

Economic policy implications of the RR paper aside, I’m personally very interested in the best practices associated with Excel modeling.  Many lifecycle assessment (LCA) tools are Excel-based since the program makes it so easy to represent and document the input-output relationships inherent in that kind of modeling.  In fact, much of my own recent work has been based around big Excel spreadsheets that I put together from scratch.  I first started using Excel when I worked in R&D prior to coming back to grad school.  The company I worked for did most of their general product design calculations in Excel, and had developed a set of best practices for model structure and documentation – practices that I still try to use today in my own work.  Beyond my own profession experience, Excel is an indispensable tool across many industries, particularly finance, yet it seems that everyone struggles with the same issues and is prone to the same types of mistakes, in most cases with much bigger repercussions.

Since Excel is such a useful and universal tool, how can we develop best practices to identify and avoid these user errors to which it is so prone?  Here are a few suggestions that I have started to implement in my own work:

Built-in error-checking-  PAY ATTENTION WHEN THE PROGRAM FLAGS A POTENTIAL PROBLEM!  The version of Excel I’m most familiar with (2011 for Mac) has a feature where, if it senses inconsistencies between adjacent cells (i.e. if you do a calculation using a series of cells but exclude some adjacent non-blank cells, or if you have two adjacent cells with the same equation, followed by a third with a different one) it flags it with a little green triangle.  It can actually be pretty annoying, since 95% of the time it’s flagging things that you did intentionally.  But for that other 5% the feature can be a lifesaver, and it was designed for exactly the type of error identified in the RR paper.

… oh wait, it does warn me!

Spreadsheet structure- Even though a lot of spreadsheets out there are extremely difficult to follow, it’s actually very easy to keep them well-organized.  I typically use a four-column ‘name-value-units-notes’ structure to ensure that the proper variable names and units follow each calculation.  I always keep unit conversions separate from actual computations (lumping it all together gets really really confusing), and sometimes break big ugly computations into two parts so you can still see what’s going on with just a glance at the equation bar.  When the analysis gets big, I color-code the different sheet tabs, and then change the color of individual cells that contain important inputs from different sheets to match, to facilitate tracing out the underlying dataflow.  Any input values based on assumptions get highlighted in bright red.  All of this requires a little time to implement, but in my experience it’s an investment that pays for itself the first time you come back to the spreadsheet later and are trying to remember what you were doing earlier.  None of these little tips guarantee that you won’t make mistakes, but they greatly increase the readability of the finished product and the likelihood that you (or someone else) will catch little errors.

Sensitivity analysis-  This is modeling 101.  If your result is dependent on any uncertain inputs or basic assumptions (and what model isn’t?), at a minimum you should be perturbing those values and assumptions, observing how much your overall result changes, and reporting on your model’s sensitivity to those factors (it’s even better to do a full, rigorous uncertainty estimation, though in many cases that can be very challenging).  I’m personally very suspicious/dismissive of any published LCA results that don’t include sensitivity or uncertainty analysis; though it happens not infrequently, it reeks of the authors having a hidden agenda.  Note that the only-3000-some-word RR paper admits that their result ‘merit(s) further sensitivity analysis.’ Indeed.

Publishing spreadsheets- This is a tough one, and expectations seem to vary a lot by discipline.  While sharing your underlying spreadsheet models is a great way to get feedback from your peers and engage the research community, it’s not without thorny IP and proper-use issues.  If I share my model, am I limiting my ability to use it for future publications?  Should I be going through some sort of copywriting exercise?  Will others modify it in a way that I disagree with, and then publish contrary results?  All of this aside from that fact that you look somewhat foolish if someone does identify an error in it (even though this is a good thing from a scientific method perspective)!  Full disclosure- I have not yet shared the spreadsheet model underlying the publication that I linked to above.  I plan on doing so, but I haven’t yet made the time to clean up the file or go through the copyrightit-makeitreadonly-wheretohostit thought process…

Does anyone have any other tips to add to the list?

Postscript (for all the other doctoral students out there)-  While Excel is recognized as a potentially powerful platform for building simple models or doing basic data analysis, it’s a pretty crappy tool for data visualization, and it’s considered Busch-league to submit manuscripts with plots using the default Excel color schemes and formatting.  You can spot them from a mile away, and it gives some readers a bad first impression of the sophistication of your work (‘if they couldn’t even be bothered to make a nice figure in SigmaPlot or R, how good could the analysis be?’)

The “future” of clean coal – is not in our future

A new report from the CRS reveals the sad state of what many believe is an impossible venture – “clean coal” that involves capturing the CO2 released as the coal is burned:

http://thehill.com/blogs/e2-wire/e2-wire/293655-report-federal-clean-coal-power-project-faces-uncertain-future

IMF report on energy subsidies, implications for a carbon tax & energy security

IMF presents some new analysis of energy subsidies which I have previously wondered about.  It is worth looking over:

http://www.washingtonpost.com/blogs/wonkblog/wp/2013/03/27/imf-want-to-fight-climate-change-get-rid-of-1-9-trillion-in-energy-subsidies/

$25/ton as the cost of CO2 seems low to me.  $502 billion in US energy subsidies including this carbon tax.. I don’t think I’ve seen a figure like this calculated before.

John’s recent post on energy security along with a comment in this article had me revisit the “hartwell paper“.. here is a poignant quote from dalyplanet from the report:

The first step is to recognise that energy policy and climate policy are not the same thing. Although they are intimately related, neither can satisfactorily be reduced to the other. Energy policy should focus on securing reliable and sustainable low-cost supply, and, as a matter of human dignity, attend directly to the development demands from the world’s poorest people, especially their present lack of clean, reliable and affordable energy. One important reason that more than 1.5 billion people presently lack access to electricity is that energy simply costs too much. Obviously, if energy were free, then its provision would be simple. Even if such access could be supplied from fossil fuels – which is plausible but also debatable – this demand for access to energy, for reasons of cost and security should not be satisfied by locking in long-term dependence on fossil fuels.

Nice summary of the RFS (and a new blog to follow!)

For those who are confused about RFS, RINs, and other acronyms in my last post, I found a great summary on a new blog that we’ll add to our blogroll.  Seems like he is making some nice posts of the recent RFS issues and may be a good place to watch as things unfold…

Here is his summary of the RFS

 

CSU’s Dr. Diana Wall wins the Tyler Prize!

A big congratulations to our own Dr. Diana Wall, University Distinguished Professor and Director of the Colorado State University School of Global Environmental Sustainability, on winning the 2013 Tyler Prize for Environmental Achievement!

• http://www.eurekalert.org/pub_releases/2013-03/bc-asr031113.php

If you want to learn more about Dr. Wall’s work, check out this video of her contribution to the CSU TILT ‘My Favorite Lecture’ series.

As you can see from the list of previous laureates, this is an extremely prestigious award- there are a lot of science rock stars on the list!  While many readers are probably familiar with the work of people like Jane Goodall, Jared Diamond, and EO Wilson, I’d like to take a moment to highlight some of the others whose work is relevant to energy research and bioenergy in particular:

• Kirk R. Smith:  An epidemiologist at UC Berkeley, Dr. Smith has spent a career studying the health and climate change impacts of the pervasive use of biomass combustion for household cooking across the developing world, highlighting how the practice contributes more to mortality and morbidity worldwide than HIV and malaria.  His research and advocacy have helped spawn an international effort to disseminate cleaner cooking technologies, and have been the inspiration for some related research and enterprise efforts closer to home.  Dr. Smith’s seminal paper on the topic is almost 20 years old now, but still serves as a great introduction:

Smith, K. R. Health, energy, and greenhouse-gas impacts of biomass combustion in household stoves. Energy for Sustainable Development 1994, 1, 23–29.

• V. “Ram” Ramanathan:  One of the main pollutants produced from inefficient biomass combustion in developing countries, as well as from diesel engines in developed countries, is black carbon particulate matter.  While the detrimental health effect of breathing particulates are well-known, Dr. Ramanathan has been a pioneer in quantifying the contribution of black carbon (BC) emissions on climate change; current understanding suggests that BC makes a greater contribution than any other greenhouse gas besides CO2, and is also perhaps one of the easiest to mitigate.  More recently, Dr. Ramanathan has made a direct contribution to the understanding and dissemination of improved cooking technologies in India through Project Surya.

Ramanathan, V.; Carmichael, G. Global and regional climate changes due to black carbon. Nature Geoscience 2008, 1, 221–227.

• Paul J. Crutzen:  Dr. Crutzen is perhaps most famous for his pioneering work on the role of man-made chlorinated compounds in stratospheric ozone degradation, the so-called ‘hole in the ozone layer’, for which he was awarded the 1995 Nobel Prize in Chemistry.  However, his work covers a variety of atmospheric chemistry and biogeochemistry issues, and he has recently weighed in on the issue of nitrous oxide (N2O) emissions from bioenergy production, arguing that a full accounting of indirect emissions can offset the technology’s other GHG reductions.  While this particular paper has been challenged as being somewhat of a strawman, the work reminds those of us doing bioenergy lifecycle assessment of the importance of including indirect emissions. Fun fact: in the late 1970s Dr. Crutzen was a professor in the CSU in the Atmospheric Sciences Department.

Crutzen, P. J.; Mosier, A. R.; Smith, K. A.; Winiwarter, W. N2O release from agro-biofuel production negates global warming reduction by replacing fossil fuels. Atmospheric Chemistry and Physics 2008, 8, 389–395.

• Mario J. Molina:   Dr. Molina was a co-recipient of the 1995 chemistry Nobel with Dr. Crutzen.  Since then, he has taken an active interest in global warming, writing on how the same emissions control protocols that brought ozone depletion under control can make a contribution to reducing non-CO2 greenhouse gases and climate forcing agents such as HFCs and black carbon, and promoting terrestrial carbon sequestration.  Reducing BC emissions from cooking and deploying biochar are two bioenergy technologies that show up high on his to-do list:

Molina, M.; Zaelke, D.; Sarma, K. M.; Andersen, S. O.; Ramanathan, V.; Kaniaru, D. Reducing abrupt climate change risk using the Montreal Protocol and other regulatory actions to complement cuts in CO2 emissions. Proceedings of the National Academy of Sciences 2009, 106, 20616–20621.

Further evidence of what a cross-cutting topic bioenergy can be

Congratulations again to Dr. Wall!

Renewable energy companies sustain collateral damage in Washington budget battles

Thanks to Sam for passing along the following article highlighting how Paul Ryan’s House Republican budget document makes an explicit, unsubstantiated attack against two solar energy companies – both of which are alive and well – as examples of wasteful government spending in renewable energy:

• http://www.washingtonpost.com/business/economy/ryan-plan-calls-two-solar-projects-ill-fated-but-the-projects-are-doing-fine/2013/03/13/add9096a-8c08-11e2-b63f-f53fb9f2fcb4_story.html?hpid=z4

I would be extremely pissed off if I were a part of either of those ventures- while it’s fine that Ryan and his allies see a more limited role for government support of renewable energy enterprise, it is completely inappropriate and unjust for a national politician to slander these companies as “ill-fated ventures” in a widely-read policy document in order to make the broader point, particularly when you offer no justification for the claim.

Example of a concentrated solar thermal plant, tower design

I find it particularly interesting that one of the companies mentioned, Solar Reserve, is a concentrated solar thermal technology company, rather than a solar panel (PV) manufacturer.  Check out this video from the company for a quick overview of the technology: http://www.youtube.com/watch?v=73SNIuZ333s.  Because thermal energy in such a system is easily and cheaply stored as molten salt, the plant can continue to produce power during periods of cloud cover and during part of the night, so the technology starts to get around some of the intermittency issues associated with PV.  For our department seminar last month we had someone from NREL speaking about concentrated solar, and while there are some really cool technology advancements going on in this area right now, the basic pieces are very straightforward with very low development risk:  there are multiple plants currently operating around the world with the same general technology, and there’s no chance of being undercut by foreign competitors like other US solar companies have been.  So, ironically, this is probably one of the least-risky DoE renewable energy loans to make an example of!

Finally, for all of the conservative ruminations about government support of the solar industry, I would point out that one of the primary reasons the sector has recently struggled is due to Chinese competition, largely the result of an exercise in unsustainable commodity dumping that is now coming to an end.  Rather than yanking government support of renewable energy enterprises and libeling individual companies, we should be showing renewable energy the same commitment we have for medical and defense research, before the US is further eclipsed by foreign competition in a sector with such economic and geopolitical importance.