Reader Comments

Post a new comment on this article

How not to use CMIP5 simulations

Posted by edhawkins on 11 Jun 2015 at 11:58 GMT

Mora et al. perform calculations on the number of plant growing days using output from a range of CMIP5 simulations.

Unfortunately they use these simulations inappropriately for a number of reasons. The examples below are limited to temperature, but apply similarly to the other climate variables considered.

1) Mora et al. only quote and use the multi-model ensemble mean, stating that it is more accurate than most of the models individually. Yet the authors have no basis for assuming that past performance is any guide to future performance. There are good reasons why the multi-model mean may look most like the observations in the past, but the models produce a range of outcomes for the future, and this range cannot be discounted as Mora et al. have done. They have not quoted any uncertainty ranges in the text, yet their Fig. S3 highlights that the spread amongst the models is as large or larger than the signal in many, if not most, locations. Their use of CMIP5 simulations in this way is in sharp contrast to standard practice and what the IPCC recommends. Not quoting an uncertainty range is indefensible.

2) The individual models have biases in their simulated present-day climate. For example, temperatures may be too warm or too cold in particular regions. Indeed, simulated global mean temperature is up to 3C different across the CMIP5 models. This has critical implications for using the raw model output to determine the number of days above or below certain absolute thresholds. If these biases are not accounted for (as seems to be the case) then the number of days calculated is incorrect. Similarly, differences in the daily variance between the observations and simulations is also critical for the number of days above or below thresholds and would also potentially need to be corrected. There are well studied ways of trying to account for these biases, which appear to have been ignored.

3) Mora et al. use the daily mean temperature in their analysis, yet much research has highlighted that the daily maxima and minima are important for plant growth. Daily minimum and maximum temperatures are projected to change at different rates, with daily minima warming faster than daily maxima. This feature has also been seen in the observations, but does not appear to be accounted for in the present analysis.

Bringing together such disparate and large datasets to address the important questions posed requires expertise from a wide range of disciplines to understand the appropriateness, subtleties and caveats in their use.

Dr Ed Hawkins, University of Reading, UK
Twitter: @ed_hawkins

No competing interests declared.

RE: How not to use CMIP5 simulations

edhawkins replied to edhawkins on 12 Jun 2015 at 15:03 GMT

I have written a blog post discussing one aspect of the use of CMIP5 simulations, and how counting changes in days above, below or between thresholds is highly sensitive to simulated variability characteristics, illustrated with a case study:


In summary - using the raw daily data is inadvisable (or simply wrong).

Ed Hawkins

No competing interests declared.