Assessment of CMIP6 climate model simulations using python-based software

Dr Ian Watterson1, Roger Bodman1, Martin Dix1, Sugata Narsey2, Michael Grose3

1CSIRO Climate Science Centre, Aspendale, Australia, 2Bureau of Meteorology, Melbourne, Australia, 3CSIRO Climate Science Centre, Hobart, Australia


The CMIP6 project ( aims to better understand climate using climate models. Over 30 groups, globally, have contributed ‘historical’ simulations over the period 1850-2014. Comparison of the simulated and observed seasonal averages are important to the assessment of the skill of each model. We have developed new python-based software and applied it to 60 models in NCI’s vast data archive ( Model skill is quantified using the ‘M’ score, a normalised rmse, for which 1000 is ‘perfect’ and 0 is no skill.

The main code is a Jupyter notebook, which is run on NCI’s VDI. ‘Clef’ is used to determine the available data and ‘Iris’ modules then read this, often from multiple NetCDF files. The seasonal means for 1985-2014 are calculated and the data are interpolated to a common grid (0.25° for Australian land). The rmse and spatial means and variance are calculated and combined to give M. Results are written as a python ‘dictionary’ for each case. These are combined using a separate code, which then outputs tables.

The seasonal scores for temperature, rainfall and pressure are averaged as an overall skill score. Over Australia the top-scoring model is the UK Met Office’s HadGEM3-GC3.1-MM at M=784, while the CSIRO-ARCCSS model ACCESS-CM2 is fifth, at 742. The average score of 60 models, 676, is better than the CMIP5 average 643. It is concluded that CMIP6 models give a more realistic depiction of climate. The better models will be used to provide suitable input data for downscaling to local scales.


Dr Ian Watterson is a research scientist in CSIRO’s Climate Science Centre, based at Aspendale. He has contributed to climate modelling and climate impact programs since 1989. Ian has over 70 journal publications and was a major contributor to the CSIRO/BoM climate projections for Australia of 2007 and 2015.


May 28 2021


8:00 am - 6:00 pm