How large does a large ensemble need to be?

Initial-condition large ensembles with ensemble sizes ranging from 30 to 100 members have become a commonly used tool to quantify the forced response and internal variability in various components of the climate system. However, there is no consensus on the ideal or even sufficient ensemble size for a large ensemble.

Here, we introduce an objective method to estimate the required ensemble size. This method can be applied to any given application. We demonstrate its use on the examples that represent typical applications of large ensembles: quantifying the forced response, quantifying internal variability, and detecting a forced change in internal variability.

We analyse forced trends in global mean surface temperature, local surface temperature and precipitation in the MPI Grand Ensemble (Maher et al., 2019). We find that 10 ensemble members are sufficient to quantify the forced response in historical surface temperature over the ocean, but more than 50 members are necessary over land at higher latitudes. 

Next, we apply our method to identify the required ensemble size to sample internal variability of surface temperature over central North America and over the Niño 3.4 region. A moderate ensemble size of 10 members is sufficient to quantify variability over North America, while a large ensemble with close to 50 members is necessary for the Niño 3.4 region.

Finally, we use the example of September Arctic sea ice area to investigate forced changes in internal variability. In a strong warming scenario, the variability in sea ice area is increasing because more open water near the coastlines allows for more variability compared to a mostly ice-covered Arctic Ocean (Goosse et al., 2009; Olonscheck and Notz, 2017). We show that at least 5 ensemble members are necessary to detect an increase in sea ice variability in a 1% CO₂ experiment. To also quantify the magnitude of the forced change in variability, more than 50 members are necessary.

These numbers might be highly model dependent. Therefore, the suggested method can also be used with a long control run to estimate the required ensemble size for a model that does not provide a large number of realisations. Therefore, our analysis framework does not only provide valuable information before running a large ensemble, but can also be used to test the robustness of results based on small ensembles or individual realisations.

References
Goosse, H., O. Arzel, C. M. Bitz, A. de Montety, and M. Vancoppenolle (2009), Increased variability of the Arctic summer ice extent in a warmer climate, Geophys. Res. Lett., 36(23), 401–5, doi:10.1029/2009GL040546.

Olonscheck, D., and D. Notz (2017), Consistently Estimating Internal Climate Variability from Climate Model Simulations, J Climate, 30(23), 9555–9573, doi:10.1175/JCLI-D-16-0428.1.

Milinski, S., N. Maher, and D. Olonscheck (2019), How large does a large ensemble need to be? Earth Syst. Dynam. Discuss., 2019, 1–19, doi:10.5194/esd-2019-70.