Learnt variability as a tool for climate prediction and predictability

Whether it is turbulence fluid flows or climate variability, there is a big gap between our ability to develop understanding of underlying phenomena/processes and our ability to produce skillful predictions. We focus on near-term prediction of climate as an example. In this context, the state-of-the-art is such that we are able to predict how 30-year global averages of surface temperature will change, but we are unable to predict shorter time scale regional changes.  We investigate a range of deep learning approaches to the problem ranging from reservoir computing to deep convolutional Long Short-Term Memory network architectures. The best performing architectures are seen to be capable of predicting an Earth System Model’s leading modes of global temperature variability with prediction lead times of up to a year. This approach is proposed as a useful practical tool for climate prediction. Further insight into the difficulty of the prediction problem is provided by considering the Lorenz '63 model: Long prediction horizons seen when the system is fully observed is seen to be progressively degraded as the system is less thoroughly observed, while noting the difficulty of fully observing the earth system