A new approach to subseasonal multi-model forecasting: Online prediction with expert advice

The benefits of multi-model combinations in climate forecasting have been previously introduced and described for different temporal scales (e.g., Siebert and Stephenson 2019, DelSole 2007, Sansom et al. 2013). Most typical combination methodologies involve weighting strategies that assign each model a constant factor, either uniformly or through a skill assessment. Given that the skill of the models can vary at different timescales, and for multiple reasons (for example, seasonally varying skill, or due to changes in the forecasting system), the fact that these weights remain constant introduces limitations.  

Within the realm of Machine Learning, a family of algorithms have been developed to perform ‘online prediction with expert advice’ (Cesa-Bianchi et al. 2006). These methods consider a set of weighted ‘experts’ (usually uniformly weighted at the start of the process) to produce subsequent predictions in which the combination or `mixture’ is updated to optimize a loss or skill function.  

These online forecasting methods potentially have several advantages for their use in climate prediction: 

• The fact that the expert combination is updated in every forecast step allows the system to adjust in certain conditions (e.g., the ones mentioned above) to preserve skill; 
• Since a different combination can be easily obtained for different quantiles of the predictand distribution, a robust system can be trained that maximizes skill for its full range. 
• The risk of including incompetent or counterproductive experts is minimized by the fact that the mixture is able to adapt and discard them (or assign them minimal weights). 

Another potential application of these online prediction methods could be on the design of ‘seamless’ forecasting systems in the sub-seasonal to seasonal sense, which is of interest to several research projects such as S2S4E (https://s2s4e.eu/). For example, the system could be trained with a set of experts that include subsequent launches of a sub-seasonal forecast as well as prior launches of a seasonal forecast. If at any point there is useful information arising from the longer lead time seasonal forecast, the mixture would assign higher weights to it. 

A set of these online prediction methods have been tested within the S2S4E project and compared to more typical multi-model combination techniques to assess their usefulness for the prediction of country-level energy demand, and potentially other variables. Results show that these innovative methods exhibit significant skill improvements (higher than 5%) with respect to more standard techniques and to individual forecasting systems for lead weeks up to 4.