Challenges of machine learning based post-processing methods for sub-seasonal forecasts

The skill of hydro-meteorological forecasts usually drops to zero at horizons beyond 10 to 14 days and predictions of daily values with a lead-time of more than two weeks deceive an unrealistic reliability and accuracy. However, the advances of ensemble forecast systems, the aggregation of daily information to weeks and the classification of variables (e.g. classifying the runoff in terciles, i.e.  below, above or within “normal” conditions in view of climatology) extend the skill of forecasts up to three weeks. With the help of post-processing methods, the skill can even be further extended up to four weeks.

Machine Learning (ML) based post-processing methods are becoming more and more popular. Several ML methods have been compared for classifying hydrological forecasts into terciles ranging from Multinomial to complex Deep Neural Networks models (DNN).  For the verification of the predicted classes and their associated probabilities the accuracy measure and the cross-entropy (log-loss) have been calculated.  The models have been applied to monthly forecasts for Switzerland divided into 300 catchments and run operationally with a 500mx500m resolution (www.drought.ch).

All the analyzed ML methods showed good results and are able to improve the accuracy and the cross-entropy equally well. The focus of further investigations has been laid on the Gaussian Process (GP) model. This method has the advantage that class probabilities can be calculated directly.  Furthermore, several studies highlighted recently the linkage between the GP and Bayesian DNN. The results of this study confirmed these similarities favoring the less complex GP model.  In order to take the uncertainty of the measurements and simulations into account, a multi-label classification (MLC) method has been introduced. Contrary to classical classification methods, the classes in MLC are not mutually exclusive. Thus, it avoids the sharp assignment to discrete classes, but allows that for example a runoff value measured (possibly with some error) at the border of one class could belong to the neighboring classes at the same time. The MLC method applied with a GP model showed the best results regarding accuracy and cross-entropy for all 300 catchments and significantly improves the skill over the whole forecast horizon of four weeks.