EMS Annual Meeting Abstracts
Vol. 20, EMS2023-602, 2023, updated on 06 Jul 2023
EMS Annual Meeting 2023
© Author(s) 2023. This work is distributed under
the Creative Commons Attribution 4.0 License.

Estimating location errors in precipitation forecasts with the Wasserstein and Attribution distances

Llorenç Lledó1, Gregor Skok2, and Thomas Haiden3
Llorenç Lledó et al.
  • 1ECMWF, Bonn, Germany (llorenc.lledo@ecmwf.int)
  • 2Faculty of Mathematics and Physics, University of Ljubljana, Ljubljana, Slovenia
  • 3ECMWF, Reading, UK

Due to the strongly fluctuating nature of precipitation both in space and time, there is a need for high-resolution forecasts in order to provide accurate information for user applications. However, as we transition to high-resolution forecasts, the predictability limits imposed by the physics of convective motions render traditional verification techniques less effective for measuring forecast quality. While high-resolution models might be able to realistically simulate convective motions and their associated precipitation, the exact location of the updrafts and the surface precipitation cannot be determined precisely. This poses a problem with classical point-to-point verification techniques such as root mean squared error (RMSE), because any displacement of the precipitation in the forecasts will result in a double penalty. There are three specific problems with RMSE when there are location errors: a) forecasts that have less variability than observations score better than misplaced but realistic forecasts in those cases, favouring unrealistic solutions; b) low-resolution forecasts can score better than more realistic high-resolution ones; and c) forecasts where an observed feature is misplaced but nearby receive the same score than forecasts that are misplaced and farther away.

Measuring the location error, i.e. the distance between precipitation spots in forecast and observation fields is an intuitive way to address the third issue. However, to measure the displacements, one needs to have an assignment between features in the forecasts and the observations. The Wasserstein distance, defined as the minimum displacement over all possible assignments, is a theoretical way forward. However, computing it is prohibitively expensive. Fortunately, there has been growing interest among the machine learning community in utilizing Wasserstein distances to circumvent too literal comparisons. As a result, new algorithms have been developed that can approximate Wasserstein distances and scale linearly with respect to the number of points to be assigned. In this presentation, we demonstrate the practical application of two fast approximate algorithms, namely the Flowtree and the Attribution distance methods, for measuring location errors. Both methods are very flexible, allowing the computation of location errors on gridded or unstructured datasets, even on the spheric geometry of the Earth. We showcase the utilization of these novel verification metrics in specific use cases with ECMWF forecasts to highlight their strengths and weaknesses.

How to cite: Lledó, L., Skok, G., and Haiden, T.: Estimating location errors in precipitation forecasts with the Wasserstein and Attribution distances, EMS Annual Meeting 2023, Bratislava, Slovakia, 4–8 Sep 2023, EMS2023-602, https://doi.org/10.5194/ems2023-602, 2023.