How to evaluate rainfall estimation performance? - A discussion of metrics, thresholds and aggregations for one year of country-wide CML rainfall estimation

In recent years, so-called opportunistic sensors for measuring rainfall, are attracting more notice due to their broad availability and low financial effort for the scientific community. These sensors are existing devices or infrastructure, which were not intentionally built to measure rainfall, but can deliver rainfall information. One example of such an opportunistic measurement system are Commercial Microwave Links (CMLs), which provide part of the backbone of modern mobile communication. CMLs can deliver path-averaged rainfall information through the relation between rainfall and attenuation along their paths. Before such an opportunistic data source can be used, either as an individual or a merged data product, its performance compared to other rainfall products must be evaluated.

We discuss the selection of performance metrics, spatial and temporal aggregation and rainfall thresholds for the comparison between a German-wide CML network and a gauge-adjusted radar product provided by the German Weather Service. The CML data set consists of nearly 4000 CMLs with minutely readings from which we will present a year of data. 

First, we show the influence of the temporal aggregation on the comparability. With higher resolution, the impact due to small temporal deviations increases. Second, CMLs represent path-averaged rainfall information, while the radar product is gridded. We discuss the choice whether the comparison should be performed on the point, line or grid scale. This choice depends on the desired future applications which already should be considered when selection evaluation tools. Third, the decision to exclude rain rates below a certain threshold or the calculation of performance metrics for certain intervals gives us a more detailed insight in the behavior of both rainfall data sets.