Uncertainties in differentiable parameter learning calibration

Recently, differentiable parameter learning was presented as a deep-learning calibration method that estimates transfer relationships between physical characteristics and calibration parameters (Tsai et al., 2021). Such methods are especially important for large-scale hydrological models, as these transfer relationships allow for estimating consistent and seamless parameter fields in regions without observations. Although parameter learning calibration is shown to efficiently improve the simulation performance, the uncertainties related to this approach are poorly understood.

Our study distinguishes and quantifies the various sources of parameter learning calibration uncertainties with a structured set of calibration experiments using a synthetic dataset generated with a physically based global hydrological model. As the “true” parameters are known in each experiment, our study can distinguish and quantify uncertainties related to: the transfer function form, deep-learning surrogate gradient transfer, deep-learning surrogate performance, geographical bias in available observations, and non-uniqueness.

Our results show that the parameter learning calibration approach is robust under a wide range of possible transfer function forms, gradient transfer through a deep-learning surrogate model, and geographical biases in available observations. In addition, parameter learning calibration is somewhat robust non-uniqueness issues. However, parameter learning is most sensitive to errors in the deep-learning surrogate model's predictions or, conversely, the observations. Moreover, estimated parameters improve the simulation performance even when they are erroneous, indicating better results for the wrong reasons.

Our study highlights the significant potential of deep learning to understand and extrapolate relationships from potentially limited observational data. However, when using the parameter learning calibration approach, care should be taken to select the appropriate parameters and introduce some form of regularization to avoid unrealistic parameterizations.  

References

Tsai, W. P., Feng, D., Pan, M., Beck, H., Lawson, K., Yang, Y., ... & Shen, C. (2021). From calibration to parameter learning: Harnessing the scaling effects of big data in geoscientific modeling. Nature communications, 12(1), 5988.