Deep Neural Networks for the reconstruction of near-surface hourly temperature

Near-surface meteorological variables can be reconstructed using various methods, including geostatistical approaches like Kriging or techniques based on inverse problem theory, such as Optimal Interpolation. In recent years, machine learning methods have also been applied to this type of problem, including the reconstruction of meteorological fields near the surface.

Machine learning methods require large training datasets. At MET Norway, we have been collecting hourly temperature observations from personal weather stations (PWS) managed by private individuals. These data have been used operationally since 2018. Compared to conventional observation networks, PWS data increase the number of available training samples by a factor of 50 or more. However, their use also introduces specific challenges, such as data quality and representativeness. We will describe the steps taken to make use of PWS data in an effective way.

We apply a neural network model to estimate the hourly temperature at a location based on the 20 nearest observations. The aim is to estimate both the expected value and its associated uncertainty. The neural network is based on an approach developed for post-processing numerical weather prediction output, which provides probabilistic forecasts in the form of quantile functions. These functions are represented as linear combinations of Bernstein basis polynomials, with the coefficients predicted by the network.

The implementation is done in Python using the JAX library and training is run on a single Nvidia A30 GPU with 24 GB RAM. Further details about the training parameters used will be provided during the presentation.

The geographical domain covers Fennoscandia and the Baltic states. We use hourly temperature data from January 2020 to July 2024 and train a separate model for each month. The number of training samples per month ranges from approximately 145 million to 180 million.

The work is still at an early stage. We are currently addressing questions such as: What is the accuracy and precision of the predicted temperature values? Are the uncertainty estimates reliable? Does including temporal and spatial context in the training data improve the results?