Basis Functions Representation for Deep Learning Models

Machine learning has seen widespread adoption across the geosciences. In particular, deep learning methods have proven effective for producing gridded datasets of climate parameters. Convolutional neural networks are commonly used, but their performance can be limited by the availability and structure of data, especially for sparse or irregularly sampled climate observations. Graph neural networks can handle irregular spatio-temporal data, but their reliance on local interactions restricts their ability to capture large-scale climate processes.

An alternative approach is DeepKriging, originally proposed by Chen et al., which embeds the spatial domain using basis functions centered at knot points across the region of interest. By using these basis functions as input features for a neural network, DeepKriging provides an efficient and flexible representation of both spatial and temporal domains, making it suitable for irregular data and capable of capturing both large-scale and local effects. However, DeepKriging requires basis functions to be manually defined before model training, which can require extensive work from the practitioner to fine-tune the model. This also limits the model’s ability to adapt to varying spatio-temporal patterns.

Here, we propose several extensions to DeepKriging, primarily by allowing basis functions to be updated throughout model training. The resulting model dynamically adapts to diverse spatio-temporal patterns while converging on a basis function representation that is optimal for the current data. We further improve the flexibility of the spatial embedding through a mesh generated via constrained Delaunay triangulation. This approach is applied to multiple climate variables, including precipitation and wind data for Ireland, demonstrating an improved performance compared with the original DeepKriging as well as several state-of-the-art deep learning and geostatistical gridding methods.

Finally, we also show how basis function representations are particularly well suited for datasets with limited availability, such as sparsely sampled climate parameters like relative humidity or soil moisture. This flexibility can be leveraged across a range of machine learning frameworks, including transfer learning with DeepKriging models or more lightweight algorithms such as Random Forests and XGBoost.