PATCH-FILL: Multiscale and Univariate Gap-Filling in Remote Sensing Data

Many applications in Earth system sciences require continuous, gap-free data sets. However, remote sensing data in particular are plagued by gaps due to clouds, incomplete coverage, or low-quality flags. Gap-Filling in remote sensing data often requires model architectures that are tailored specifically to underlying dataset characteristics such as scale, resolution or range of values. This limits the transferability to other gap-filling scenarios. Training these models is further hindered by the lack of adequate training samples, as they must be gathered from gap-afflicted data themselves. In this work, we present a spatiotemporal, univariate and multiscale gap-filling method that is independent of any specific dataset. A modular implementation allows for the customization of system parameters, so that the method can be adjusted and applied to various datasets, even outside the Earth Science domain. By employing a patch-wise gap-filling approach, introducing masked loss functions, and providing effective methods for synthetic gap generation, we are able to leverage gap-afflicted datasets and gather large amounts of training samples from them. To demonstrate the flexibility of the system, we perform gap-filling on multiple climatic variables from Earth System Data Cubes (ESDC) (Mahecha et al. 2020) using a 3D CNN architecture, making this the first global-scale gap-filling solution on ESDC. By capturing both spatial and temporal relations, the model is able to generate predictions that are coherent on large scale and across patches, thus demonstrating the potential of the patch-wise gap-filling framework and the use of 3D CNN architectures for spatiotemporal gap-filling tasks.