Understanding the relationship between clouds and surface downward radiation forecast errors with Unsupervised Deep Learning

Cloudiness is a difficult parameter to forecast and has improved relatively little over the last decade in numerical weather prediction models as the EMCWF IFS. However, surface downward solar radiation forecast (ssrd) errors are becoming more important with higher penetration of photovoltaics in Europe as forecasts errors induce power imbalances that might lead to high balancing costs. This study continues recent approaches to better understand clouds using satellite images with Deep Learning. Unlike other studies which focus on shallow trade wind cumulus clouds over the ocean, this study investigates the European land area. To better understand the clouds, we use the daily MODIS optical cloud thickness product which shows both water and ice phase of the cloud. This allows to consider both cloud structure and cloud formation during learning. It is also much easier to distinguish between snow and cloud in contrast to using visible bands. Methodologically, it uses the Unsupervised Learning approach tile2vec to derive a lower dimensional representation of the clouds. Three cloud regions with two similar neighboring tiles and one tile from a different time and location are sampled to learn lower-rank embeddings. In contrast to the initial tile2vec implementation, this study does not sample arbitrarily distant tiles but uses the fractal dimension of the clouds in a pseudo-random sampling fashion to improve model learning.

The usefulness of the cloud segments is shown by applying them in a case study to investigate statistical properties of ssrd forecast errors over Europe which are derived from hourly ECMWF IFS forecasts and ERA5 reanalysis data. This study shows how Unsupervised Learning has high potential despite its relatively low usage compared to Supervised Learning in academia. It further shows, how the generated land cloud product can be used to better characterize ssrd forecast errors over Europe.