Training CNNs for cloud classification from RGB pictures

Clouds play an important role in atmospheric processes like radiation and moisture transport and knowing the currently occurring cloud type can give the observer insights into the dynamics going on in the atmosphere. However, the number of human cloud observers is rather decreasing than increasing and thus efficient methods for automated cloud classification are sought as a way to continuously gather such information. Moreover, such a method is also less subjective than any human observer. Machine Learning methods and especially Convolutional Neural Networks (CNNs) have shown exceptionally good results in a broad range of image classification tasks. Despite the fact that machine learning has already been used to classify clouds from satellite data during the last few decades, the field of cloud classification from conventional RGB pictures taken at the Earth's surface is rather untouched. Although the WMO's classification scheme is defined based on visual properties of clouds only visible from below their undersurface. Hence, in this work CNNs are trained to discriminate between up to 30 cloud types from conventional RGB pictures. The used data set consists of all sky panorama images taken by the cloud observation system of the Department of Meteorology and Geophysics at the University of Vienna during the period 2016-2019 as well as from 2022 onwards. Ground truth labels are taken from hourly operational observations at the station Vienna Hohe Warte. The fact that more than one cloud type can occur at once makes this task a multi class multi label classification problem and even harder to solve. However, once trained properly the resulting algorithm can be used together with an ordinary camera to classify clouds with a high temporal resolution. Possible applications may be, e.g. model verification or to efficiently monitor the current status of the weather as well as its short-time evolution. First results will be shown.