Utilizing a residual neural network ensemble for ground-based cloud classifications

Clouds of any kind play a substantial role in a wide variety of atmospheric processes. They are directly linked to the formation of precipitation, and significantly affect the atmospheric energy budget via radiative effects and latent heat. Hence, knowledge of currently occurring cloud types allows the observer to draw conclusions about the short-term evolution of the state of the atmosphere and hence also the weather. However, the number of operational cloud observations is rather decreasing than increasing due to high monetary and personnel expenses.

To show that automatized methods can be used to close this emerging gap, we trained an ensemble of 10 identically initialized residual neural network architectures from scratch to classify clouds from ground-based RGB pictures into 30 different classes. 4 pictures are used as input at each instance, so that the whole visible sky is covered. Operational manual cloud classification reports at the nearby station Vienna HoheWarte are used as ground truth, where for each instance up to 3 out of 30 categories are reported according to the state-of-the-art WMO cloud classification scheme for operational synoptic observations, making this a multi-label classification task. To the best of our knowledge we are the first to automatically classify clouds based on this elaborate classification scheme. Weutilize class specific resampling to reduce prediction biases because of highly imbalanced observation frequencies among categories. Results show that precision and recall scores are high in all classes, although in initially small classes overfitting is supposed to be the reason for exceptionally high accuracy. Still, every member of our ensemble outperforms both random and climatological predictions in each class. A substantial ratio of wrongly assigned pictures is made up by false negative predictions, where the model recognized the correct class in the input but the assigned probability was too small. For further improvement of current results, we aim to include also satellite images and measurement data, e.g. cloud base height, into our classifier. Though additional data is not supposed to solve overfitting issues, we expect to reduce the number of false negative and false positive predictions substantially.

Autonomy and output consistency are the main advantages of such a trained classifier, hence we consider operational cloud monitoring as main application. Either for consistent cloud class observations or to observe the current state of the weather and its short time evolution with high temporal resolution, e.g. in proximity of solar power plants. There, upcoming clouds can substantially change the possible energy output, which leads to the necessity of taking precautions.