Deep Learning for image based weather estimation: a focus on the snow

The road traffic is highly sensitive to weather conditions. Accumulation of snow on the road can cause important safety problems. But road conditions monitoring is as hard as critical: in mid-latitude countries, on the one hand, the spatial variability of snowfall is high and on the other hand, accurate characterization of snow accumulation mainly relies on costly sensors.

In recent decades, webcams have become ubiquitous along the road network. The quality of these webcams is variable but even low-resolution images capture information about the extent and the thickness of the snow layer. Their images are also currently used by forecasters to refine their analysis. The automatic extraction of relevant meteorological information is hence very useful.

Recently, generic and efficient computer vision methods have emerged. Their application to image-based weather estimation has become an attractive field of research. However, the scope of existing work is generally limited to high-resolution images from one or a few cameras.

In this study, we show that for a moderate effort of labelling, recent Machine Learning approaches allow us to predict quantitative indices of the snow depth for a large variety of webcam settings and illumination.

Our approach is based on two datasets. The smallest one contains about 2.000 images coming from ten webcams that were set up near sensors devoted to snow depth measurements.

The largest one contains 20,000 images coming from 200 cameras of the AMOS dataset. Meteorological standard rules of human observation and the specifics of the webcams have been taken into account to manually label each image. These labels are not only about the thickness and the extent of the snow layer but also describe the precipitation (rain or snow, presence of streaks), the optical range and the foreground noise. Both datasets contain night images (45%) and at least 15% of images corrupted by foreground noise (filth, droplets, and snowflakes on the lens).

 

The labels of the AMOS subset allowed us to train ranking models for snow depth and visibility using a multi-task setting. The models are then calibrated on the smallest dataset. We tested several versions, built from pre-trained CNNs (ResNet152, DenseNet161, and VGG16).

Results are promising with up to 85% accuracy for comparison tasks, but a 10% decrease can be observed when the test webcams have not been used during the training phase.

A case study based on a widespread snow event over the French territory will be presented. We will show the potential of our method through a comparison with operational model forecasts.