Fluvial land cover classification by using CSC deep learning method with UAV airborne images

This study investigated a deep learning method for fluvial land cover classification using aerial imagery of the UAV (Unmanned Aerial Vehicles). The deep learning used in this study was the CNN-Supervised Classification (CSC) developed by Carbonneau et al. (Remote Sensing of Environment, 2020). The analysis was based on 51 river sections of RGB orthorectified images taken aerially in 2015-2019 in several river channels of the Kinu River in Japan. They were obtained by applying the SfM (Structure from Motion) processing to the UAV aerial images acquired during field observations. The spatial resolution of the images was approximately 4 cm per pixel. The seven land cover types classified by CSC were water surface, gravel, sand, grass, tree, farmland, and artificial land. The deep learning algorithm CSC in this study was a classification model combining two-stage Convolution Neural Networks (CNNs). The first stage of the CSC classified the input image into 200 x 200-pixel image tiles and created a training dataset to be used in the second stage. Then, the training dataset was used to train a second-stage small-scale CNN (hereafter called mini-CNN) to optimise the model hyper-parameters. Finally, the trained CSC performed pixel-based land cover classification of the RGB orthoimages. In the first stage, this study used an existing CNN architecture, VGG16. The fine-tuning dataset had more than 2,500 images for each land cover class, resulting in a total of 85,800 through data augmentation. The hyper-parameters examined were the learning rate, patch size and the number of frozen layers. The F-measures for the CSC first stage with the optimised parameters were 99.1, 96.9, 92.6, 91.4, 93.6, 95.7 and 96.1% for water surface, gravel, sand, grass, tree, farmland, and artificial land, respectively. Then, the architecture of the mini-CNN, learning rate, patch size, patch number and filter size were optimised for the CSC second stage. The weighted average F-measure for the optimised CSC model was 90.4%. This confirmed that the optimised CSC could reproduce the land cover classes with enough accuracy. The CSC application to the RGB orthorectified images of the Kinu River in Japan showed that the CSC deep learning method could accurately classify temporal changes in fluvial geomorphologies such as gravel beds and sandbars as well as riparian vegetation, including the significant differences before and after the severe floods in 2015 and 2019. Future work would be needed to verify the applicability of the proposed CSC deep learning method to other rivers with different fluvial characteristics.