UAV-based training for fully fuzzy classification of Sentinel-2 fluvial scenes

In current fluvial remote sensing approaches, there exists a certain dichotomy between the analysis of small channels at local scales which is generally done with airborne data and the analysis of entire basins at regional and national scales with satellite data. One possible solution to this challenge is to use low-altitude imagery from low-cost UAVs to provide sub-metric scale class information which can then be used to train fuzzy classification models for entire Sentinel 2 tiles. The fuzzy classification approach can allow for sub-pixel information and when extended to entire Sentinel 2 tiles, the method therefore develops information at a resolution of less than 10 meters (the best spatial resolution of Sentinel 2 bands) at regional scales. In this contribution, we present such a method where UAV imagery is used as the training data for the fully fuzzy classification of Sentinel 2 imagery. We partition the fluvial corridor in three simple classes: water, dry sediment and vegetation.  Then we manually classify the local UAV imagery into highly accurate class rasters. In order to augment the value of the Sentinel 2 data, we use an established super-resolution method that delivers 10 meter spatial resolution across all 11 Sentinel 2 bands. We then use the sub-metric UAV classifications as training data for the 10 meter super-resolved Sentinel 2 imagery and we train fuzzy classification models using random forests, dense neural networks and convolutional neural networks (CNN). We find that CNN architectures perform best and can predict class membership within a pixel of a new Sentinel 2 tile not seen in the training phase with a mean error of 0% and an RMS error of 18%. Crisp class predictions derived from the fuzzy models range in accuracy from 88% to 99%, even in the case of tiles never seen in the training phase. With this approach, it is now possible to deploy a low-cost UAV in order to train a transferable CNN model that can predict fuzzy classes at very large scales from freely available Sentinel 2 imagery. This approach can therefore serve as the basis for multi temporal classification and change detection of the Sentinel 2 archives.