Tree species classification by using computer vision and deep learning techniques for the analysis of drone images of mixed forests in Japan

Forests become more important in times of changing climate, increasing demand of renewable energies and natural resources, as well as the high demand of information for economical and management issues. Several previous studies were carried out in the field of forest plantations but there is still a gap in knowledge when it comes to natural mixed forests, which are ecological complex due to varying distributions and interaction of different species. The applicability of Unmanned Aerial Vehicles (UAVs) for forest applications by using image analysis became a common tool because it is cost-efficient, time-saving and usable on a large-scale. Additionally, technologies like Deep Learning (DL) fasten the proceeding of a high number of images. Deep learning is a relatively new tool in forest applications and especially in the case of natural dense mixed forests in Japan. Our approach is to introduce the DL-based ResNet50 network for automatic tree species classification and segmentation, which uses transfer learning to reduce the amount of required data. A comparison between the ResNet50 algorithm and the common UNet algorithm, as well as a quantitative analysis of model setups are presented in this study. Furthermore, the data were analysed regarding difficulties and opportunities. We showed the outperformance of UNet with a DICE coefficient of 0.6667 for deciduous trees and 0.892 for evergreen trees, while ResNet 50 was reaching 0.733 and 0.855. A refinement of the segmentation was performed by the watershed algorithm increasing the DICE coefficient to values of up to 0.777 and 0.873. The results of the transfer learning analysis confirmed the increasing accuracy by adding image classification data basis for the model training. We were able to reduce the number of images required for the application. Therefore, the study showed the applicability and effectiveness of those techniques for classification approaches. Furthermore, we were able to reduce the training time by 16 times for the ResNet 50 performance and by 3.6 times with the watershed approach in comparison to the UNet algorithm. To the best of our knowledge this is the first study using deep learning applications for forestry research in Japan and the first study dealing with images of natural dense mixed forests.