A point-cloud deep learning model based on RGB-D images: Application of riverbed grain size survey

The water level and discharge of river are crucial parameters to understand the variance in riverbed scour. The detail behavior of scouring can be studied by the hydraulic simulation. The grain-size distribution of riverbed is also one of crucial parameter for modeling. Thus, how to investigate the grain-size of riverbed efficiently and swiftly is the urgent issue. However, the conventional measurement methods including Wolman counts (particles sampled at a fixed interval) which are a long and laborious task cannot survey the grain-size efficiently in the large area. In recent years, with an advantage of image segmentation and recognition has been applied to the investigation of grain-size, for example, capturing images through UAV and generating orthoimage is one of commonly used image technique. Although above the method can investigate the grain-size in the large area, it does not provide the information in the field immediately. Hence, a recent study developed the low-cost portable scanner to obtain the information of grain-size distribution in the field. However, the calibrating parameters of camera (e.g., height camera capture) are necessary before survey, and the uncertainties in calculation of image resolution will significantly affect the accuracy of grain-size analysis. Therefore, this study provides the additional algorithm to analyze the grain-size by using RGB-D image as inputs. The application of RGB-D can be categorized into two-dimensional (2D) and three-dimensional (3D) spaces. In a case of 2D, it integrates depth information with traditional RGB image processing to separate the grain-size of riverbed from the background (e.g., bottomland). Furthermore, depth information is also applied for grain-size edge detection. In a case of 3D, the collected RGB-D image information is transformed into point cloud data, then extract 3D features of grain particle by Deep learning, specifically PointNet. Our study demonstrates that clustering of 3D features can achieve the automatic identification of particle. The grain-size of particle can also be estimated by fitting 3D ellipsoid geometry. In the end, results show the grain-size distribution curves with the RGB、RGB-D、PointNet recognition, and compare with the true observations. 3D image information provides the cloud points of grain object, leading the possibility of estimating the 3D geometric morphology of the object. Our study successfully overcomes the limitations of conventional RGB-based process, which could only capture size and shape information in 2D planar. RGB-D-based image recognition, is an innovative technique for the hydraulic problem, not only advances survey efficiency but also addresses the intricate steps required for field investigations.

 

Key words: Riverbed grain size, RGB-D image, Point cloud, Deep Learning