Machine learning assisted delineation and measurement of grains in sediment images – the potential of transfer learning

The size of coarse sedimentary particles in fluvial systems is key for quantifying sedimentation and transport conditions in both active and ancient fluvial systems. In particular, the grain size of the bed load in gravel-bed rivers allows inferring information on sediment entrainment or deposition mechanisms, and on the hydraulic conditions controlling them. However, collecting data on such coarse-grained sedimentary particles traditionally involved time-intensive and costly fieldwork, leading to the development of image-based techniques for grain size estimation over the last two decades. Nevertheless, despite much progress and the recent deployment of deep learning methods that were trained on large datasets (i.e., > 100 000 manually annotated grains; Lang et al., 2021; Chen et al., 2022), image-based grain size data is limited to single percentile values, often due to a systematic bias and/or a low accuracy (e.g., Chardon et al., 2020; Mair et al., 2022). Specifically, the core challenge for most existing methods is the accurate segmentation, i.e., the identification and delineation of individual grains, across distinctly different types of data.

Here we present a new approach designated to improve the segmentation in images, which is based on the capability of transfer learning of deep learning models. Such a strategy allows us to re-train existing models for new tasks that are similar to their original purpose. In particular, we use the python-based and open-source tool cellpose (Stringer et al., 2021), which is a state-of-the-art machine-learning model based on neural networks and designed to segment cells in biomedical images. We retrained such a cellpose model on several image datasets of fluvial gravel. The rationale for our approach is based on an inferred geometric similarity between cell nuclei and rock pebbles. Our re-trained models outperform existing methods designed for the segmentation of fluvial pebbles in all datasets, despite an order of magnitude smaller number of training data than currently used in machine learning models. Furthermore, our results show that models trained on specialized datasets for a specific sediment setting yield significantly better results than models trained on larger and more diverse datasets. Fortunately, the model’s flexibility, accessibility, and ability for easy and fast training (Pachitariu and Stringer, 2022) enable the training of task- or image-type-specific models. To facilitate the segmentation power of such models, we built an open-source software tool, ImageGrains. This tool allows for easy use of the models we trained, or of other custom models, as well as streamlined grain size and shape measurements. This allows for fast and nearly automated measurements of large numbers of coarse sedimentary particles with high precision and across vastly different image settings.

References

Chardon, V., et al., 2022: River Res. Appl., 38, 358–367, https://doi.org/10.1002/rra.3910.

Chen, X., et al., 2022: Earth Surf. Dyn., 10, 349–366, https://doi.org/10.5194/esurf-10-349-2022.

Lang, N., et al. 2021: Hydrol. Earth Syst. Sci., 25, 2567–2597, https://doi.org/10.5194/hess-25-2567-2021.

Mair, D., et al. 2022: Earth Surf. Dyn., 10, 953–973, https://doi.org/10.5194/esurf-10-953-2022.

Pachitariu, M. and Stringer, C. 2022: Nat. Methods, 19, 1634–1641, https://doi.org/10.1038/s41592-022-01663-4.

Stringer, C., et al. 2021: Nat. Methods, 18, 100–106, https://doi.org/10.1038/s41592-020-01018-x.