Evaluating the Capabilities of Backbones for Scanning Electron Microscopy Images of Opalinus Clay

The characterization of pores in materials such as Opalinus clay is crucial for understanding the physical properties, including permeability and strength, which are important for the safe disposal of radioactive waste. Scanning electron microscopy (SEM) is a technique that allows high-resolution imaging of these pores at the nanoscale. However, the analysis of SEM images can be challenging due to the resolution limits of nanoscale pores and their manual segmentation. In the development of automatic segmentation methods, approaches of supervised or unsupervised machine learning (ML) and deep learning (DL) methods are increasingly applied. The main advantage of these methods is to achieve fast and more consistent results that do not rely on user input.

An essential component in DL is the so-called backbones, which can learn object features that are necessary for object recognition. In image processing, objects are recognized through groups of specific features that allow an unambiguous identification. Pre-trained backbones, which have been trained on large datasets such as ImageNet containing millions of everyday images, possess a wide range of features that are useful during image processing tasks. However, specialized applications, such as the automatic analysis of microscope images using DL may require features that differ from those of pre-trained backbones. The limited availability of SEM images makes it difficult to effectively train DL models, as these models typically require a large amount of data to learn new features. In these cases, ML methods may perform better due to their ability to use carefully selected, expert-defined features [Maitre et al., 2019].

In this study, the training behavior of eight different DL backbones was examined using a dataset of 2000 SEM images showing both the background and pores of an Opalinus clay sample. The backbones studied included VGG16, VGG19, ResNet50, Desenet, Xception, and Mobilenet. To train these models with the relatively small amount of training data available, a transfer learning technique was applied. We analyzed gradient-weighted class activation mappings (grad-CAM) [Selvaraju et al.,2019] during the learning process to obtain a general sense of the behavior of the different backbones. Through analysis of the model's adaptation efforts, the present study demonstrates which pre-trained backbones show good training behavior on SEM images and provides an estimation of the amount of data needed for effective training.

 

References

[Maitre et al., 2019] Maitre, J., Bouchard, K., and Bédard, L. P. (2019). Mineral grains recognition using computer vision and machine learning. Computers & Geosciences, 130:84–93.

[Selvaraju et al., 2019] Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., and Batra, D. (2019). Grad-cam: Visual explanations from deep networks via gradient-based localization.