Generating a pixel-wise annotated training dataset to train ML algorithms for mineral identification in rock thin sections

Machine learning approaches and deep learning-based methods are efficient tools to address problems for which large amounts of observations and data are documented. They have proven excellent performance for many applications in the geosciences and remote sensing area. However, to one of the most fundamental data types in geoscientific studies, mineral thin sections, they have not yet been applied to its full potential. Mineral thin sections contain a treasure of information. It is anticipated that thin section samples can be systematically and quantitatively analyzed with a specifically designed system equipped with ML approaches or deep learning methods such as CNNs. The development of any artificial intelligence application that enables automated image analysis requires consistent and sufficiently large training datasets with ground truth labels. However, a dataset which serves for visual object detection in petrographic thin sections analysis is still missing. We wish to close this data gap by generating a large dataset of pixel-wise annotated microscopic images for thin sections.

The variation of optical features of certain minerals under different settings of a petrographic microscope is closely related to crystallographic characteristics that can be indicative for a mineral. In order to fully capture optical features into digital images, we generated raw data of microscopic images for different rock samples by using virtual petrographic microscopy (ViP), a cutting-edge methodology that is able to automatically scan entire thin sections in Gigapixel resolution under various polarization angle and illumination conditions. We proved that using ViP data will result in better segmentation result compared to single image acquisition.

Image annotation, especially pixel-wise annotation is always a time-consuming and inefficient process. Moreover, it would be particularly challenging when to manually create dense semantic labels for ViP data in view of its size and dimensionality. To address this problem, we proposed a human-computer collaborative annotation pipeline where computers extract image boundaries by splitting images into superpixels, while human-annotators subsequently associate each superpixel manually with a class label with a single mouse click or brush stroke. This frees the human annotator from the burden of painstakingly delineating the exact boundaries of grains by hand and it has the potential to significantly speed up the annotation process.

Instead of providing a discrete representation of images, superpixels are better aligned with region boundaries and largely reduce the image complexity. The use of superpixel segmentation in the annotation pipeline not only significantly reduce the manual workload for human annotators but also provides a significant dataset reduction by reducing the number of image primitives to operate on. In order to find the most suitable algorithms to generate superpixel segmentation, we evaluated state-of-art superpixel algorithms with regard to standard error metrics based on scanned ViP images and corresponding boundary maps traced by hand. We also proposed a novel adaption of the SLIC superpixel extraction algorithm that can cope with the multiple information layers of ViP data. We plan to use these superpixel algorithms in our pipeline to generate open data sets of several types of mineral thin sections for training of ML and DL algorithms.