High accuracy doesn’t prove that a deep learning model is accurate: a case study from automatic rock classification of thin section photomicrographs

As deep learning (DL) is gathering remarkable attention for its capacity to achieve accurate predictions in various fields, enormous applications of DL in geosciences also emerged. Most studies focus on the high accuracy of DL models by model selections and hyperparameter tuning. However, the interpretability of DL models, which can be loosely defined as comprehending what a model did, is also important but comparatively less discussed. To this end, we select thin section photomicrographs of five types of sedimentary rocks, including quartz arenite, feldspathic arenite, lithic arenite, dolomite, and oolitic packstone. The distinguishing features of these rocks are their characteristic framework grains. For example, the oolitic packstone contains rounded or oval ooids. A regular classification model using ResNet-50 is trained by these photomicrographs, which is assumed as accurate because its accuracy reaches 0.97. However, this regular DL model makes their classifications based on the cracks, cements, or even scale bars in the photomicrographs, and these features are incapable of distinguishing sedimentary rocks in real works. To rectify the models’ focus, we propose an attention-based dual network incorporating the microphotographs' global (the whole photomicrographs) and local features (the distinguishing framework grains). The proposed model has not only high accuracy (0.99) but also presents interpretable feature extractions. Our study indicates that high accuracy should not be the only metric of DL models, interpretability and models incorporating geological information require more attention.