Assessing the Geological Plausibility of Machine Learning Borehole Interpretations: A Case Study in the Roer Valley Graben

Expert interpretation of borehole data is a critical component of geological modelling, offering essential insights into the spatial distribution of geological units within the subsurface. For large-scale regional mapping efforts, expert interpretation of all available data is impractical due to the sheer volume of boreholes. Therefore, many 3D geological subsurface models rely only on a small portion of all available data. Machine learning (ML) models can be used to automate borehole data interpretation, increasing data density. However, these automated interpretations must adhere to strict spatial and stratigraphical relationships to be consistent with the established geological knowledge of the area. Using a dataset of 1,400 boreholes with expert interpretations from the Roer Valley Graben (Southeast Netherlands), we explore how ML models can be integrated into geological modelling workflows, highlighting the challenge of ensuring compatibility with geological principles and known spatial relationships. We evaluate the model performance using traditional metrics such as accuracy, Cohen's kappa and F1 Score and newly proposed geology-inspired metrics to quantify the ability of Random Forest and Neural Network models to interpret borehole data into lithostratigraphic units while preserving key geological relationships. Our results demonstrate that while many models achieve accuracy values of 75% to 80%, Neural Networks perform significantly better in capturing the expected sequential relationships between geological units, achieving up to 96% of geological transitions between geological units that are plausible, compared to 65% for the best-performing Random Forest model selected based on traditional metrics. This study underscores the need for domain-specific metrics in evaluating model performance and the potential for ML to increase the volume of data incorporated in subsurface models.