Post hoc visual interpretation of convolutional neural network model for earthquake detection using feature maps, optimal solutions, and relevance values

In recent years, it became clear that the seismological community is adopting deep learning (DL) models for many diverse tasks such as problems of discrimination and classification of seismic events, earthquake detection and phase picking, generalised phase detection, earthquake early warning etc. Many models that have been developed and tested reach quite high accuracy values. However, it has been showed that their performances depend on the DL architecture, on the training hyperparameters and on the datasets that are used for training. To help the community to understand how final results and a model’s performance depend on each of these different aspects, we propose implementing some techniques that target the black-box nature of DL models. In this study we applied three visualisation technique to a convolutional neural network (CNN) classification model for the earthquake detection. The implemented techniques are: feature map visualisation, backward optimisation and layer-wise relevance propagation methods. These can help us answer questions such as: How is an earthquake represented within a CNN model? What is the optimal earthquake signal according to a CNN? Which parts of the earthquake signal are more relevant for the model to correctly classify an earthquake sample? These findings can help us understand why the model might fail, how to build better model architectures, but also whether there is a physical meaning embedded in a model from training samples. The CNN used in this study had been trained for single-station detection, where an input sample is a 25 seconds long three-component waveform. The model outputs a binary target: an earthquake (positive) and a noise (negative) class. Following our two output classes, our training database contains a balanced number of samples from both classes. The positive samples span a wide range of earthquakes, from local to teleseismic, with a focus on the local and regional ones. Our analysis showed that the CNN model correctly identifies earthquakes within the sample window, while the position of the earthquake in the window is not explicitly given (based on the high relevance values). The model handles well earthquakes of different distance and magnitude values, without having any physical information about them during the training process. Thus, the model constructs highly abstract latent space where different earthquakes can eventually fit (can be shown by visualising feature maps). We also notice that having non-filtered training samples with low signal to noise ratio does not disrupt the model to generate distinct feature maps, which is crucial for the successful earthquake detection process. Finally, interpretation techniques proved to be useful for having an insight of how the CNN model treats input samples, which is beneficial for understanding whether the architecture is well designed for this task.