Optimizing the performance of EQTransformer by parameter tuning and comparison to handpicked benchmark datasets in a subduction setting

As the number of seismic stations and experiments greatly increases due to ever greater availability of instrumentation, automated data processing becomes more and more necessary and important. Machine Learning (ML)methods are becoming widespread in seismology, with programsthat identify signals and patterns or extract features that can eventually improve our understanding of ongoing physical processes.

We here focus on critically evaluating the performance of a popular machine-learning-based seismic event detection and arrival time picking program, EQ-Transformer, forseismic data from the IPOC deployment in Northern Chile, using handpicked benchmark datasets.By using the open-source framework SeisBench, we can test the effect of using different pre-trained models as well as modify critical parameters such as probability thresholds.

By performing this evaluation, we want to decide whether it is necessary to retrain EQTransformer with local data, or if its performance with one of the supplied pre-trained sets is sufficiently good for our purposes.

We prepared alarge handpicked benchmark dataset for Northern Chile, which we use to find the optimal configuration of EQTransformer. For this benchmark dataset, we select a total of 35 days distributed throughout the 15 years covered by the IPOC deployment. Our goal was to pick all of the many small events in the dataset, even when they are only visible at one or two stations, with high accuracy. We found around three hundred events per day, which highlights the very high seismic activity of the region. We then ran EQTransformer for the same days, using a wide range of parameter choices and pre-trained models.

We need to find if our data is similar to seisbench benchmark dataset or if we should use our data to calibrate the EQTransformer for picking in subduction zones.

We use our handpicked benchmark dataset to evaluate the detection rate (missed events, false detections) as well as the picking accuracy (residuals to handpicks) achieved with EQTransformerin the various tested configurations. We present results of choosing different event detection thresholds, showingtrue positive rate vs. false positive rate plots in order to find optimal thresholds, and evaluate the pick accuracy of obtained arrival time picks by comparing to the handpicked benchmark. This comparison of picking times (P & S) is visualized with residual histograms. Lastly,we also show examples for a visual comparison of picks fromEQTransformer with manual picks.

The present study is the first step towards the design of an automated workflow that comprises event detection and phase picking, phase association and event location and will be used to evaluate subduction zone microseismicity in different locations.