The Impact of Probability Thresholds and Model Training on Phase Picking Performance

The transition from manual to deep-learning automated seismic phase picking has revolutionized seismology applications such as seismic catalog building and fault structures analysis. However, the reliability of these AI-driven catalogs is often hindered by a "black-box" approach to model selection and decision thresholds. While deep learning models like PhaseNet (Zhu and Beroza, 2019) offer unprecedented efficiency, their performance is sensitive to the data they were trained on and to the probability thresholds used to define a "phase pick". 

In this work, we present a comparative study focused on the Amatrice–Visso–Norcia 2016-2017 seismic sequence in Central Italy. We investigate the influence of the training models and the threshold variation on the phase picking detections.  

In particular, we compare the performance of the default PhaseNet model (STEAD) against i) a model trained on the AQ2009 dataset (specific for the Central Apennines, from Bagagli et al., 2023) and ii) a model obtained through Transfer Learning on STEAD fine-tuned with the INSTANCE model (Michelini et al., 2021) via the SeisBench platform (Woollam et al., 2022). We also analyze how varying the confidence threshold (from 0.1 to 0.9) affects the final catalog's completeness and precision.

Preliminary results show that regional training significantly outperforms default models in specific noise conditions and that the optimal threshold is influenced by station geometry and signal-to-noise ratios. By providing a statistical framework for automated threshold calibration, this study offers a roadmap for more objective and reproducible signal detection, applicable not only to seismology but to any domain dealing with continuous time-series classification.