FEEDS: Validation of the Framework for Evaluation of Early Detection Systems

Monitoring volcanic activity is a complex task, given the intricate nature of volcanic processes and the diverse eruptive styles exhibited by different volcanoes. Early Detection (ED) systems have emerged as indispensable tools for mitigating potential risks associated with volcanic eruptions. The effectiveness of these systems is contingent upon their ability to provide timely and accurate alerts, as false alarms or missed warnings can lead to economic repercussions and pose risks to infrastructure and human safety. Evaluating the reliability of the ED systems may be paramount not only for effective hazard mitigation but also for facilitating the implementation and optimization of an ED model. However, developing an ED model is a challenging and labor-intensive endeavor, also requiring a deep understanding of advanced techniques and a meticulous calibration of various parameters. 

In response to these challenges, we present the Framework for Evaluation of Early Detection Systems (FEEDS). FEEDS is a comprehensive Python-based package designed to automatically assess the generalization capability of generic ED systems through cross-validation. The framework introduces a generic class representing the ED model identified solely through data, enabling a systematic assessment based on essential predictive parameters, including True Positive Rate, False Discovery Rate, prediction time, and Fraction of Time in Alarm, by performing a simulation. 

To validate the effectiveness of FEEDS, we utilized tiltmeter and strainmeter data recorded at Stromboli volcano between 2007 and 2019. These datasets, managed by Istituto Nazionale di Geofisica e Vulcanologia and Università di Firenze, were employed to implement FEEDS with a customized model for the early detection of the paroxysmal activity affecting the volcano during the period of the study, demonstrating the practical applicability and reliability of this framework in real-world volcanic monitoring scenarios.

FEEDS may represent a valuable contribution to the ongoing efforts to enhance ED systems and their application in mitigating volcanic hazards. The development of a robust framework that automates the standard evaluation process not only streamlines system implementation but also reduces user efforts and establishes a common ground for assessing the reliability and performance of different ED models, contributing significantly to the advancement of volcanic monitoring capabilities.