Improved ANN-based earthquake prediction system with reference model engaged in the evaluation metrics

Artificial Neural Networks (ANNs) are well known for their ability to find hidden patterns in data. This technique has also been widely used for predicting the time, location, and magnitude of future earthquakes. Using various data and neural networks, previous works claimed that their models are effective for predicting earthquakes. However, these scores provided by the evaluation metrics with poor reference models, which are non-professional in statistical seismology, are not robust. In this work, we first take the Nighttime Light Map (NLM) as the input of Long-short Term Memory (LSTM) networks to predict the earthquakes with M>=5.0 for the whole Chinese Mainland, and NLM records the lumens of nighttime artificial light, and it is retrieved from the nighttime satellite imagery. The NLM is not physically related to earthquakes; however, the scores provided by Receiver Operating Characteristics curve, Precision-Recall plot, and Molchan diagram with spatial invariant Poisson model indicated that NLM is effective for predicting earthquakes. These results reaffirmed that researchers should be cautious when using these evaluation metrics with poor reference models to evaluate earthquake prediction models. Moreover, the original loss functions of ANNs, such as Cross Entropy (CE), Balanced Cross Entropy (BCE), Focal Loss (FL), and Focal Loss alpha (FL-alpha), contain no knowledge about seismology. To differentiate the hard and easy examples of earthquake prediction models during the training steps of ANNs, the punishment of CE, BCE, FL, and FL-alpha for positive examples will be further weighted by P₀ and the punishment for negative examples will be weighted by P₁, where P₁/P₀ is the prior probability provided by the reference model that at least one or no earthquakes will occur for the given example and P₁+P₀=1. The reference models are supposed to be as close to the real spatial-temporal distribution of earthquakes as possible, and the spatial variable Poisson (SVP) model is the simplest version which is also friendly to data mining experts. In this work, we choose the SVP as the reference model to revise these previous loss functions and take the estimated cumulative earthquake energy in the time-space unit (1 degree*1 degree*10 days) as the input of the LSTM to predict the earthquakes with M>=5.0 in the whole Chinese Mainland, and we use the Molchan diagram (SVP) and Area Skill Score (ASS) to evaluate the performance of these models. Results show that the majority of models (134 out of 144) trained by original loss functions are ineffective for predicting earthquakes; however, the scores of models trained using the revised loss functions have been obviously improved, and 83 out of 144 models are proved to be better than SVP in predicting earthquakes. Our results indicate that designing a more complex structure for ANN and neuron is not the only way to improve the performance of ANNs for predicting earthquakes, and how combining the professional knowledge of data mining experts and seismologists deserves more attention for the future development of ANN-based earthquake prediction models.