A decade-long pseudo-prospective evaluation of UCERF3-ETAS next-day seismicity forecasts

Rigorous evaluation of earthquakes forecasts is a crucial step in understanding and improving the capabilities of earthquakes forecasting models. The UCERF3-ETAS model is currently the most advanced seismicity model combining a long-term seismicity model incorporating hypotheses of fault rupture dynamics and elastic rebounding with an Epidemic-Type Aftershock Sequence (ETAS) model for short-term seismicity. UCERF3-ETAS has also been used on demand for operational earthquake forecasting of important seismic sequences like the 2019 Ridgecrest one. Here, we have evaluated a very large database of UCERF3-ETAS next-day forecasts for California from 1 August 2008 to 31 August 2018. Each next-day forecast is composed of 100,000 synthetic catalogs generated by the model. The synthetic catalogs comprise events with magnitude $M_w \geq 2.5$, start at 00:00:00 UTC, last 24 hours, and include all events prior to midnight in the history for generating the next day’s forecasts. We evaluate the consistency of the model against 17,655 $M_w \geq 2.5$ earthquakes that occurred in California in the period 2007-2018 using the statistical tests for catalogue based forecasts developed by the Collaboratory Study of Earthquake Predictability. We find that the number of events provided by the forecast is generally consistent with the observations, especially during relevant seismic sequences such as the $7.2 M_w$ El-Mayor Cucapah, while swarm type sequences are more challenging. The magnitude distribution is also consistent overall. We also study the spatial evolution of the magnitude distribution to highlight regions where the model is expecting large earthquakes to happen and find that they are coherent with observed seismicity. Finally, we compare UCERF3-ETAS forecasts against fully prospective next-day forecasts produced by 27 different models operated by CSEP during between 2007 and 2018, and collected in a openly available database which constitutes a natural benchmark for the problem. We find that UCERF3-ETAS improves upon older models by providing positive information gains in most periods. The information gain tends to be zero or negative during swarms when UCERF3-ETAS is compared against models having a non-parametric component signaling possible benefits of including one to better describe this type of seismic sequences.