Effects of Spatial Grid Resolution on the Statistical Power of Testing Earthquake Forecast Models

The Collaboratory for the Study of Earthquake Predictability (CSEP) is an international effort to independently evaluate earthquake forecasting models and to provide the cyber-infrastructure together with a suite of testing methods. For global forecasts, CSEP defines a grid-based format to describe the expected rate of earthquakes, which is composed of 6.48 million cells for a 0.1º spacing. The spatial performance of the forecast is tested using the Spatial test (S-test), based on joint log-likelihood evaluations. The high-resolution grid combined with sparse and inhomogeneous earthquake distributions leads to many empty cells that may never experience an earthquake, biasing the S-test results. To explore this issue, we conducted a global earthquake forecast experiment. We tested a spatially uniform forecast model, which is non-informative and should be rejected by the S-test. However, it is not rejected by the S-test when the spatial resolution is high enough to allocate each observed earthquake in individual cells, thus raising questions about the test statistical power.

The number of observed earthquakes used to evaluate global forecasts is usually only a few hundred, in contrast to the millions of spatial cells. Our analysis shows that for such disparity, the statistical power of tests for single-resolution grids also depends on the number of earthquakes available to evaluate a model. With few earthquakes, the S-test does not allow powerful testing.

We propose to use a multi-resolution grid to generate and test earthquake forecast models, in which the resolution can be set freely based on available data, e.g., by the number of earthquakes per cell. Data-driven multi-resolution grids demonstrate the ability to reject the uniform forecast, contrary to a high-resolution grid. Furthermore, multi-resolution grids offer powerful testing with as minimum as four earthquakes available in the test catalog. Therefore, we propose to use multi-resolution grids in future CSEP global forecast experiments and to further study its application in regional and local experiments, where such sparsity of observations is present.