Hamlet: An Application for Seismic Hazard Model Evaluation and Testing

Reliable seismic hazard modeling requires careful calibration to the datasets used to create the model. Additionally, assessing the performance of an existing model involves statistical comparisons to data not used in model construction. However, despite the benefits of such statistical evaluations, comparing a model with the observed data is inherently challenging due to the following reasons:

1) A probabilistic seismic hazard model is essentially a structured collection of 10⁵-10⁸ unique potential earthquake ruptures and the occurrence rates and ground motion fields associated with each rupture. Seismological or geological observations usually include much smaller samples of earthquakes (in the order of 10³-10⁴) and active faults (in the order of 10¹-10²), and the data are usually incomplete.

2) To construct a model from such observations, various assumptions and processing are involved (e.g., Poisson time independence of modeled seismicity, declustering of seismic catalogs, and modeling earthquake occurrence based on fault slip rates).

To facilitate model evaluations, we developed Hamlet (Hazard Model Evaluation and Testing), an application designed to process seismic hazard models and perform rigorous statistical comparisons between the model and observations efficiently and flexibly. Hamlet performs the M-, N-, S-, and L-tests recommended by the Center for the Study of Earthquake Predictability (CSEP) and other statistical comparisons (e.g., maximum earthquake magnitudes and seismic moments) by generating a large number of simulated earthquake catalogs of the same duration as an observed seismic catalog, mitigating many of the concerns about differences between the model and the observations. Hamlet can perform statistical comparisons both over the entire model domain and within equal-area hexagonal spatial cells of user-specified size, to better constrain the spatial differences in model performance. Moreover, Hamlet can subset a seismic source model by subregion, logic tree branch, and source type and perform the tests for the subset. This is particularly useful during model construction, as it allows the modeler to understand and adjust how individual components of the model perform.

We have used Hamlet to analyze over 30 hazard models to gauge the applicability of different tests and evaluations to a variety of cases (e.g., seismic hazard models with different characteristics), and to test some widely used assumptions in hazard models to identify which work and which do not. Hamlet is a work in progress, and we are continually adding features and evaluations. Functionality in development or for consideration for the future includes matching the most similar rupture in the source model to each earthquake in an observed seismic catalog, and statistical evaluations against a seismic catalog with different durations for different magnitude ranges. We are also exploring testing of predicted ground motion values and loss (risk) for observed events.