Statistical power of spatial earthquake forecast tests

Asim Khawaja*, Sebastian Hainzl, Danijel Schorlemmer, Pablo Iturrieta, Jose A Bayona, William Savran, Max Werner, Warner Marzocchi

*Corresponding author for this work

Research output: Contribution to journalArticle (Academic Journal)peer-review

3 Citations (Scopus)


The Collaboratory for the Study of Earthquake Predictability (CSEP) is an international effort to evaluate earthquake forecast models prospectively. In CSEP, one way to express earthquake forecasts is through a grid-based format: the expected number of earthquake occurrences within 0.1○ × 0.1○ spatial cells. The spatial distribution of seismicity is thereby evaluated using the Spatial test (S-test). The high-resolution grid combined with sparse and inhomogeneous earthquake distributions leads to a huge number of cells causing disparity in the number of cells and the number of earthquakes to evaluate the forecasts, thereby affecting the statistical power of the S-test. In order to explore this issue, we conducted a global earthquake forecast experiment, in which we computed the power of the S-test to reject a spatially non-informative uniform forecast model. The S-test loses its power to reject the non-informative model when the spatial resolution is so high that every earthquake of the observed catalog tends to get a separate cell. Upon analyzing the statistical power of the S-test, we found, as expected, that the statistical power of the S-test depends upon the number of earthquakes available for testing, e. g. with the conventional high-resolution grid for the global region, we would need more than 32000 earthquakes in the observed catalog for powerful testing, which would require approximately 300 years to record M ≥ 5.95. The other factor affecting the power is more interesting and new; it is related to the spatial grid representation of the forecast model. Aggregating forecasts on multi-resolution grids can significantly increase the statistical power of the S-test. Using the recently introduced Quadtree to generate data-based multi-resolution grids, we show that the S-test reaches its maximum power in this case already for as few as 8 earthquakes in the test period. Thus we recommend for future CSEP experiments the use of Quadtree-based multi-resolution grids, where available data determine the resolution.
Original languageEnglish
Pages (from-to)2053-2066
Number of pages14
JournalGeophysical Journal International
Issue number3
Publication statusPublished - 24 Jan 2023

Bibliographical note

Funding Information:
The authors thank Antonio Morales, Chris Rollins and one anonymous reviewer for reviewing and providing constructive feedback to improve the manuscript. The authors are grateful to friends and colleagues for their valuable suggestions. This project has received funding from the European Union's Horizon 2020 research and innovation program under Grant Agreement Number 821115, Real-Time Earthquake Risk Reduction for a Resilient Europe (RISE). This researchwas also supported by the Southern California Earthquake Center (Contribution No. 12691). SCEC is funded by NSF Cooperative Agreement EAR-1600087 & USGS Cooperative Agreement G17AC00047. In particular, we want to thank the opensource community for the Linux operating system and the many programs used in this study, e.g. Python (, Spyder (Raybaut 2009) and PyCharm ( m/pycharm), OpenStreetMap ( and QGIS (

Publisher Copyright:
© 2023 The Author(s).


  • Earthquake interaction, forecasting, and prediction
  • Statistical seismology
  • Earthquake hazards
  • Earthquake forecast testing
  • Statistical power analysis


Dive into the research topics of 'Statistical power of spatial earthquake forecast tests'. Together they form a unique fingerprint.

Cite this