Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Verification test suite for the conditioned GMF calculator #8542

Merged
merged 30 commits into from
Mar 27, 2023
Merged

Conversation

raoanirudh
Copy link
Member

@raoanirudh raoanirudh commented Mar 17, 2023

Adding a set of verification tests for the conditioned GMF calculator. Final part of #8317. This test set currently replicates the set of well-designed simplified verification tests used by the ShakeMap model module.

In addition, this PR also makes the following changes to the conditioned GMF calculator:

  • Move the module from hazardlib/shakemap to hazardlib/calc
  • Relax hardcoding of the spatial correlation model usable with the module
  • Avoid small negative values in the conditioned covariance matrices, by clipping to zero
  • Correct the formula used for the nominal_bias_stddev (avoid a double sqrt)

Plots of the test case results are shown in the table below:

USGS ShakeMap Model Module OpenQuake-engine Conditioned GMFs Module
shakemap_verification_test_0001_PGA oq_test_case_01
shakemap_verification_test_0002_PGA oq_test_case_02
shakemap_verification_test_0003_PGA oq_test_case_03
shakemap_verification_test_0004_PGA oq_test_case_04
shakemap_verification_test_0004b_PGA oq_test_case_04b
shakemap_verification_test_0005_PGA oq_test_case_05
shakemap_verification_test_0006_PGA oq_test_case_06
shakemap_verification_test_0007_spectra_plot oq_test_case_07
shakemap_verification_test_0008_PGA oq_test_case_08
shakemap_verification_test_0009_PGA oq_test_case_09
shakemap_verification_test_0010_PGA oq_test_case_10

Minor refactoring to move the ConditionedGmfComputer from the
hazardlib/shakemap module to hazardlib/calc, which is the more
appropriate location for the class, given the extent of code
shared with the regular GmfComputer.

The shakemap module instead is better suited to support the
different implementations of ShakeMap, including ones run by
the  Swiss Seismological Servicehazardlib/calc, which is the more
appropriate location for the class, given the extent of code
shared with the regular GmfComputer.

The shakemap module instead is better suited to support the
different implementations of ShakeMap, including ones run by
the Swiss Seismological Service (SED) in Switzerland,
INGV in Italy, Institute of Engineering Seismology and
Earthquake Engineering (ITSAK) in Greece, and ShakeMapEU.
Test cases 01–10 are based on the verification tests described in the
USGS ShakeMap 4.1 Manual.
Ref: Worden, C. B., E. M. Thompson, M. Hearne, and D. J. Wald (2020).
ShakeMap Manual Online: technical manual, user’s guide, and software guide,
U.S. Geological Survey. DOI: https://doi.org/10.5066/F7D21VPQ, see
https://usgs.github.io/shakemap/manual4_0/tg_verification.html`.
And switch to numpy.testing.assert_allclose for all checks in
previous cases
Case not tested, but used for generating a plot for comparison
These three cases are included for plotting purposes only
No assertions are made for these tests
These two cases are used for plotting purposes only.
No assertions are made for these test cases.
@raoanirudh raoanirudh self-assigned this Mar 17, 2023
@micheles micheles added this to the Engine 3.17.0 milestone Mar 20, 2023
@raoanirudh
Copy link
Member Author

Updated case_07 plots

USGS ShakeMap Model Module OpenQuake-engine Conditioned GMFs Module
shakemap_verification_test_0007_spectra_plot oq_test_case_07

@raoanirudh
Copy link
Member Author

The case_06 discrepancy is due to the parameter bias_max_range on ShakeMap, the equivalent of which is not implemented in the OpenQuake-engine. Stations beyond this distance are not included in the bias calculation. The default bias_max_range for verification_test_0006 is set to 120 km. Artificially setting the same cutoff in the ConditionedGmfComputer resolves the discrepancy.

Updated case_06 plots

USGS ShakeMap Model Module OpenQuake-engine Conditioned GMFs Module, artificially ignoring stations beyond 120 km
shakemap_verification_test_0006_PGA oq_test_case_06 (debug mode)

@raoanirudh raoanirudh merged commit 6e74250 into master Mar 27, 2023
raoanirudh added a commit that referenced this pull request Mar 30, 2023
Test suite added in #8542
@raoanirudh raoanirudh mentioned this pull request Mar 30, 2023
@raoanirudh raoanirudh deleted the cgmf-test branch May 4, 2023 10:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants