Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update develop-ref after #1688 and #1689 #1690

Merged
merged 13 commits into from
Mar 2, 2021
Merged

Conversation

JohnHalleyGotway
Copy link
Collaborator

@JohnHalleyGotway JohnHalleyGotway commented Mar 2, 2021

Pull Request Testing

Update develop-ref after #1688 and #1689.

  • Describe testing already performed for these changes:

The MET NB for the develop branch failed on kiowa for NB20210301. The differences are all expected and caused by 2 recent pull requests.

MET PR #1688 for issue #1454 removes two calls to plot_data_plane for input NetCDF files that MET should not have been reading.

ERROR: folder MET-develop/test_output missing 2 files
    plot_data_plane/CESM_b.e11.B20TRC5CNBDRD.f09_g16.002.cice.h1.aice_d_nh.19200101-20051231.ps 
    plot_data_plane/CESM_ssmi.ifrac.gx3v5.2003.ps 

MET PR #1689 for issues #1684 and #1685 makes the following changes:

  • Adds a new Ensemble-Stat output for a new test in unit_climatology.xml:
ERROR: folder MET-develop-ref/test_output missing 3 files
    climatology/ensemble_stat_ONE_CDF_BIN_20120410_120000V_ecnt.txt 
    climatology/ensemble_stat_ONE_CDF_BIN_20120410_120000V_ens.nc 
    climatology/ensemble_stat_ONE_CDF_BIN_20120410_120000V.stat 
  • Changes to the BRIERCL, BRIERCL_NCL, BRIERCL_NCU, and BSS columns of the PSTD output line type, as noted in the diffs:
file1: MET-develop-ref/test_output/climatology/grid_stat_PROB_GFS_CLIMO_1.0DEG_150000L_20120409_120000V_pstd.txt
ERROR: found    16 differences in row type PSTD#52      column BRIERCL       - max abs: 0.017542 
ERROR: found    12 differences in row type PSTD#52      column BRIERCL_NCL   - max abs: 0.017396 
ERROR: found    12 differences in row type PSTD#52      column BRIERCL_NCU   - max abs: 0.017687 
ERROR: found    16 differences in row type PSTD#52      column BSS           - max abs: 0.45214 
ERROR: failed tests for PSTD#52: num
file1: MET-develop-ref/test_output/climatology/grid_stat_PROB_GFS_CLIMO_1.0DEG_150000L_20120409_120000V.stat
ERROR: found    16 differences in row type PSTD#52      column BRIERCL       - max abs: 0.017542 
ERROR: found    12 differences in row type PSTD#52      column BRIERCL_NCL   - max abs: 0.017396 
ERROR: found    12 differences in row type PSTD#52      column BRIERCL_NCU   - max abs: 0.017687 
ERROR: found    16 differences in row type PSTD#52      column BSS           - max abs: 0.45214 
ERROR: failed tests for PSTD#52: num
--
file1: MET-develop-ref/test_output/climatology/point_stat_PROB_GFS_CLIMO_1.0DEG_150000L_20120409_120000V_pstd.txt
ERROR: found    25 differences in row type PSTD#52      column BRIERCL       - max abs: 0.022105 
ERROR: found    21 differences in row type PSTD#52      column BRIERCL_NCL   - max abs: 0.35047 
ERROR: found    21 differences in row type PSTD#52      column BRIERCL_NCU   - max abs: 0.34047 
ERROR: found    25 differences in row type PSTD#52      column BSS           - max abs: 1.73703 
ERROR: failed tests for PSTD#52: num
file1: MET-develop-ref/test_output/climatology/point_stat_PROB_GFS_CLIMO_1.0DEG_150000L_20120409_120000V.stat
ERROR: found    25 differences in row type PSTD#52      column BRIERCL       - max abs: 0.022105 
ERROR: found    21 differences in row type PSTD#52      column BRIERCL_NCL   - max abs: 0.35047 
ERROR: found    21 differences in row type PSTD#52      column BRIERCL_NCU   - max abs: 0.34047 
ERROR: found    25 differences in row type PSTD#52      column BSS           - max abs: 1.73703 
ERROR: failed tests for PSTD#52: num

All of these changes are expected based on the 2 PR's included.

  • Recommend testing for the reviewer(s) to perform, including the location of input datasets, and any additional instructions:

    No additional testing needed.

  • Do these changes include sufficient documentation and testing updates? [Yes]
    This PR serves as documentation for why we updated the reference.

  • Will this PR result in changes to the test suite? [Yes]

    If yes, describe the new output and/or changes to the existing output:

    Yes, as described above.

Pull Request Checklist

See the METplus Workflow for details.

  • Complete the PR definition above.
  • Ensure the PR title matches the feature or bugfix branch name.
  • Define the PR metadata, as permissions allow.
    Select: Reviewer(s), Project(s), and Milestone
  • After submitting the PR, select Linked Issues with the original issue number.
  • After the PR is approved, merge your changes. If permissions do not allow this, request that the reviewer do the merge.
  • Close the linked issue and delete your feature or bugfix branch from GitHub.

JohnHalleyGotway and others added 13 commits February 26, 2021 09:26
* Per #1429, enhance error message from DataLine::get_item().

* Per #1429, I realize that the line number actually is readily available in the DataLine class... so include it in the error message.
* Per #1588, updated pair_data_point.h/.cc to add detailed Debug(4) log messages, as specified in the GitHub issue. Do still need to test each of these cases to confirm that the log messages look good.

* Per #1588, switch very detailed interpolation details from debug level 4 to 5.

* Per #1588, remove the Debug(4) log message about duplicate obs since it's been moved up to a higher level.

* Per #1588, add/update detailed log messages when processing point observations for bad data, off the grid, bad topo, big topo diffs, bad fcst value, and duplicate obs.
…e_CESM_sea_ice_nc becaues of not evenly spaced
…ercentage of the delta instead of fixed tolerance
…ercentage of the delta instead of fixed tolerance
* Per #1684, move an instance of the ClimoCDFInfo class into PairBase. Also define derive_climo_vals() and derive_climo_prob() utility functions.

* Add to VxPairDataPoint and VxPairDataEnsemble functions to set the ClimoCDFInfo class.

* Per #1684, update ensemble_stat and point_stat to set the ClimoCDFInfo object based on the contents of the config file.

* Per #1684, update the vx_statistics library and stat_analysis to make calls to the new derive_climo_vals() and derive_climo_prob() functions.

* Per #1684, since cdf_info is a member of PairBase class, need to handle it in the PairDataPoint and PairDataEnsemble assignment and subsetting logic.

* Per #1684, during development, I ran across and then updated this log message.

* Per #1684, working on log messages and figured that the regridding climo data should be moved from Debug(1) to at least Debug(2).

* Per #1684 and #1685, update the logic for the derive_climo_vals() utility function. If only a single climo bin is requested, just return the climo mean. Otherwise, sample the requested number of values.

* Per #1684, just fixing the format of this log message.

* Per #1684, add a STATLine::get_offset() member function.

* Per #1684, update parse_orank_line() logic. Rather than calling NumArray::clear() call NumArray::erase() to preserve allocated memory. Also, instead of parsing ensemble member values by column name, parse them by offset number.

* Per #1684, call EnsemblePairData::extend() when parsing ORANK data to allocate one block of memory instead of bunches of litte ones.

* Per #1684 and #1685, add another call to Ensemble-Stat to test computing the CRPSCL_EMP from a single climo mean instead of using the full climo distribution.

* Per #1684 and #1685, update ensemble-stat docs about computing CRPSS_EMP relative to a single reference model.

* Per #1684, need to update Grid-Stat to store the climo cdf info in the PairDataPoint objects.

* Per #1684, remove debug print statements.

* Per #1684, need to set cdf_info when aggregating MPR lines in Stat-Analysis.

* Per #1684 and #1685, update PairDataEnsemble::compute_pair_vals() to print a log message indicating the climo data being used as reference:

For a climo distribution defined by mean and stdev:
DEBUG 3: Computing ensemble statistics relative to a 9-member climatological ensemble.

For a single deterministic reference:
DEBUG 3: Computing ensemble statistics relative to the climatological mean.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment