Enhance Series-Analysis to compute the BRIERCL statistic from the PSTD line type

[j-opatz]

Describe the New Feature

In working with CPC on NMME data and Probability of Exceedance (POE) calculations, there have been numerous times where a score (i.e. Brier) will line up correctly, but the skill score (i.e. BSS) will not. In these situations, having access to a 2D field for the reference value, BRIERCL, would be incredibly useful for diagnosing where the discrepancies are coming from between CPC and MET output.

This was discussed with Tara, and both parties agreed that while it would be a useful diagnosis tool, it may not necessarily belong in the full release (i.e., this may be a feature in beta only). More input will be needed from the C++ engineering members.

Acceptance Testing

NMME data from CPC can be used. Located on Kiowa.

METplus GenEnsProd config file: /d1/projects//CPC_data/scripts/GenEnsProd-SA_NMME.conf
Python scripts to ingest files: /d1/projects/CPC_data/scripts/forecast_read-in_CFSv2.py and /d1/projects//CPC_data/scripts/preprocessFun.py
Ensemble input file: /d1/projects/CPC_data/input/NMME/new_data/raw_fcst/.fcst.nc
GenEnsProd output files: /d1/projects/CPC_data/output/NMME_out/GenEnsProd-SA/.nc

METplus SeriesAnalysis config file: /d1/projects/CPC_data/scripts/SA_testing_for_CFSv2_GeorgeFix.conf
Climo mean field input file: /d1/projects/CPC_data/input/NMME/new_data/ghcn_cams.1x1.1982-2010.mon.clim.nc
Climo Stddev field input file: /d1/projects/CPC_data/input/NMME/new_data/ghcn_cams.1x1.1982-2010.mon.stddev.nc
Obs field input file: /d1/projects/CPC_data/input/NMME/new_data/ghcn_cams.1x1.1982-2020.mon.nc

Where to request this output is open for suggestion: because the workflow is GenEnsProd > SeriesAnalysis, it would be ideal if the PSTD line type could provide support for a BRIERCL column request in series-analysis output. I can also see a path forward in grid-stat, using an nc_pairs_flag option.

Time Estimate

Will require engineer input
Issues should represent approximately 1 to 3 days of work.

Sub-Issues

Consider breaking the new feature down into sub-issues.

• Add a checkbox for each sub-issue here.

Relevant Deadlines

List relevant project deadlines here or state NONE.

Funding Source

2702691

Define the Metadata

Assignee

• Select engineer(s) or no engineer required
• Select scientist(s) or no scientist required

Labels

• Select component(s)
• Select priority
• Select requestor(s)

Projects and Milestone

• Select Repository and/or Organization level Project(s) or add alert: NEED PROJECT ASSIGNMENT label
• Select Milestone as the next official version or Future Versions

Define Related Issue(s)

Consider the impact to the other METplus components.

• METplus, MET, METdatadb, METviewer, METexpress, METcalcpy, METplotpy

New Feature Checklist

See the METplus Workflow for details.

• Complete the issue definition above, including the Time Estimate and Funding source.
• Fork this repository or create a branch of develop.
  Branch name: feature_<Issue Number>_<Description>
• Complete the development and test your changes.
• Add/update log messages for easier debugging.
• Add/update unit tests.
• Add/update documentation.
• Push local changes to GitHub.
• Submit a pull request to merge into develop.
  Pull request: feature <Issue Number> <Description>
• Define the pull request metadata, as permissions allow.
  Select: Reviewer(s) and Linked issues
  Select: Repository level development cycle Project for the next official release
  Select: Milestone as the next official version
• Iterate until the reviewer(s) accept and merge your changes.
• Delete your fork or branch.
• Close this issue.

[JohnHalleyGotway]

@j-opatz regarding Series-Analysis, I took a look starting at this line of code. And I see that "BRIERCL" is NOT included as one of the output stat types. However on this line of code, I see that Series-Analysis is storing the climo information, assuming it has been provided.

So adding it to the output to Series-Analysis should be relatively straight-forward... assuming there isn't some logical issue that arises in the computation of that statistic over the time series of data. But I don't anticipate that.

I don't think it can be added to the NetCDF matched pairs output from Grid-Stat. BRIERCL is just a statistic derived from an Nx2 contingency table, just like the BRIER score is. So it's computed by aggregating over "something". In Grid-Stat, it's a spatial aggregation over multiple grid points. In Series-Analysis its (typically) a temporal aggregation, separately for each grid point. That being said... I suppose it's possible that there's some climo-related field that already is (or could be) computed during the processing logic... and that could be added to the NetCDF output. But I'd need you to clarify exactly what that is.

[j-opatz]

Thanks for looking into this @JohnHalleyGotway, and it seems like you've grasped the situation perfectly.

An output from series-analysis would be sufficient, and I understand that a similar output from grid-stat isn't doable, given the aggregation need. As this is using a gen-ens-prod --> series-analysis route, it's a good outcome that BRIERCL be accessible via series-analysis.