Fix ASCII2NC to check the return status when reading ASCII input files.

[JohnHalleyGotway]

Describe the Enhancement

This issue arose in testing by @georgemccabe for dtcenter/METplus#1223.
He found that even though the python embedding script he ran in ascii2nc returned bad status, the ascii2nc tool itself did not.

Modify the logic on

MET/met/src/tools/other/ascii2nc/ascii2nc.cc

Line 240 in e903f8f

file_handler->readAsciiFiles(asfile_list);

to check the return status and error out with a useful error message for bad status.

Here's some input from @georgemccabe about what/how to test:

In this command, the Python script crashes but ascii2nc does not error (you'll likely need to change the output path):

/d1/projects/MET/MET_releases/met-10.1.0-beta3/bin/ascii2nc "/usr/local/METplus/parm/use_cases/model_applications/convection_allowing_models/Point2Grid_obsLSR_ObsOnly_PracticallyPerfect/read_ascii_storm.py /d1/projects/METplus/METplus_Data/development/feature_617/model_applications/convection_allowing_models/practically_perfect/200205_rpts_filtered.csv" /d1/personal/mccabe/out2/model_applications/convection_allowing_models/practically_perfect/StormReps.2020020500.nc -format python -v 2

This command succeeds because the file exists:

/d1/projects/MET/MET_releases/met-10.1.0-beta3/bin/ascii2nc "/usr/local/METplus/parm/use_cases/model_applications/convection_allowing_models/Point2Grid_obsLSR_ObsOnly_PracticallyPerfect/read_ascii_storm.py /d1/projects/METplus/METplus_Data/model_applications/convection_allowing_models/practically_perfect/200205_rpts_filtered.csv" /d1/personal/mccabe/out2/model_applications/convection_allowing_models/practically_perfect/StormReps.2020020500.nc -format python -v 2

In general, we let the user's python script determine the error condition. If handed bad status, ascii2nc should error out. Note though that retrieving 0 observations from a python script should not, as a rule, be considered an error. For example, finding 0 storm reports on a given day may not be an error. But if the python script cannot be run or it is run and returns bad status, ascii2nc should error out.

For this issue, test as many combinations of bad behavior as is reasonably possible.

Time Estimate

4 hours.

Sub-Issues

Consider breaking the enhancement down into sub-issues.
None needed.

Relevant Deadlines

List relevant project deadlines here or state NONE.

Funding Source

Define the source of funding and account keys here or state NONE.

Define the Metadata

Assignee

• Select engineer(s) or no engineer required: @JohnHalleyGotway
• Select scientist(s) or no scientist required: none needed

Labels

• Select component(s)
• Select priority
• Select requestor(s)

Projects and Milestone

• Select Repository and/or Organization level Project(s) or add alert: NEED PROJECT ASSIGNMENT label
• Select Milestone as the next official version or Future Versions

Define Related Issue(s)

Consider the impact to the other METplus components.

• METplus, MET, METdatadb, METviewer, METexpress, METcalcpy, METplotpy
  None needed.

Enhancement Checklist

See the METplus Workflow for details.

• Complete the issue definition above, including the Time Estimate and Funding Source.
• Fork this repository or create a branch of develop.
  Branch name: feature_<Issue Number>_<Description>
• Complete the development and test your changes.
• Add/update log messages for easier debugging.
• Add/update unit tests.
• Add/update documentation.
• Push local changes to GitHub.
• Submit a pull request to merge into develop.
  Pull request: feature <Issue Number> <Description>
• Define the pull request metadata, as permissions allow.
  Select: Reviewer(s) and Linked issues
  Select: Repository level development cycle Project for the next official release
  Select: Milestone as the next official version
• Iterate until the reviewer(s) accept and merge your changes.
• Delete your fork or branch.
• Close this issue.

[JohnHalleyGotway]

Tested using existing python unit tests for ascii2nc.

This runs without error:

> $MET_BASE/../../bin/ascii2nc "$MET_BASE/python/read_ascii_point.py $MET_BUILD_BASE/data/sample_obs/ascii/sample_ascii_obs.txt" ascii2nc_python.nc -format python

(1) Providing a bad python filename currently runs without error and returns good status:

> $MET_BASE/../../bin/ascii2nc "$MET_BASE/python/read_ascii_point_BAD.py $MET_BUILD_BASE/data/sample_obs/ascii/sample_ascii_obs.txt" ascii2nc_python.nc -format pythonModuleNotFoundError: No module named 'read_ascii_point_BAD'
WARNING: 
WARNING: PythonHandler::do_straight() -> an error occurred importing module "read_ascii_point_BAD"
WARNING: 
> echo $?
0

After switching to check the return status, we get an error that returns bad status from ascii2nc:

WARNING: 
WARNING: PythonHandler::do_straight() -> an error occurred importing module "read_ascii_point_BAD"
WARNING: 
ERROR  : 
ERROR  : ascii2nc-> encountered an error while reading input files!
ERROR  : 
> echo $?
1

(2) Providing a good path to python script but with no arguments does currently return bad status:

ERROR: read_ascii_point.py -> Must specify exactly one input file.

(3) But providing a good path to python with an input file that doesn't exist returns good status:

FileNotFoundError: [Errno 2] No such file or directory: 'abc'
WARNING: 
WARNING: PythonHandler::do_straight() -> an error occurred importing module "read_ascii_point"
WARNING: 

(4) Similarly dividing by 0 in the python script returns good status.

After adding a check for the return status, these commands all now error out with:

ERROR  : 
ERROR  : ascii2nc-> encountered an error while reading input files!
ERROR  : 

And return a non-zero status.