Adapt NOAA Marine Vx python code into a MET python embedding script to serve up gridded data.

[JohnHalleyGotway]

Describe the Task

On 12/17/20, DTC staff met with NOAA/EMC staff to discuss the transition of the existing EMC Marine Verification functionality into METplus. One major topic of conversation was the handling of Marine data in NetCDF4 format on a tri-polar grid. No existing NetCDF CF conventions exist for defining tri-polar grids in the metadata. Likewise tri-polar grids do not appear to be definable based on GRIB2 documentation.

So the prospects for supporting tri-polar grids directly in MET, as we do for other projection types (latlon, polar stereographic, mercator, lambert conformal), are slim.

Also be aware that the xesmf python package (https://pypi.org/project/xesmf/) can be used to pretty efficiently regrid tri-polar data to a standard grid.

This task is meant to be an initial first step for this project. Start by...
(1) Inspecting the existing Marine vx python functionality.
(2) Based on the Marine code, write a python-embedding script for MET to... read tri-polar data, call xesmf to regrid it to some standard grid, and serve up the result in memory for MET.
(3) Run this script via plot_data_plane to confirm that MET is reading the data correctly (oriented correctly at the right spot on earth).

After this initial step is done, consider if/how any of the Marine vx python functionality should be factored out of this script into a higher level spot in METplus. Also consider options to make it easy for the user to specify the target grid. For example, a latlon grid over the mid-latitudes is very useful, as well as a polar stereographic grid over the north pole.

Time Estimate

3 days(?)

Sub-Issues

Consider breaking the task down into sub-issues.

• Add a checkbox for each sub-issue here.

Relevant Deadlines

ASAP but by June 2021 at the latest

Funding Source

2788881 (updated)

Define the Metadata

Assignee

• Select engineer(s) or no engineer required: not sure yet
• Select scientist(s) or no scientist required: John Opatz is leading this effort

Labels

• Select component(s)
• Select priority
• Select requestor(s)

Projects and Milestone

• Review projects and select relevant Repository and Organization ones or add "alert:NEED PROJECT ASSIGNMENT" label
• Select milestone to next major version milestone or "Future Versions"

Define Related Issue(s)

Consider the impact to the other METplus components.

• METplus, MET, METdatadb, METviewer, METexpress, METcalcpy, METplotpy
  No impacts.

Task Checklist

See the METplus Workflow for details.

• Complete the issue definition above, including the Time Estimate and Funding Source.
• Fork this repository or create a branch of develop.
  Branch name: feature_<Issue Number>_<Description>
• Complete the development and test your changes.
• Add/update log messages for easier debugging.
• Add/update unit tests.
• Add/update documentation.
• Push local changes to GitHub.
• Submit a pull request to merge into develop.
  Pull request: feature <Issue Number> <Description>
• Define the pull request metadata, as permissions allow.
  Select: Reviewer(s), Project(s), Milestone, and Linked issues
• Iterate until the reviewer(s) accept and merge your changes.
• Delete your fork or branch.
• Close this issue.

[georgemccabe]

I was able to get tri-polar read into MET using the xesmf Python package.

This is an image from ncdump of the input data:

This following images are output from running PlotDataPlane using the python embedding script as input:

Northern hemisphere:

Southern hemisphere:

Both images are using 1 degree longitude spacing and 0.5 degree latitude spacing. This can be adjusted in the script.

Python Embedding Script: read_tripolar_grid.py.txt

Commands run on kiowa:

conda activate /home/met_test/.conda/envs/metplus_tripolar
export MET_PYTHON_EXE=python3

/usr/local/met/bin/plot_data_plane PYTHON_NUMPY /home/mccabe/tripolar-test/out.ps 'name="~/tripolar-test/read_tripolar_grid.py ~/tripolar-test/input/20200127/rtofs_glo_2ds_n048_daily_diag.nc ice_coverage north";'
convert -rotate 90 /home/mccabe/tripolar-test/out.ps /home/mccabe/tripolar-test/north-1x0.5.png

/usr/local/met/bin/plot_data_plane PYTHON_NUMPY /home/mccabe/tripolar-test/out.ps 'name="~/tripolar-test/read_tripolar_grid.py ~/tripolar-test/input/20200127/rtofs_glo_2ds_n048_daily_diag.nc ice_coverage south";'
convert -rotate 90 /home/mccabe/tripolar-test/out.ps /home/mccabe/tripolar-test/south-1x0.5.png

[georgemccabe]

Updated script based on feedback from Todd Spindler. Added additional optional argument to specify the path to the weight file to use. This drastically speeds up execution time. The weight file should be included with the input data for the use case.

Updated script: read_tripolar_grid.py

[j-opatz]

Tying this to #791, which focuses on bring this capability to a PlotDataPlane use case. This may be tied to a future GitHub as well, after a conversation with Todd Spindler on 2/3 indicated a need for higher resolution (1/12th deg) processing of tripolar coordinate datasets and it is unknown how much additional work will be required to adapt this code to that need.

[j-opatz]

With the creation of overarching #800 and in preparing for the next major METplus release, this issue is being closed. If a new implementation is needed for higher resolution processing of tripolar data, then a new issue will be created.