Fb_global_unstr #335

aliabdolali · 2021-03-18T18:27:07Z

Pull Request Summary

This pull request addresses the periodicity fix for unstructured grids with global coverage, so energy can transfer at the two ends of the mesh. This PR solves the problem for two parallelizations implemented in WW3 for unstructured grids (Card-Deck and Domain Decomposition-PDLIB).

Description

The connectivity between elements is checked to find elements that connect two ends of the mesh with a longitudinal difference of ~360 degrees and lets energy transfer between them.
Corrects calculation of element areas and edge lengths across the
dateline in w3triamd.ftn
Fixes calculation to determine whether a point is inside an element that spans the dateline in w3triamd.ftn
Corrects calculation of element centers and corners across dateline in wmscrpmd.ftn
Write out SCRIP file in netCDF format at the end of ww3_grid (for creating offline remaping files). This required adding netCDF support to ww3_grid
Also includes minor fix for the /O7a switch in w3iopomd.ftn
A new regression test is added to check this enhancement (ww3_tp2.21). The inputs (mesh, wind forcing, UOST.in files) are added to the tar file on FTP server.

Issue(s) addressed

fixes Periodicity fix for global unstructured grids #334
fixes noaa-emc/ww3/issues/Periodicity fix for global unstructured grids #334
No documentation update is required.

Developers: @sbrus89, @menta78 and @aliabdolali

Reviewers: @JessicaMeixner-NOAA

Testing

regtests/ww3_tp2.21 is added for a global unstructured grid with both card deck and domain decomposition.
The matrix_ncep has been run on Hera machine using intel/18.0.5.274 impi/2018.0.4 netcdf/4.6.1.
Here is the summary output of matrix.comp on NOAA HPC (Hera):
MatrixDiff.zip
expected changes in the outputs in mww3_test_02, mww3_test_03, mww3_test_04, mww3_test_06, ww3_tp2.17 ust in grid stdout for the nodes sign check (SI nbPlus,nbMinus,nbZero)
mww3_test_03, mww3_test_07, ww3_tp2.10, ww3_tp2.18 are expected non-identical tests.

to ensure they comply with the limits of the nameslist.

Changes to add support to 360-day and 365-day (no leap year) calendar - see ticket #209 * Additional CALTYPE namelist parameter in MISC section * New ww3_tc1 regtest.

* Updated ww3_bound and ww3_bounc to handle model grids formulated on a rotated pole. * Manual and nml/inp files to updated clarify that ww3_bound/ww3_bounc only accept input spectra formulated on a standard pole grid.

Updates to allow a coupling time step that is different from the model time step. * Includes new regtest (in ww3_tp2.14) for non-default oasis time step. * ww3_tp2.14 regtest added to matrix.base.

- Corrects calculation of element areas and edge lengths across the dateline in w3triamd.ftn - Fixes calculation to determine whether a point is inside an element that spans the dateline in w3triamd.ftn - Corrects calculation of element centers and corners across dateline in wmscrpmd.ftn - Write out SCRIP file in netCDF format at the end of ww3_grid (for creating offline remaping files). This required adding netCDF support to ww3_grid - Also includes minor fix for the /O7a switch in w3iopomd.ftn

…ge from global grids from min(longitude)=-180, max(longitude)=180 to \delta(longitude)=360 degrees and added the regression test ww3_tp2.21

…develop_rebase

…' into FB_Global_Unstr

PDLIB/yowpdlibmain.ftn: fix to handle global meshes

…nputs from ww3_tp2.21

model/bin/make_makefile.sh

JessicaMeixner-NOAA · 2021-03-18T18:40:42Z

model/bin/w3_make

-  reg_programs="ww3_grid" 
-  reg_programs="$reg_programs ww3_strt"
+#  reg_programs="ww3_grid" 
+#  reg_programs="$reg_programs ww3_strt"


clean up commented out text

JessicaMeixner-NOAA · 2021-03-18T18:42:23Z

model/bin/w3_make

@@ -154,6 +155,7 @@
  cdf_programs="$cdf_programs ww3_ounp"
  cdf_programs="$cdf_programs ww3_bounc"
  cdf_programs="$cdf_programs ww3_trnc"
+  cdf_programs="$cdf_programs ww3_grid"


Is there an option that says if ww3_grid should be a netcdf program or not? The way this currently is, you are now requiring netcdf for ww3_grid, which essentially is now requiring netcdf as a library, no? While I know there are ongoing discussions of integrating more netcdf, is there a way to make this optional or is it always essential now?

@JessicaMeixner-NOAA It is required when SCRIPNC is in the switch.
From a communication with @sbrus89:
The scrip.nc can be used with ESMF_regrid_gen_weights (or similar). I added this to aid in the creation of coupler mapping files for unstructured grids for E3SM. I wasn't sure if this was something that would be useful to include in the PR or not, so I decided to leave it in just in case. We can certainly leave it out if you think that would be better.

So, as Steven suggested, I can exclude SCRIP related developments, and we no longer need netcdf for ww3_grid, but I thought it might be helpful.

Are all of the netcdf SCRIP related updates behind a switch?

What I don't want to do, is require everyone even someone who is using all ascii files to now have a netcdf dependency. If we move all output to netcdf, then yes all programs will have netcdf dependency and it will be a requirement. Even in the make_makefile script, this looks like an option, not a requirement, so when deciding if ww3_grid is a netcdf or regular program, shouldn't this also be an option?

I realized it when I was checking mww3_test_04 with switch_PR3_UQ_MPI_SCRIPNC, so I had to modify the make_makefile.sh, the rest of tests went well without netcdf dependency, so I think these are behind SCRIPNC if I am not wrong.

I think this needs to be moved into the if/else section starting on line 172: https://github.com/aliabdolali/WW3/blob/FB_Global_Unstr/model/bin/w3_make#L172

If it's scripnc then add ww3_grid to netcdf programs, if not, it should be a regular program.

@JessicaMeixner-NOAA Thanks, it was a very helpful hint, I made the changes and ran the regtests again, all in good shape. Could you approve the review?

JessicaMeixner-NOAA · 2021-03-19T13:27:13Z

@aliabdolali thanks for updating the makefile. I'll continue the rest of my review now. Hopefully it'll be done by Monday.

model/bin/w3_make

JessicaMeixner-NOAA · 2021-03-19T14:53:45Z

@aliabdolali I'm okay with these changes, but I don't understand why this PR exists and why this was not included in PR #303 instead? Shouldn't anything here have instead been pushed to that PR? Why doesn't the target of this PR get changed to the branch used in PR#303. Once that is done, we can re-run regtests and it looks like the updates are good to go.

aliabdolali · 2021-03-19T15:08:39Z

@aliabdolali I'm okay with these changes, but I don't understand why this PR exists and why this was not included in PR #303 instead? Shouldn't anything here have instead been pushed to that PR? Why doesn't the target of this PR get changed to the branch used in PR#303. Once that is done, we can re-run regtests and it looks like the updates are good to go.

@JessicaMeixner-NOAA This one is PR #303 plus works from Lorenzo on PDLIB, so he was not able to push his changes to Steven PR and I made it on my own fork. We can continue with this one and merge as it has periodicity fixes for both Card Deck and Domain Decomposition. I'll check with Steven if he has any objections.

JessicaMeixner-NOAA · 2021-03-19T15:12:45Z

Couldn't push to his branch or make a PR there? I thought anyone could make PRs to others branches? Please check with @sbrus89 and let me know.

sbrus89 · 2021-03-19T15:15:15Z

@aliabdolali and @JessicaMeixner-NOAA, I'm okay with whatever you guys think works best. Since Ali already has the changes merged here, I'm perfectly fine with going ahead with #335 in place of #303.

sbrus89 · 2021-03-19T15:16:20Z

I had not run any PDLIB cases, so I'm glad that bug got identified and addressed.

aliabdolali · 2021-03-19T15:34:30Z

I had not run any PDLIB cases, so I'm glad that bug got identified and addressed.

Thanks @sbrus89 All in good shape with your development, and all types of options for unstructured are now functioning well for global meshes.

aliabdolali · 2021-03-20T13:41:24Z

@JessicaMeixner-NOAA
I reran the test, it is similar to the previous comparison.
MatrixDiffMarch20.zip

UKMO-lsampson and others added 23 commits July 22, 2020 11:44

Added boundary checks to the SMC grid input files for ww3_grid,

98c5702

to ensure they comply with the limits of the nameslist.

Fb 360 calendar (#8)

71b9ba9

Changes to add support to 360-day and 365-day (no leap year) calendar - see ticket #209 * Additional CALTYPE namelist parameter in MISC section * New ww3_tc1 regtest.

RTD support for ww3_boun[dc] (#10)

6c42d17

* Updated ww3_bound and ww3_bounc to handle model grids formulated on a rotated pole. * Manual and nml/inp files to updated clarify that ww3_bound/ww3_bounc only accept input spectra formulated on a standard pole grid.

Fb coupling time (#9)

5c361aa

Updates to allow a coupling time step that is different from the model time step. * Includes new regtest (in ww3_tp2.14) for non-default oasis time step. * ww3_tp2.14 regtest added to matrix.base.

Merge remote-tracking branch 'upstream/staging' into develop

cc34122

bug fix for ukmet development

33a7af3

Merge remote-tracking branch 'upstream/develop' into develop

68c3c96

Merge remote-tracking branch 'upstream/develop' into develop

663d983

Merge remote-tracking branch 'upstream/develop' into develop

d4887f5

Merge remote-tracking branch 'upstream/develop' into develop

dc0cea9

modify the model/ftn/w3triamd.ftn and model/ftn/wmscrpmd.ftn and chan…

083a910

…ge from global grids from min(longitude)=-180, max(longitude)=180 to \delta(longitude)=360 degrees and added the regression test ww3_tp2.21

Merge remote-tracking branch 'upstream/develop' into global_unstr_PR_…

8917422

…develop_rebase

bug fix for ww3_grid make without SCRIP switch

febddae

Fix for corner node periodicity in wmscrpmd.ftn

6bc0e5f

Merge remote-tracking branch 'upstream/global_unstr_PR_develop_rebase…

4ee6dc9

…' into FB_Global_Unstr

PDLIB/yowpdlibmain.ftn: fix to handle global meshes

558e7bf

Merge pull request #1 from menta78/FB_Global_Unstr

7a7d90a

PDLIB/yowpdlibmain.ftn: fix to handle global meshes

bug fix for SCRIPNC switch

77d513b

update info for ww3_tp2.21 for domain decomposition and PDLIB option

0ca3076

fix for mww3_04 link with SCRIP and SCRIPNC switch

ef2da97

add inputs to tar file, update model/bin/ww3_from_ftp.sh and remove i…

2ff1b57

…nputs from ww3_tp2.21

reduce the duration of ww3_tp2.21 for the sake of regtest time

a487bdb

aliabdolali requested a review from JessicaMeixner-NOAA March 18, 2021 18:27

aliabdolali added the enhancement New feature or request label Mar 18, 2021

JessicaMeixner-NOAA reviewed Mar 18, 2021

View reviewed changes

model/bin/make_makefile.sh Show resolved Hide resolved

JessicaMeixner-NOAA reviewed Mar 18, 2021

View reviewed changes

Ali added 2 commits March 18, 2021 19:22

small editorial fixes

69ca74a

fixes for SCRIPNC switch

e443b62

aliabdolali requested a review from JessicaMeixner-NOAA March 19, 2021 13:23

Ali added 2 commits March 19, 2021 13:28

change date for ww3_ounf and ww3_ounp for ww3_tp2.21

4177081

Merge remote-tracking branch 'upstream/develop' into FB_Global_Unstr

3a51cdf

JessicaMeixner-NOAA requested changes Mar 19, 2021

View reviewed changes

model/bin/w3_make Show resolved Hide resolved

edit w3_make

af86335

JessicaMeixner-NOAA approved these changes Mar 22, 2021

View reviewed changes

aliabdolali merged commit 854fc86 into NOAA-EMC:develop Mar 22, 2021

sbrus89 mentioned this pull request Apr 2, 2021

Update WW3 submodule and related changes E3SM-Project/OMEGA-backup#1

Merged

sbrus89 mentioned this pull request Aug 17, 2022

Global mesh for WAVEWATCH3 CHLNDDEV/OceanMesh2D#273

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fb_global_unstr #335

Fb_global_unstr #335

aliabdolali commented Mar 18, 2021

JessicaMeixner-NOAA Mar 18, 2021

aliabdolali Mar 18, 2021

JessicaMeixner-NOAA Mar 18, 2021

aliabdolali Mar 18, 2021

JessicaMeixner-NOAA Mar 18, 2021

JessicaMeixner-NOAA Mar 18, 2021

aliabdolali Mar 18, 2021

JessicaMeixner-NOAA Mar 18, 2021

aliabdolali Mar 19, 2021

JessicaMeixner-NOAA commented Mar 19, 2021

JessicaMeixner-NOAA commented Mar 19, 2021

aliabdolali commented Mar 19, 2021

JessicaMeixner-NOAA commented Mar 19, 2021

sbrus89 commented Mar 19, 2021

sbrus89 commented Mar 19, 2021

aliabdolali commented Mar 19, 2021

aliabdolali commented Mar 20, 2021

Fb_global_unstr #335

Fb_global_unstr #335

Conversation

aliabdolali commented Mar 18, 2021

Pull Request Summary

Description

Issue(s) addressed

Testing

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

JessicaMeixner-NOAA commented Mar 19, 2021

JessicaMeixner-NOAA commented Mar 19, 2021

aliabdolali commented Mar 19, 2021

JessicaMeixner-NOAA commented Mar 19, 2021

sbrus89 commented Mar 19, 2021

sbrus89 commented Mar 19, 2021

aliabdolali commented Mar 19, 2021

aliabdolali commented Mar 20, 2021