Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

GFS.v16: using NCO installed upp lib on wcoss #212

Merged
merged 5 commits into from
Oct 3, 2020
Merged

GFS.v16: using NCO installed upp lib on wcoss #212

merged 5 commits into from
Oct 3, 2020

Conversation

junwang-noaa
Copy link
Collaborator

Description

  1. NCO changed post lib name to upp lib on wcoss for GFS.v16 implementation. Model needs to use UPP related lib name and module include path.
  2. Post control files have been updated with decimal scale change, the post control files need to be update to be consistent with the ones in POST repo.

Testing

Code has been tested with fv3 parallel test case.

Dependencies

fv3atm PR#182

@junwang-noaa junwang-noaa mentioned this pull request Oct 3, 2020
Merged
WenMeng-NOAA
WenMeng-NOAA approved these changes Oct 3, 2020
View reviewed changes
Copy link
Contributor

@WenMeng-NOAA WenMeng-NOAA left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good to me.

@junwang-noaa junwang-noaa changed the title Upp8011 GFS.v16: using NCO installed upp lib on wcoss Oct 3, 2020
@junwang-noaa junwang-noaa merged commit 5e818ad into ufs-community:production/GFS.v16 Oct 3, 2020
ShanSunNOAA pushed a commit to ShanSunNOAA/ufs-weather-model that referenced this pull request Oct 28, 2020
@climbfuji
Merge NEMSfv3gfs gsd/develop into ufs-weather-model dtc/develop, squa…
6fc17ca 
…shed commit of the following:

commit a9afb3607115ba35cc3c841bf48d410a048912c8
Merge: 6c9a70dc 2d13e99d
Author: Dom Heinzeller <[email protected]>
Date:   Thu Nov 21 15:44:02 2019 -0700

    Merge pull request #279 from climbfuji/update_submodule_pointers_rdlai

    Initialize Leaf Area Index (LAI) from climatology and other submodule updates

commit 2d13e99dc0c5a39ece155e6ac6eb2bd5ddcc1037
Author: Dom Heinzeller <[email protected]>
Date:   Thu Nov 21 15:43:07 2019 -0700

    Update submodule pointers for ccpp/physics and FV3

commit 1653ff116ddca30b690a7375006139eab55237de
Author: Dom Heinzeller <[email protected]>
Date:   Thu Nov 21 12:59:39 2019 -0700

    Update submodule pointers for FV3 and ccpp/physics

commit caed402a8f5a6a535e58284f63f678e692f341d9
Author: Dom Heinzeller <[email protected]>
Date:   Wed Nov 20 16:24:56 2019 -0700

    Update submodule pointer for stochastic physics to point to the head of the master branch

commit 38c66d903797bdc7dbbc99589badece212a63412
Author: Dom Heinzeller <[email protected]>
Date:   Wed Nov 20 16:13:30 2019 -0700

    Update FMS to the head of the GFS-FMS branch

commit 29fbfcad795536eb763499ea98e9596d03c9d483
Author: Dom Heinzeller <[email protected]>
Date:   Wed Nov 20 16:13:08 2019 -0700

    Update submodule WW3 to point back to the NOAA-EMC version, set correct hash

commit 6c9a70dcdf5da8f1f678ad17523ef7533291062f
Merge: 8b1bd280 fd8b7b48
Author: Dom Heinzeller <[email protected]>
Date:   Tue Nov 19 20:54:26 2019 -0700

    Merge pull request #278 from climbfuji/add_gsd_sar_regression_test

    gsd/develop: add GSD SAR regression test

commit fd8b7b487a7b6a2cb3ad0a2e395aaf874b6efd8f
Author: Dom Heinzeller <[email protected]>
Date:   Tue Nov 19 20:53:00 2019 -0700

    tests/tests/fv3_ccpp_gsd_sar: bugfix, correct name of control directory

commit 86cc553107ace9e66429fbfee6e6e2a1dca88c73
Author: Dom Heinzeller <[email protected]>
Date:   Mon Nov 18 14:39:06 2019 -0700

    tests/tests/fv3_ccpp_gsd_sar: correct list of files to check

commit 43dbf0504cf5f86b9ad416e2d6358043df3863fb
Author: Dom Heinzeller <[email protected]>
Date:   Mon Nov 18 09:22:07 2019 -0700

    Add regression test for GSD SAR

commit 8b1bd2805a2543bb1a434dcc0718ca89e27ba284
Merge: c73f665b 13329bf9
Author: Dom Heinzeller <[email protected]>
Date:   Sat Nov 2 06:08:45 2019 +0900

    Merge pull request #277 from climbfuji/gsd_develop_update_submodule_pointer_ccpp_physics_20191102g

    gsd/develop: update submodule pointer for ccpp-physics 2019/11/01

commit 13329bf9b7d8191a4e85789d52f4a6174bf61c50
Author: Dom Heinzeller <[email protected]>
Date:   Sat Nov 2 06:06:24 2019 +0900

    Update submodule pointer for ccpp/physics

commit c73f665b5838a2db86ba5bf65157c89f4ed90cc5
Merge: fafa77ee 0f0778d4
Author: Dom Heinzeller <[email protected]>
Date:   Mon Oct 28 12:54:28 2019 +0900

    Merge pull request #276 from climbfuji/gsd_debug_test_and_submodule_pointer_update_20191028

    gsd/develop: add CCPP GSD suite debug test, reduce timestep for Thompson MP tests, update submodule pointers

commit 0f0778d4cb84150c2b1b3a2e0fe7501b897f2d9f
Author: Dom Heinzeller <[email protected]>
Date:   Mon Oct 28 12:39:54 2019 +0900

    Update submodule pointer for ccpp/physics

commit c3c0379e1cb10b7060095b9a85c045d6b3058e8b
Author: Dom Heinzeller <[email protected]>
Date:   Mon Oct 28 12:33:09 2019 +0900

    Add new regression test for GSD suite in DEBUG mode

commit cc2706d9c35fbbd69048c244dc0997107b9a3c78
Author: Dom Heinzeller <[email protected]>
Date:   Fri Oct 25 11:14:30 2019 +0900

    Reduce timestep for regression tests using Thompson MP from 1200s to 600s

commit fafa77eeae3696ff1fd9922b88becd3644b9dd55
Merge: f3d1f6d2 e6b48fbe
Author: Dom Heinzeller <[email protected]>
Date:   Thu Oct 24 11:00:48 2019 +0900

    Merge pull request #275 from climbfuji/gsd_develop_update_submodule_pointer_20191024

    gsd/develop: update submodule pointer, remote theia

commit e6b48fbea416696a5b23f6910df2697334c8d656
Author: Dom Heinzeller <[email protected]>
Date:   Thu Oct 24 10:57:13 2019 +0900

    Update submodule pointer for ccpp/physics

commit 1bc10a8b7da94ed4d3648157ad9a8dfcef25180a
Author: Dom Heinzeller <[email protected]>
Date:   Mon Oct 21 08:20:15 2019 +0900

    Update submodule pointer for ccpp-physics

commit b583bd011b9e4139c3ff9efc7fa0240554d39a28
Author: Dom Heinzeller <[email protected]>
Date:   Mon Oct 21 08:19:12 2019 +0900

    tests/rt_ccpp_gsd.conf: remove theia

commit f3d1f6d212199af9eb47cd4e723eecbbdbd1f76c
Merge: eac6d996 deec3095
Author: Dom Heinzeller <[email protected]>
Date:   Sun Oct 13 14:59:39 2019 +0900

    Merge pull request #265 from climbfuji/update_submodule_pointers_20191011_1

    gsd/develop: update submodule pointers for FMS and ccpp/physics

commit deec3095e43efd3b1599052525a631faea17d7d0
Author: Dom Heinzeller <[email protected]>
Date:   Fri Oct 11 08:01:14 2019 +0900

    Update submodule pointers for ccpp/physics and FMS

commit eac6d996d84085d4010e8e1bfc8dff1855f9e21f
Merge: 49f6c48e e49075d7
Author: Dom Heinzeller <[email protected]>
Date:   Tue Sep 24 14:21:16 2019 -0600

    Merge pull request #257 from climbfuji/update_submodule_pointers_gsd_develop_20190924

    gsd/develop: update submodule pointers 2019/09/24

commit e49075d70b330b2a7dfa907d9bc7ad6e97a48992
Author: Dom Heinzeller <[email protected]>
Date:   Tue Sep 24 14:19:43 2019 -0600

    Update submodule pointers for FV3, NEMS, ccpp-framework, ccpp-physics

commit 49f6c48edc94ba9d9ac0c143b6068ce56159cb00
Merge: 5ea40b23 221bd404
Author: Dom Heinzeller <[email protected]>
Date:   Tue Sep 24 12:56:42 2019 -0600

    Merge pull request #256 from climbfuji/update_gsd_develop_from_gmtb_develop_20190924

    Update gsd/develop from gmtb/develop 2019/09/24

commit 221bd404496a25f133b1f4e7aaf068225dfd9446
Author: Dom Heinzeller <[email protected]>
Date:   Tue Sep 24 12:51:44 2019 -0600

    Update CODEOWNERS and .gitmodules for gsd/develop, update submodule pointers

commit e30fc6c67c3c98ceab5b0859fe47780de0859b91
Merge: e7691783 3042e609
Author: Dom Heinzeller <[email protected]>
Date:   Tue Sep 24 10:53:31 2019 -0600

    Merge pull request #255 from climbfuji/cleanup_remove_thompson_satadj

    gmtb/develop: cleanup work 2019/09/24

commit 3042e6090e4e4b6d3f22d2e3ee0677df057263ae
Author: Dom Heinzeller <[email protected]>
Date:   Tue Sep 24 09:52:12 2019 -0600

    Update submodule pointer for ccpp-physics, remove Thompson MP + saturation adjustment tests

commit e7691783692215fb713ba60ae2799f33a2147efe
Merge: f1b4c707 d5051cd2
Author: Dom Heinzeller <[email protected]>
Date:   Mon Sep 23 10:16:50 2019 -0600

    Merge pull request #254 from climbfuji/update_submodule_pointers_20190923

    Update submodule pointers 2019/09/23

commit d5051cd2f28c7cb2df686db02664f98584c3fb70
Author: Dom Heinzeller <[email protected]>
Date:   Mon Sep 23 10:15:15 2019 -0600

    Update submodule pointers following https://github.com/NCAR/NEMSfv3gfs/pull/251 et al.

commit f1b4c70787e949ac3557a4f4f89e549574c4065c
Merge: 4d78478e e86179ec
Author: Dom Heinzeller <[email protected]>
Date:   Mon Sep 23 09:33:06 2019 -0600

    Merge pull request #251 from climbfuji/gmtb_develop_update_from_master_20190920

    gmtb/develop: update from master 2019/09/20, remove TRANSITION mode

commit e86179ec519baba659f29c71ea119cf010e6020c
Author: Dom Heinzeller <[email protected]>
Date:   Fri Sep 20 19:59:53 2019 -0600

    Update submodule pointer for ccpp/physics

commit 136b3af34a6c4c9c759fa13f49ca0009e8e9d1c4
Author: Dom Heinzeller <[email protected]>
Date:   Fri Sep 20 08:55:40 2019 -0600

    Remove TRANSITION mode

commit 20a5de0f2bab4ddea82df0bb2682738244ac86f0
Merge: c97254f5 31e98b23
Author: Dom Heinzeller <[email protected]>
Date:   Fri Sep 20 07:41:07 2019 -0600

    Merge branch 'master' of gerrit:NEMSfv3gfs into HEAD

commit c97254f5dc9e84281fa26098b603e3436cead0ac
Merge: 4d78478e fadc8080
Author: Dom Heinzeller <[email protected]>
Date:   Fri Sep 20 07:24:46 2019 -0600

    Merge branch 'master' of https://github.com/NCAR/NEMSfv3gfs into HEAD

commit 4d78478e98a4aa1e547dc714b9dd7b895f71f351
Merge: af03bb30 4c1b7e7b
Author: Dom Heinzeller <[email protected]>
Date:   Thu Sep 19 16:10:13 2019 -0600

    Merge pull request #250 from climbfuji/update_submodule_pointers_20190919

    Update submodule pointers for FV3, ccpp-physics, ccpp-framework 2019/09/19

commit 4c1b7e7b99568e8e3d19c5727e6ba41b2c3ae2d7
Author: Dom Heinzeller <[email protected]>
Date:   Thu Sep 19 16:08:16 2019 -0600

    Update submodule pointers for FV3, ccpp-physics, ccpp-framework

commit af03bb30daac3be4b6aca41043612743f66d618c
Merge: 80ff65a6 5aab0bee
Author: Dom Heinzeller <[email protected]>
Date:   Thu Sep 19 14:40:18 2019 -0600

    Merge pull request #249 from climbfuji/myj_with_dom_mods

    gmtb/develop: MYJ PBL/SFC, Noah MP, Cheyenne modulefiles, cleanup rt*.conf

commit 5aab0bee5e2952529cafb2191f0b8c6f53ea4f43
Author: Dom Heinzeller <[email protected]>
Date:   Wed Sep 18 13:54:29 2019 -0600

    Update submodule pointer for FMS and ccpp/physics

commit 1d09c3b363c6c0ac05cabe4ce6ec2d4cc86499ae
Author: Dom Heinzeller <[email protected]>
Date:   Wed Sep 18 13:49:09 2019 -0600

    Bugfix for modulefiles/macosx.gnu/fv3

commit 04321dc32e522e2ece1ddd952c5771922b45ce6a
Author: Dom Heinzeller <[email protected]>
Date:   Mon Sep 16 15:40:52 2019 -0600

    Update submodule pointers for FV3 and ccpp/physics

commit 2bc9874472eb499d7381a37782fbf3ac04af266b
Author: Dom Heinzeller <[email protected]>
Date:   Mon Sep 16 15:37:57 2019 -0600

    ccpp/CMakeLists.txt: for GNU and REPRO mode, add -g to compiler options

commit 9f5a6ca3be704d40d2473f5bb8017195dbd5d93e
Author: Dom Heinzeller <[email protected]>
Date:   Fri Sep 13 17:21:26 2019 -0600

    Add hera.intel to rt_ccpp_caps.conf, rt_ccpp_gmtb.conf, rt_ccpp_gsd.conf

commit a747a0687153c1e69f8a741bcbdbc67eff766943
Author: Dom Heinzeller <[email protected]>
Date:   Fri Sep 13 16:43:45 2019 -0600

    Update submodule pointers for FV3, ccpp-framework, ccpp-physics

commit 9091411f3fe01d44701545ed75d0b5fc90c313b0
Author: Dom Heinzeller <[email protected]>
Date:   Fri Sep 13 16:43:24 2019 -0600

    Update ESMF version for macOS and linux platforms

commit f1c3c4a37351e7097505cfe5ca136e4b2d5e5d2e
Merge: 1aedd2bb 80ff65a6
Author: Dom Heinzeller <[email protected]>
Date:   Fri Sep 13 16:41:38 2019 -0600

    Merge branch 'gmtb/develop' of https://github.com/NCAR/NEMSfv3gfs into myj_with_dom_mods

commit 1aedd2bb06899f3c273fb151d54e6c9c0b2bf2d8
Author: Dom Heinzeller <[email protected]>
Date:   Fri Sep 13 16:25:52 2019 -0600

    tests/rt_ccpp_gmtb.conf: formatting changes

commit 80ff65a69a144127015aa1703a73dbc123df008e
Merge: a6b70173 4183ee57
Author: Dom Heinzeller <[email protected]>
Date:   Fri Sep 13 16:23:34 2019 -0600

    Merge pull request #246 from climbfuji/gmtb_develop_metadata2html_batchconvert

    gmtb/develop: metadata2html batch processing

commit 141f41e44c3977287808bb4d08f8adfeb2ecbcc1
Author: Dom Heinzeller <[email protected]>
Date:   Fri Sep 13 16:10:38 2019 -0600

    Remove redundant regression test configurations

commit 940b7d1a6de99438a9e5a33bab836dab52e8a0a6
Merge: 26acc72b 3c961fa0
Author: Dom Heinzeller <[email protected]>
Date:   Fri Sep 13 16:08:26 2019 -0600

    Merge branch 'noahmp' of https://github.com/grantfirl/NEMSfv3gfs into myj_with_dom_mods

commit 26acc72b9c895c28f213a73aa17dbe4a953fdaab
Author: Dom Heinzeller <[email protected]>
Date:   Fri Sep 13 16:01:40 2019 -0600

    Combining CCPP GFDLMP namelist template with MYJ changes, add default values to tests/default_vars.sh

    Remove separate regression test config for MYJ, add to rt_ccpp_gmtb.conf

commit abf19ed89d7d0a84840cfccf759d2cad7b8c5526
Author: Dom Heinzeller <[email protected]>
Date:   Fri Sep 13 16:00:17 2019 -0600

    modulefiles/cheyenne.*/fv3: update ESMF to 8.0.0bs48, use modules instead of setting environment variables manually

commit 8a1cc14102eab44fdda81daaec0c0f6ba6a60a51
Author: Dom Heinzeller <[email protected]>
Date:   Fri Sep 13 15:59:30 2019 -0600

    ccpp/suites/suite_FV3_GFS_2017_myj.xml: update with stochastic_physics and UGWD changes

commit 3c961fa0bcf21dc5e5020b1a326caf47c580c58b
Author: Grant Firl <[email protected]>
Date:   Fri Sep 13 14:52:04 2019 -0600

    add noahmp tests to rt.conf and rt_ccpp_gmtb.conf

commit db00f41e14c62aeadad503119e95f287562bd4d7
Merge: b34799b2 282ad6b6
Author: Dom Heinzeller <[email protected]>
Date:   Fri Sep 13 10:46:04 2019 -0600

    Merge branch 'myj_from_qingfu' into detect_machine_cheyenne_update_20190912

commit a6b701733c4b1571a8a54c12db2142b3b5f9b815
Merge: 079b0b14 b34799b2
Author: Dom Heinzeller <[email protected]>
Date:   Fri Sep 13 09:37:14 2019 -0600

    Merge pull request #247 from climbfuji/detect_machine_cheyenne_update_20190912

    gmtb/develop: update tests/detect_machine.sh for Cheyenne

commit b34799b26f6c956adc313027ffd6c51204e19acb
Author: Dom Heinzeller <[email protected]>
Date:   Thu Sep 12 21:17:03 2019 -0600

    tests/detect_machine.sh: detect cheyenne with and without .ib. in hostname

commit 4183ee576bfedea4fec85c72d3f0d8f26c2900e7
Author: Dom Heinzeller <[email protected]>
Date:   Thu Sep 12 15:38:08 2019 -0600

    ccpp/config/ccpp_prebuild_config.py: remove outdated version number from comment

commit 92d6056d8b90d98e3ae2ffca275d5767f634b449
Author: Dom Heinzeller <[email protected]>
Date:   Thu Sep 12 08:45:09 2019 -0600

    Update submodule pointer for ccpp/framework

commit 25486e833de3d752824ef391474f939daf370ca4
Author: Dom Heinzeller <[email protected]>
Date:   Thu Sep 12 08:44:55 2019 -0600

    ccpp/config/ccpp_prebuild_config.py: add new variable METADATA_HTML_OUTPUT_DIR required for metadata2html batch processing

commit d5dff46bb109e9a9c4707941013db1e47f221760
Author: Grant Firl <[email protected]>
Date:   Tue Sep 10 17:42:35 2019 -0600

    remove stochastic physics calls from suite_FV3_GFS_2017_gfdlmp_noahmp.xml

commit d114de7b13be53593146c39cacf939affda4c56f
Merge: 2c378470 079b0b14
Author: Grant Firl <[email protected]>
Date:   Tue Sep 10 17:38:07 2019 -0600

    Merge branch 'gmtb/develop' into noahmp

commit 079b0b14c8dabc0e510119de4be1ab0ec0c39dc2
Merge: c28e4020 20a8a674
Author: Dom Heinzeller <[email protected]>
Date:   Sun Sep 8 20:44:52 2019 -0600

    Merge pull request #245 from climbfuji/gmtb_develop_update_submodule_pointers_20190908

    gmtb/develop: update submodule pointers 2019/09/08

commit 20a8a674fd496965423d07936039a355df1da0ff
Author: Dom Heinzeller <[email protected]>
Date:   Sun Sep 8 20:42:53 2019 -0600

    Update submodule pointers for FV3, NEMS, ccpp-framework, ccpp-physics

commit c28e4020d3616f8b2de6d42f7b15e49a34f6e8fa
Merge: 5b120105 69b55a7c
Author: Dom Heinzeller <[email protected]>
Date:   Sun Sep 8 20:32:26 2019 -0600

    Merge pull request #243 from climbfuji/gmtb_develop_monster_merge_20190904

    Convert all schemes to new metadata, merge gsd/develop, update from Vlab, update of CCPP physics with IPD physics changes, GNU portability

commit 2c378470a94f672d726a865ee67fc18e61e5e86c
Author: Grant Firl <[email protected]>
Date:   Fri Sep 6 15:13:34 2019 -0600

    move calculation of precipitation rates for NoahMP to pre-NoahMP scheme rather than GFS_MP_generic_post for B4B testing

commit 69b55a7c7c39ece01f97fef334012cbcc5a6e079
Author: Dom Heinzeller <[email protected]>
Date:   Thu Sep 5 21:48:25 2019 -0600

    Update submodule pointer for ccpp/framework

commit e67e618bb89dd396235e5dff186d11b458169b0b
Author: Dom Heinzeller <[email protected]>
Date:   Wed Sep 4 21:18:48 2019 -0600

    Bugfix for GSD suites: replace gwdps_pre with GFS_GWD_generic_pre

commit fadc80809693b39557f8ab4b75329bbf0106c519
Merge: 64f209ce 097468ca
Author: Dom Heinzeller <[email protected]>
Date:   Wed Sep 4 16:45:51 2019 -0600

    Merge pull request #244 from climbfuji/github_master_update_submodule_pointer_FV3_20190904

    GitHub master: update submodule pointer for FV3

commit 097468ca99392eb208ee90039f3fc9b1e07ce903
Author: Dom Heinzeller <[email protected]>
Date:   Wed Sep 4 16:44:20 2019 -0600

    Update submodule pointer for FV3

commit 64f209cee5a7f364d5a93df27ad8e071f7f4682d
Merge: 45b389ae 8fe8d004
Author: Dom Heinzeller <[email protected]>
Date:   Wed Sep 4 16:41:54 2019 -0600

    Merge pull request #242 from climbfuji/github_master_update_from_vlab_master_20190904

    Update GitHub master from Vlab master 2019/09/04

commit a105b2c92f91557b74cf439740628cfbddbc845c
Author: Dom Heinzeller <[email protected]>
Date:   Wed Sep 4 15:41:35 2019 -0600

    Update submodule pointers for FV3 and ccpp/physics

commit d5c4e5faa2994d2d75c48d27121ccb5bc46b9c9e
Author: Dom Heinzeller <[email protected]>
Date:   Wed Sep 4 15:41:18 2019 -0600

    ccpp/config/ccpp_prebuild_config.py: add new dependency ccpp/physics/physics/samfaerosols.F

commit bd52519b589955ecf40e8e3dc2dfe6b087419210
Author: Grant Firl <[email protected]>
Date:   Wed Sep 4 13:36:21 2019 -0600

    add noahmp-specific rt.conf

commit 7d142eb30608f14e8a2a7995f8f37bd8b52445af
Author: Jun Wang <[email protected]>
Date:   Tue Sep 3 01:39:37 2019 +0000

    NEMSfv3gfs: this commits #refs 67151, Add aerosol transport to SAMF deep convection scheme, #refs 67152, Export instantaneous convective specific humidity tendency

    Change-Id: Iaad6fd2aef539341bccec8665a790421a1794951

commit da04e9ae8ecd7448732903b138ae29a675509b37
Author: Dom Heinzeller <[email protected]>
Date:   Wed Sep 4 11:04:02 2019 -0600

    ccpp/suites/suite_FV3_GFS_v15_gf.xml: fix merge conflict

commit 3b81d224bdb9b33a7b5743a6c7f1bd836d46cb26
Author: Dom Heinzeller <[email protected]>
Date:   Wed Sep 4 11:00:34 2019 -0600

    ccpp/suites/*.xml: remove stochastic_physics and stochastic_physics_sfc schemes from new SDFs

commit 3c247950ad1ddcfe34a3f9a53f53127309a5b3d1
Author: Dom Heinzeller <[email protected]>
Date:   Wed Sep 4 10:57:18 2019 -0600

    Update submodule pointers for ccpp/framework and ccpp/physics

commit 8fe8d0041c672b12da0e360f59ab2aa746efbef2
Author: Jun Wang <[email protected]>
Date:   Tue Sep 3 01:39:37 2019 +0000

    NEMSfv3gfs: this commits #refs 67151, Add aerosol transport to SAMF deep convection scheme, #refs 67152, Export instantaneous convective specific humidity tendency

    Change-Id: Iaad6fd2aef539341bccec8665a790421a1794951

commit 7b9cd36b2d17dfbe7693b1669077c7a07b4b8342
Author: Dom Heinzeller <[email protected]>
Date:   Wed Sep 4 10:11:08 2019 -0600

    ccpp/config/ccpp_prebuild_config.py: add sfc_cice scheme

commit 79e5cd20c3b40effb990573a013e040b03ccb08b
Merge: b9e05586 5ea40b23
Author: Dom Heinzeller <[email protected]>
Date:   Wed Sep 4 09:18:44 2019 -0600

    Merge branch 'gsd/develop' of https://github.com/NCAR/NEMSfv3gfs into gaea_update_stochastic_physics_metadata_conversion

commit b9e05586d33eac6902fb3e64d7646489c3b25fe1
Author: Dom Heinzeller <[email protected]>
Date:   Wed Sep 4 08:22:02 2019 -0600

    modulefiles/macosx.gnu/fv3: improve setting compilers so that it works when MPIF90 etc are not set

commit ceef678c966308e22ba2d08fd26a6526e6f4064b
Author: Dom Heinzeller <[email protected]>
Date:   Wed Sep 4 08:18:39 2019 -0600

    ccpp/config/ccpp_prebuild_config.py: add ccpp_types to converted metadata dictionary

commit 0aafc0a34ed6649d8e54d058f96e83712162bbda
Author: Grant Firl <[email protected]>
Date:   Tue Sep 3 15:08:02 2019 -0600

    update SDF using noahmp

commit 99626b8c2da1a22d6abb9da32cc3fc9168614ba7
Merge: 235b712b 5b120105
Author: Grant Firl <[email protected]>
Date:   Tue Sep 3 15:06:00 2019 -0600

    Merge branch 'gmtb/develop' of https://github.com/NCAR/NEMSfv3gfs into noahmp

commit caa9037ab43b21fd727d1d030b4cef9ecab7f1d3
Author: Dom Heinzeller <[email protected]>
Date:   Wed Jul 10 08:05:30 2019 -0600

    compsets/fv3.input: set variables DO_SPPT, DO_SHUM, DO_SKEB, DO_SFCPERTS for fv3_stochy test

commit bf8c2ded4b6779c9e1446507057f38a384b98167
Author: Dom Heinzeller <[email protected]>
Date:   Tue Jul 9 14:33:37 2019 -0600

    parm/input.nml.IN, tests/tests/fv3_stochy: add new namelist options do_sppt, do_shum, do_skeb, do_sfcperts to namelist template, set for fv3_stochy test

commit 85b7a7a2c32a19971ac89255f9b9d595829902a7
Author: Dom Heinzeller <[email protected]>
Date:   Tue Jul 9 14:31:48 2019 -0600

    ccpp/config/ccpp_prebuild_config.py, ccpp/suites/*: remove stochastic physics from CCPP config and suite definition files

commit d33df0427b1d6df35ff92edcab2e152d08499f15
Author: Dom Heinzeller <[email protected]>
Date:   Tue Jul 9 14:30:55 2019 -0600

    Add new submodule stochastic_physics, located in directory with same name

commit c061af4a2d8f1a8318ca40aacdfb25ccec353f15
Merge: 2483ed32 5b120105
Author: Dom Heinzeller <[email protected]>
Date:   Tue Sep 3 09:22:32 2019 -0600

    Merge branch 'gmtb/develop' of https://github.com/NCAR/NEMSfv3gfs into gaea_update_stochastic_physics_metadata_conversion

commit 5b1201056ec3187c04f7edb9d8afaac2465b4e9b
Merge: 0ab90a12 fd7d0f1e
Author: Dom Heinzeller <[email protected]>
Date:   Tue Sep 3 09:19:54 2019 -0600

    Merge pull request #239 from climbfuji/gmtb_develop_update_from_master_20190830

    gmtb/develop: update from master 2019/08/30

commit 2483ed324e53de72fa67de24cb1d04d492b1c862
Author: Dom Heinzeller <[email protected]>
Date:   Tue Sep 3 08:31:04 2019 -0600

    ccpp/build_ccpp.sh modulefiles/gaea.intel/fv3: update for gaea following wcoss_cray to avoid setting NETCDF and HDF5 in modulefile

commit 45b389ae0e2fea91bb71edcceef481601827a5c5
Merge: 257f2a2e ce0c61f6
Author: Dom Heinzeller <[email protected]>
Date:   Tue Sep 3 08:23:51 2019 -0600

    Merge pull request #240 from climbfuji/update_submodule_pointer_in_NEMSfv3gfs_20190903

    master: update submodule pointers for FV3, NEMS, WW3

commit ce0c61f644c95afc7f8dfa49ce6678d40356f54b
Author: Dom Heinzeller <[email protected]>
Date:   Tue Sep 3 08:22:21 2019 -0600

    Update submodule pointers for FV3, NEMS, WW3

commit 257f2a2eec6f9ca902c4a2aac9fd312f24470d9b
Merge: a1a54d1f 95583de5
Author: Laurie Carson <[email protected]>
Date:   Mon Sep 2 14:15:08 2019 -0600

    Merge pull request #238 from climbfuji/update_from_noaaemc_dev_20190830

    Update GitHub NCAR master from Vlab master

commit fd7d0f1e022c60a010691a51dbc866b82a48c1c1
Author: Dom Heinzeller <[email protected]>
Date:   Fri Aug 30 15:43:28 2019 -0600

    Update submodule pointer for ccpp-physics

commit 15c64a190aa7af38dce3a9154d7bae9bbe1e381c
Author: Jun Wang <[email protected]>
Date:   Tue Aug 27 20:23:05 2019 +0000

    NEMSfv3gfs: this commits #refs 66699, Calling radiative transfer parameterizations on each physics time step, #refs 66931,fix time window for accumulated fields in fv3 with IAU, #refs 62918,output omega field in fv3 history file and #65894,add WW3 component in NEMSfv3gfs(part2)

    Change-Id: I85286a530943fe67839e1afe2c94653faba27b2d

commit 50962a88c93b90930d747e1e007f10c21b042fdb
Author: Jun Wang <[email protected]>
Date:   Sun Aug 25 02:56:05 2019 +0000

    NEMSfv3gfs: this commits #refs 59519, separate FV3 dynamical core repository from FV3

    Change-Id: I0f7ba54ca67ef6aeef0d529b3005f3fef7bc3aa9

commit 5f481c8f5a36a2c0ae56dd02645bd5ec591dcc6f
Author: Dom Heinzeller <[email protected]>
Date:   Fri Aug 30 15:33:53 2019 -0600

    Update .gitmodules to point to NCAR repro and branch gmtb/develop for new submodule WW3, update submodule pointer

commit 872693bea7e160edb478b05cfb68e96e3671ec8c
Author: Jessica.Meixner <[email protected]>
Date:   Tue Aug 20 12:10:34 2019 +0000

    NEMSfv3gfs: this commit refs #65894, adding WW3

commit 95583de5ce6c4cbb83abe35b969476ba88628cca
Author: Dom Heinzeller <[email protected]>
Date:   Fri Aug 30 11:29:32 2019 -0600

    Add default branch develop for WW3 and update submodule pointer

commit a2fc15ec2140ef5b5e80dc4bd4b2d88430a9d59d
Author: Jun Wang <[email protected]>
Date:   Tue Aug 27 20:23:05 2019 +0000

    NEMSfv3gfs: this commits #refs 66699, Calling radiative transfer parameterizations on each physics time step, #refs 66931,fix time window for accumulated fields in fv3 with IAU, #refs 62918,output omega field in fv3 history file and #65894,add WW3 component in NEMSfv3gfs(part2)

    Change-Id: I85286a530943fe67839e1afe2c94653faba27b2d

commit c15a5f76e78c08c68e3afb0d5e5194f55dcdd855
Author: Jun Wang <[email protected]>
Date:   Sun Aug 25 02:56:05 2019 +0000

    NEMSfv3gfs: this commits #refs 59519, separate FV3 dynamical core repository from FV3

    Change-Id: I0f7ba54ca67ef6aeef0d529b3005f3fef7bc3aa9

commit 100dc801cb2901618f778720c6ad4426070ce4c7
Author: Dom Heinzeller <[email protected]>
Date:   Fri Aug 30 10:17:47 2019 -0600

    Update .gitmodules so that WW3 points to the NCAR repository

commit 25aa6fe7d55240903a132fc81dd9fb9b8aca0631
Author: Jessica.Meixner <[email protected]>
Date:   Tue Aug 20 12:10:34 2019 +0000

    NEMSfv3gfs: this commit refs #65894, adding WW3

commit 0ab90a125b63b89d29bb73e0f640000b6c5b74a0
Merge: cee7b3fe b77f1471
Author: Dom Heinzeller <[email protected]>
Date:   Fri Aug 30 07:42:51 2019 -0600

    Merge pull request #237 from climbfuji/gmtb_develop_update_submodule_pointers_20190830_1

    gmtb/develop: update submodule pointers for FV3, NEMS, ccpp/framework, ccpp/physics

commit b77f14719ac17b564f85d69383c99b8b161de817
Author: Dom Heinzeller <[email protected]>
Date:   Fri Aug 30 07:40:55 2019 -0600

    Update submodule pointers for FV3, NEMS, ccpp/framework, ccpp/physics

commit cee7b3fe354e07a29dbc9fa49235d909f4cf528a
Merge: f4671a1c d02d0c16
Author: Dom Heinzeller <[email protected]>
Date:   Fri Aug 30 07:39:01 2019 -0600

    Merge pull request #234 from climbfuji/bugfix_rt_conf

    gmtb/develop: bugfix for rt.conf

commit a1a54d1f5159aa927ac4d4b01abaa6548040ff52
Merge: 8a1e235d ed0c436a
Author: Dom Heinzeller <[email protected]>
Date:   Thu Aug 29 15:32:08 2019 -0600

    Merge pull request #236 from grantfirl/master

    add Julie as codeowner

commit ed0c436ad2aa1c52a6a6a5fa1ca7a9fc210956d3
Author: Grant Firl <[email protected]>
Date:   Thu Aug 29 15:00:06 2019 -0600

    add Julie as codeowner

commit f4671a1c9f394609f33e0f04a893ab1a8147f8bb
Merge: ea22d467 a667940f
Author: Dom Heinzeller <[email protected]>
Date:   Thu Aug 29 14:58:30 2019 -0600

    Merge pull request #235 from grantfirl/gmtb/develop

    add Julie as codeowner

commit a667940f9cfff12e41cb5134f86e66babc76081f
Author: Grant Firl <[email protected]>
Date:   Thu Aug 29 14:52:42 2019 -0600

    add Julie as codeowner

commit d02d0c1679ffd3b57b2d2d049f86efdc531b51fd
Author: Dom Heinzeller <[email protected]>
Date:   Thu Aug 29 08:06:47 2019 -0600

    Update submodule pointers for FV3 NEMS ccpp/physics

commit 65238966d6c574b314bc1b0e487e59f78d9b5552
Author: Dom Heinzeller <[email protected]>
Date:   Thu Aug 29 08:06:02 2019 -0600

    tests/rt.conf: bugfix, correct name of suite FV3_GFS_2017_gfdlmprad_gwd to FV3_GFS_2017_gfdlmp_gwd

commit 282ad6b657ff250eefba8092975b484681479d8a
Author: Qingfu Liu <[email protected]>
Date:   Wed Aug 28 19:53:24 2019 +0000

    correct file content tests/tests/fv3_ccpp_gfs_myj

commit 29993a06e32ea08b588bed93d7e088ea386daf7c
Author: Qingfu Liu <[email protected]>
Date:   Wed Aug 28 19:49:13 2019 +0000

    correct file content tests/tests/fv3_ccpp_gfs_myj

commit 5ea40b233539ed0b8a86d8ef54ed59cba4c89621
Merge: e06da970 01f6a0f8
Author: Dom Heinzeller <[email protected]>
Date:   Tue Aug 27 16:40:43 2019 -0600

    Merge pull request #232 from climbfuji/gsd_develop_update_submodule_pointers_20190827

    gsd/develop: update submodule pointers for FV3 and ccpp/physics

commit 01f6a0f87727751a8252a576bceed7bc5b9729a7
Author: Dom Heinzeller <[email protected]>
Date:   Tue Aug 27 16:39:04 2019 -0600

    Update submodule pointers for FV3 and ccpp/physics

commit ea22d467cb4e6289768ca625133f68d803cc4772
Merge: 8ba3d4ae 3150c4b4
Author: Dom Heinzeller <[email protected]>
Date:   Tue Aug 27 16:32:58 2019 -0600

    Merge pull request #231 from climbfuji/update_submodule_pointers_gmtb_develop_20190827

    gmtb/develop: update submodule pointers for FV3, NEMS, ccpp/physics

commit 3150c4b432ac93119c578a1d0cb12a7b0e1e3fd6
Author: Dom Heinzeller <[email protected]>
Date:   Tue Aug 27 16:30:38 2019 -0600

    Update submodule pointers for FV3 NEMS ccpp/physics

commit 8ba3d4ae5bfd786895d54c26f398821d28601821
Merge: 6aba4551 f91d7846
Author: Dom Heinzeller <[email protected]>
Date:   Tue Aug 27 15:45:47 2019 -0600

    Merge pull request #230 from climbfuji/hera_update_20190827

    gmtb/develop: update hera config, enable regression testing

commit f91d78461f1608bfdbe1de0d38a9527f8381cea7
Author: Dom Heinzeller <[email protected]>
Date:   Tue Aug 27 15:04:42 2019 -0600

    Update submodule pointer for NEMS

commit 235b712ba15d59c4d56aa990d3a9edeb30999496
Author: Grant Firl <[email protected]>
Date:   Mon Aug 26 15:40:00 2019 -0600

    add CCPP-based NoahMP test to RTs for rt_ccpp_gmtb.conf (plus new SDF based on suite_FV3_GFS_2017_gfdlmp.xml)

commit e74e0d2ebd6430e39c2f869c7241be858776f385
Author: Dom Heinzeller <[email protected]>
Date:   Tue Aug 27 13:53:55 2019 -0600

    Update submodule pointer for NEMS

commit 290a484f1e3be763aced44c2dfd099c6d832ce2e
Author: Dom Heinzeller <[email protected]>
Date:   Tue Aug 27 13:53:33 2019 -0600

    rt_hera_test.conf: temporary regression test config for testing on hera

commit c52d27b0d8e99029d1e723ecde07d3ba0813da09
Author: Dom Heinzeller <[email protected]>
Date:   Tue Aug 27 13:52:20 2019 -0600

    tests/rt.sh: enable regression testing with rt.sh, changes from Ratko, update STMP and PTMP for GMTB

commit 6b9508c93f81383418ad1b779ce8d4bc2358aba4
Author: Dom Heinzeller <[email protected]>
Date:   Tue Aug 27 13:48:01 2019 -0600

    modulefiles/hera.intel/fv3: update path to modules following transition from Juno test system to hera

commit eb229d3cb145a7811d69348cb2167fbd5a5a097d
Author: Dom Heinzeller <[email protected]>
Date:   Tue Aug 27 13:47:33 2019 -0600

    ccpp/build_ccpp.sh and conf/configure.fv3.hera.intel: don't use the AVX512 flags yet on hera

commit 6aba455146a4a8d4629704e5226570116091691f
Merge: 5c289838 75b5715e
Author: Dom Heinzeller <[email protected]>
Date:   Tue Aug 27 07:17:17 2019 -0600

    Merge pull request #229 from climbfuji/gmtb_develop_update_submodule_20190827

    gmtb/develop: update submodule pointers for FV3, ccpp-physics, ccpp-framework

commit 75b5715e41c6f978defaa04e38d41b8d20326488
Author: Dom Heinzeller <[email protected]>
Date:   Tue Aug 27 07:15:40 2019 -0600

    Update submodule pointer for FV3, ccpp-physics, ccpp-framework

commit 5c289838c1bc9d443319e4cdad4b23df5215fc77
Merge: eb7b7ae8 48000a93
Author: Dom Heinzeller <[email protected]>
Date:   Tue Aug 27 07:13:20 2019 -0600

    Merge pull request #218 from bluefinweiwei/cires_ugwp

    UGWP Version 0

commit 51299cadc6514199225605df04246e54446d6486
Author: Qingfu Liu <[email protected]>
Date:   Tue Aug 27 11:03:43 2019 +0000

    add MYJ surface layer scheme and MYJ PBL scheme to CCPP

commit 75596f60b0d684413cf8a6f42567cd92195b3661
Author: Grant Firl <[email protected]>
Date:   Mon Aug 26 13:34:15 2019 -0600

    remove noahmp_pre.F90 from CCPP config since code was moved to existing interstitial scheme

commit d913c69b7e11b23f5666c4f8432a6e0c80fe7036
Author: Grant Firl <[email protected]>
Date:   Fri Aug 23 18:04:07 2019 -0600

    remove module_wrf_utl.f90 from ccpp_prebuild_config scheme dependency list

commit 48000a93dfd9b07159d3d006016c854e14a20886
Merge: 266c9885 e38eda36
Author: Weiwei <[email protected]>
Date:   Fri Aug 23 13:53:36 2019 -0600

    Merge pull request #3 from climbfuji/ugwd_dom

    Delete tests/rt_with_ugwp.conf

commit e38eda36b8d1d7181400059c3d7cc1af3511534d
Merge: 789ab6bc 266c9885
Author: Dom Heinzeller <[email protected]>
Date:   Fri Aug 23 13:50:02 2019 -0600

    Merge branch 'cires_ugwp' of https://github.com/bluefinweiwei/NEMSfv3gfs into ugwd_dom

commit 789ab6bc2ae4e04cdeaa580abbb0cbcf0c4d9cb7
Author: Dom Heinzeller <[email protected]>
Date:   Fri Aug 23 13:48:59 2019 -0600

    Delete tests/rt_with_ugwp.conf

commit 266c9885f97153154fa6caaaf0bbac87addda38b
Author: Weiwei <[email protected]>
Date:   Fri Aug 23 13:47:39 2019 -0600

    Delete rt_ugwp_only.conf

    Removed.

commit b27a08aa3d2544024a0fb6adf0a38d339ac3a18c
Merge: 5ffd3ed8 adc9df36
Author: Weiwei <[email protected]>
Date:   Fri Aug 23 13:22:27 2019 -0600

    Merge pull request #2 from climbfuji/ugwd_dom

    UGWD update for open PR to gmtb/develop

commit adc9df36fbe5a866bcab77f709d368286d2a496d
Author: Dom Heinzeller <[email protected]>
Date:   Fri Aug 23 11:59:13 2019 -0600

    Update submodule pointers

commit 4410e285d4f0a99a68f2da799fa8d9d117e8ef1a
Merge: 1499c32c eb7b7ae8
Author: Dom Heinzeller <[email protected]>
Date:   Fri Aug 23 09:34:58 2019 -0600

    Merge branch 'gmtb/develop' of https://github.com/NCAR/NEMSfv3gfs into ugwd_dom

commit 1499c32c862f4471a72c85204208d01c847f8d2d
Author: Dom Heinzeller <[email protected]>
Date:   Fri Aug 23 09:11:39 2019 -0600

    Update submodule pointers for FV3, ccpp/framework, ccpp/physics

commit 7ae19849e6a6365adab0642e674a3e16b04cbbf3
Author: Dom Heinzeller <[email protected]>
Date:   Fri Aug 23 09:10:02 2019 -0600

    ccpp/suites/*: rename gwdps_pre to GFS_GWD_generic_pre, remove duplicate UGWP test, rename CCPP UGWP test to match existing IPD test, update regression test configuration files

commit eb7b7ae83ec4cc828f640810512ab528f6f7a7c8
Merge: 91e485ca cb382b97
Author: Dom Heinzeller <[email protected]>
Date:   Fri Aug 23 07:52:49 2019 -0600

    Merge pull request #226 from climbfuji/update_submodule_pointers_gmtb_develop_20190823

    gmtb/develop: update submodule pointers for FV3 and ccpp/framework 2019/08/23

commit cb382b97d933d35a3b40ab7dfa6428079ce810fa
Author: Dom Heinzeller <[email protected]>
Date:   Fri Aug 23 07:50:39 2019 -0600

    Update submodule pointers for FV3 and ccpp/framework

commit 91e485cac197bb902c095770526dfc503f09f028
Merge: 00168544 4741dd2f
Author: Dom Heinzeller <[email protected]>
Date:   Fri Aug 23 07:49:50 2019 -0600

    Merge pull request #225 from climbfuji/github_gmtb_develop_wcoss_cray_gaea_jet_cheyenne_20190819

    gmtb/develop: bugfixes and updates for wcoss/cray/gaea/jet/cheyenne

commit 4741dd2f282be63e8e3ceefc0c1e8ad3deb487ab
Author: Dom Heinzeller <[email protected]>
Date:   Thu Aug 22 16:25:46 2019 -0600

    modulefiles/theia.intel/fv3: workaround to fix EMC's broken yaml-cpp module

commit 0dd206c8fc9afde82ee11d145e6021e5e36a6d10
Author: Grant Firl <[email protected]>
Date:   Thu Aug 22 13:40:10 2019 -0600

    add noahmp_pre.F90 to ccpp_prebuild_config.py

commit 816818024cc3b512794a1a5507628157de8ce23d
Author: Dom Heinzeller <[email protected]>
Date:   Thu Aug 22 12:25:37 2019 -0600

    doc/README_CENTOS.txt, modulefiles/linux.gnu/fv3: final touches on documentation, update of modulefiles to match the defaults in READ_{CENTOS,UBUNTU}.txt (strictly speaking not necessary)

commit 0ab1908a4add63af6250cb33a68505a618a24202
Author: Dom Heinzeller <[email protected]>
Date:   Wed Aug 21 23:20:31 2019 -0600

    doc/README_UBUNTU.txt: add missing environment variables for buidling CCPP version on UBUNTU

commit 6624c650b92a10bbbe282bf8f3f10618c4d4630c
Author: Dom Heinzeller <[email protected]>
Date:   Wed Aug 21 22:09:22 2019 -0600

    doc/README_{UBUNTU,CENTOS,MACOSX_*}.txt: add note that 'make check' for ESMF is optional and that it can take a very long time

commit 3893de3c5e7e7c561017cf0a6428c41ac07f7704
Author: Grant Firl <[email protected]>
Date:   Wed Aug 21 11:39:39 2019 -0600

    update ccpp_prebuild_config.py for NoahMP

commit 7415d501a0d9ba1a1ef4235f15141cf254ca4eb5
Merge: 00168544 5ffd3ed8
Author: Dom Heinzeller <[email protected]>
Date:   Wed Aug 21 14:24:29 2019 -0600

    Merge branch 'cires_ugwp' of https://github.com/bluefinweiwei/NEMSfv3gfs into HEAD

commit 6151127e825b9c195a2e3088989910359736355f
Author: Dom Heinzeller <[email protected]>
Date:   Wed Aug 21 12:29:42 2019 -0600

    doc/README_CENTOS.txt: instructions for configuring CentOS linux systems, similar to the existing Ubuntu Linux instructions

commit 0d4dd7f1140793f21e9bb7989cf4ada7fb4f0245
Author: Dom Heinzeller <[email protected]>
Date:   Wed Aug 21 11:27:38 2019 -0600

    doc/README_UBUNTU.txt: update documentation to latest state, simplify installation of netCDF

commit 1d9d132652c04a8c1c7ad8f4393cb7fb7d31f4a1
Author: Dom Heinzeller <[email protected]>
Date:   Wed Aug 21 11:25:51 2019 -0600

    tests/rt_ccpp_gmtb.conf: remove PGI compiler, this allows to remove multiple entires for runs that only work with Intel and GNU; remove wcoss entries (no dynamic builds), replace gaea (no dynamic build) with jet

commit f9ebd24090c38d5037d315f42b5b2f7029c0437b
Author: Dom Heinzeller <[email protected]>
Date:   Wed Aug 21 10:03:28 2019 -0600

    tests/rt_ccpp_gsd.conf: enable running GSD regression tests on other platforms than theia (regression test baseline now contains GSD input data)

commit b2a7580ced26fd21dd9b98688911747e89cfe10b
Author: Dom Heinzeller <[email protected]>
Date:   Wed Aug 21 10:00:35 2019 -0600

    tests/rt_ccpp_caps.conf: remove PGI

commit d1e2b5c8f23cb5c1c78186270b39c54540fcf8c6
Author: Dom Heinzeller <[email protected]>
Date:   Wed Aug 21 09:58:44 2019 -0600

    conf/before_components.mk, modulefiles/theia.gnu/fv3: replace user-based libraries with modules located in GMTB project directory, enable theia.gnu for testing

commit ffc518e0f4f139fcab0b4138f5e4b78542c25f06
Author: Dom Heinzeller <[email protected]>
Date:   Wed Aug 21 09:56:32 2019 -0600

    Update ESMF on Cheyenne, Gaea, Jet; for Jet, also remove the module load anaconda (no longer needed and causes ccpp_prebuild.py to fail)

commit e64d8a75dd76d5bed8c1eb47b2ae74b18ee6006f
Merge: 00168544 73d2015f
Author: Dom Heinzeller <[email protected]>
Date:   Mon Aug 19 09:54:08 2019 -0600

    Merge branch 'gaea_jet_cheyenne_support_and_initial_gnu_tests' of gerrit:NEMSfv3gfs into HEAD

commit e06da970ac5fb5f177fe6e3baba9ff70e3f44ebd
Merge: 18cd1a1a 175971b7
Author: Dom Heinzeller <[email protected]>
Date:   Mon Aug 19 08:53:49 2019 -0600

    Merge pull request #224 from climbfuji/gsd_develop_update_submodule_pointers_20190819

    gsd/develop update submodule pointers 2019/08/19

commit 175971b7d34d49c371f659adeba870b91628cbc0
Author: Dom Heinzeller <[email protected]>
Date:   Mon Aug 19 08:51:05 2019 -0600

    Update submodule pointer for FV3

commit 18cd1a1aadddb816b0f8903fcb059556768b001b
Merge: 9fbef48c f663d381
Author: Dom Heinzeller <[email protected]>
Date:   Mon Aug 19 08:48:34 2019 -0600

    Merge pull request #221 from climbfuji/update_gsd_develop_with_vlab_master_20190816

    Update GitHub gsd/develop with Vlab master 2019/08/16

commit 001685445cb219fd9c4565c0bc3885800c68f813
Merge: 50eca7bb df214421
Author: Dom Heinzeller <[email protected]>
Date:   Sat Aug 17 16:32:16 2019 -0600

    Merge pull request #223 from climbfuji/gmtb_develop_update_submodule_pointers_20190817

    gmtb/develop: update submodule pointers 2019/08/17

commit df214421c29449cb1f800ed33ffe3b7a9a87663c
Author: Dom Heinzeller <[email protected]>
Date:   Sat Aug 17 16:30:49 2019 -0600

    Update submodule pointers for FV3 NEMS ccpp/framework ccpp/physics

commit 50eca7bbf0cb47d05591286257acd6c5fa4c2edc
Merge: de32c317 1686f099
Author: Dom Heinzeller <[email protected]>
Date:   Sat Aug 17 16:28:47 2019 -0600

    Merge pull request #212 from climbfuji/make_linux_and_macos_modulefiles_flexible

    Generalize and update Linux modulefile, add support for hera

commit f663d38105a20ddc0c60467493972c47f0aa35e9
Merge: 2a1c6868 9fbef48c
Author: Dom Heinzeller <[email protected]>
Date:   Fri Aug 16 13:47:46 2019 -0600

    Merge branch 'gsd/develop' of https://github.com/NCAR/NEMSfv3gfs into update_gsd_develop_with_vlab_master_20190816

commit 9fbef48c1b10bc348acc0ca073248e04be7da5b5
Merge: f86243fc a995462a
Author: Dom Heinzeller <[email protected]>
Date:   Fri Aug 16 13:47:28 2019 -0600

    Merge pull request #217 from climbfuji/update_submodule_pointer_NEMS_20190816

    gsd/develop: update submodule pointer for new NEMS repo

commit 2a1c6868319d02394b1012be4fe4bd05bc77d1cd
Author: Ratko Vasic <[email protected]>
Date:   Thu Aug 8 16:59:04 2019 +0000

    Fix hash to NEMS

commit 9dfe49f9e0d244186055e272d1298f1e3b18c799
Author: Ratko Vasic <[email protected]>
Date:   Thu Aug 8 16:29:12 2019 +0000

    NEMS transitioned from Vlab to GitHub.

    NEMS documentation is moved to separate repository: NEMS-doc (https://github.com/NOAA-EMC/NEMS-doc)

    Clone NEMS from new repository:
    git clone https://github.com/NOAA-EMC/NEMS

commit ef403be0795a476ead513ff57ffb5c1f9e75aa7a
Author: Ratko Vasic <[email protected]>
Date:   Mon Aug 5 18:34:48 2019 +0000

    This commit is part of group effort (Raffaele, Gerhard, Jessica, Dusan and Ratko):

    Summary of updates in NEMS commit:
    -- removes 'module load slurm' commands
    -- Gerhard's mediator updates to run large number of ATM/MED pets and have
       ATM/MED on diff PETs: https://vlab.ncep.noaa.gov/redmine/issues/66089
    -- Increase path size of directory for building WW3 error capturing in
       NEMS: https://vlab.ncep.noaa.gov/redmine/issues/65507
    -- Other WW3 NEMS makefile updates: https://vlab.ncep.noaa.gov/redmine/issues/66085
    -- Raffaele's change on debug mode of compiling fv3_gsdchem

    Applications:

    EMC_FV3-MOM6-CICE5
    -- Updated module to point to ESMFv8.0.0bs40
    EMC_FV3-GSDCHEM-WW3
    -- Updatedmodule on theia to point to ESMFv8.0.0bs32 on theia and 40 on phase2 (Does not run GSDCHEM compsets here).
    EMC_FV3GFS-GSDCHEM
    -- Updates to run ESMFv8.0.0bs40 required pointing to the GSDCHEM 0.8.7_esmf8_bs40 branch, updating modules to point
       to ESMFv8.0.0bs40 and creating a baseline on WCOSS Cray
    NEMSfv3gfs
    -- Updated module to point to ESMFv8.0.0bs40

commit 9d7ddc8695a0742b4865582dabc38824b0b321b6
Author: Dusan Jovic <[email protected]>
Date:   Fri Jul 26 13:51:22 2019 +0000

    Fix an issue introduced in the previous commit where merge conflict markers were inadvertently committed.

    Change-Id: Ib699fae7acda1bbb2cf6475461b41d09a6fba578

commit b53a1117e79822ae158ba8e7f41f10b206c1bf68
Author: Jun.Wang <[email protected]>
Date:   Sat Jul 20 14:17:26 2019 +0000

    NEMSfv3gfs: This commits #refs 65267 adding obs-based roughness length formulations and adjustment of K for hurricane simulations

    Change-Id: Ia71047b284c7ae09b390a2e375b8cba94fdfc46c

commit 1686f099f08ac2a7f0909c292619fededec1aabf
Author: Dom Heinzeller <[email protected]>
Date:   Fri Aug 16 12:25:22 2019 -0600

    Update submodule pointer for ccpp/framework

commit 4418499222f2ba30e6965160c4e59493baa202d4
Merge: 0cd98189 4524ac0b
Author: Dom Heinzeller <[email protected]>
Date:   Fri Aug 16 12:03:42 2019 -0600

    Merge branch 'update_gmtb_develop_from_vlab_master_20190816' of https://github.com/climbfuji/NEMSfv3gfs into make_linux_and_macos_modulefiles_flexible

commit 4524ac0b7a4982d28e50c6cebb1b514ce6ac7b7e
Author: Ratko Vasic <[email protected]>
Date:   Thu Aug 8 16:59:04 2019 +0000

    Fix hash to NEMS

commit 8b114af25680fe70f8d302cac8c087e87c472b14
Author: Ratko Vasic <[email protected]>
Date:   Thu Aug 8 16:29:12 2019 +0000

    NEMS transitioned from Vlab to GitHub.

    NEMS documentation is moved to separate repository: NEMS-doc (https://github.com/NOAA-EMC/NEMS-doc)

    Clone NEMS from new repository:
    git clone https://github.com/NOAA-EMC/NEMS

commit f61d93cb6fb5bf5bacf6c0d6c38429bb71af4f71
Author: Ratko Vasic <[email protected]>
Date:   Mon Aug 5 18:34:48 2019 +0000

    This commit is part of group effort (Raffaele, Gerhard, Jessica, Dusan and Ratko):

    Summary of updates in NEMS commit:
    -- removes 'module load slurm' commands
    -- Gerhard's mediator updates to run large number of ATM/MED pets and have
       ATM/MED on diff PETs: https://vlab.ncep.noaa.gov/redmine/issues/66089
    -- Increase path size of directory for building WW3 error capturing in
       NEMS: https://vlab.ncep.noaa.gov/redmine/issues/65507
    -- Other WW3 NEMS makefile updates: https://vlab.ncep.noaa.gov/redmine/issues/66085
    -- Raffaele's change on debug mode of compiling fv3_gsdchem

    Applications:

    EMC_FV3-MOM6-CICE5
    -- Updated module to point to ESMFv8.0.0bs40
    EMC_FV3-GSDCHEM-WW3
    -- Updatedmodule on theia to point to ESMFv8.0.0bs32 on theia and 40 on phase2 (Does not run GSDCHEM compsets here).
    EMC_FV3GFS-GSDCHEM
    -- Updates to run ESMFv8.0.0bs40 required pointing to the GSDCHEM 0.8.7_esmf8_bs40 branch, updating modules to point
       to ESMFv8.0.0bs40 and creating a baseline on WCOSS Cray
    NEMSfv3gfs
    -- Updated module to point to ESMFv8.0.0bs40

commit 8a1e235d57a13d40e8ba71d3ce174d62dfa4f265
Merge: 62eaf40f 8ad5805c
Author: Dom Heinzeller <[email protected]>
Date:   Fri Aug 16 11:41:45 2019 -0600

    Merge pull request #220 from climbfuji/update_submodule_pointer_fv3

    Update submodule pointer for FV3

commit 8ad5805ce1c219f8ba2962d6e84403dd11dfeac2
Author: Dom Heinzeller <[email protected]>
Date:   Fri Aug 16 11:40:27 2019 -0600

    Update submodule pointer for FV3

commit 62eaf40f77eae7a450913e9aba54b967da37b9b3
Merge: a8e8fa58 f6eae698
Author: Dom Heinzeller <[email protected]>
Date:   Fri Aug 16 11:36:46 2019 -0600

    Merge pull request #219 from climbfuji/update_github_master_from_vlab_master_20190816

    Update GitHub master from Vlab master 2019/08/16

commit f6eae698bf40e3adca1b8d95417cb2c8905df7f4
Author: Ratko Vasic <[email protected]>
Date:   Thu Aug 8 16:59:04 2019 +0000

    Fix hash to NEMS

commit dcc7a818cc8a19c0ec894015c4bd0d8003768fc0
Author: Ratko Vasic <[email protected]>
Date:   Thu Aug 8 16:29:12 2019 +0000

    NEMS transitioned from Vlab to GitHub.

    NEMS documentation is moved to separate repository: NEMS-doc (https://github.com/NOAA-EMC/NEMS-doc)

    Clone NEMS from new repository:
    git clone https://github.com/NOAA-EMC/NEMS

commit a9cb714dd858e2d46571cd89776275847e2ced7a
Author: Ratko Vasic <[email protected]>
Date:   Mon Aug 5 18:34:48 2019 +0000

    This commit is part of group effort (Raffaele, Gerhard, Jessica, Dusan and Ratko):

    Summary of updates in NEMS commit:
    -- removes 'module load slurm' commands
    -- Gerhard's mediator updates to run large number of ATM/MED pets and have
       ATM/MED on diff PETs: https://vlab.ncep.noaa.gov/redmine/issues/66089
    -- Increase path size of directory for building WW3 error capturing in
       NEMS: https://vlab.ncep.noaa.gov/redmine/issues/65507
    -- Other WW3 NEMS makefile updates: https://vlab.ncep.noaa.gov/redmine/issues/66085
    -- Raffaele's change on debug mode of compiling fv3_gsdchem

    Applications:

    EMC_FV3-MOM6-CICE5
    -- Updated module to point to ESMFv8.0.0bs40
    EMC_FV3-GSDCHEM-WW3
    -- Updatedmodule on theia to point to ESMFv8.0.0bs32 on theia and 40 on phase2 (Does not run GSDCHEM compsets here).
    EMC_FV3GFS-GSDCHEM
    -- Updates to run ESMFv8.0.0bs40 required pointing to the GSDCHEM 0.8.7_esmf8_bs40 branch, updating modules to point
       to ESMFv8.0.0bs40 and creating a baseline on WCOSS Cray
    NEMSfv3gfs
    -- Updated module to point to ESMFv8.0.0bs40

commit f595bcfeb55c8503c3b3f290cb22169fe75b6065
Author: Dusan Jovic <[email protected]>
Date:   Fri Jul 26 13:51:22 2019 +0000

    Fix an issue introduced in the previous commit where merge conflict markers were inadvertently committed.

    Change-Id: Ib699fae7acda1bbb2cf6475461b41d09a6fba578

commit ae772b0a3fc5b75e3814f2328ac35139d4f4ed99
Author: Jun.Wang <[email protected]>
Date:   Sat Jul 20 14:17:26 2019 +0000

    NEMSfv3gfs: This commits #refs 65267 adding obs-based roughness length formulations and adjustment of K for hurricane simulations

    Change-Id: Ia71047b284c7ae09b390a2e375b8cba94fdfc46c

commit 5ffd3ed8b437297040036dc835f86142da8e8879
Author: Weiwei <[email protected]>
Date:   Wed Jul 3 12:23:33 2019 -0600

    	new file:   ccpp/config/ccpp_prebuild_config.py
            new file:   ccpp/suites/suite_FV3_GFS_2017_gfdlmp_ugwp.xml
    	new file:   parm/ccpp_ugwp.nml.IN
            new file:   tests/rt_ugwp_only.conf
    	new file:   tests/rt_ugwp.sh
            new file:   tests/tests/fv3_gfdlmprad_ugwp
    	new file:   tests/tests/fv3_ccpp_gfdlmprad_ugwp
            new file:   tests/rt_ccpp_gmtb_with_ugwp.conf
            new file:   tests/rt_with_ugwp.conf

commit a995462aeaa86ce2640c75b77847ff2b9709ce5e
Author: Dom Heinzeller <[email protected]>
Date:   Fri Aug 16 10:08:44 2019 -0600

    Update submodule pointer for NEMS following transition of EMC NEMS repo Vlab -> GitHub

commit 0cd98189b738c9d1fafef7d8c8011582fb6b677b
Author: Dom Heinzeller <[email protected]>
Date:   Fri Aug 16 09:55:10 2019 -0600

    Update submodule pointer for NEMS following the move from Vlab to GitHub

commit ac49f23be40492ab8908161e5c48cd7f10f6c499
Merge: a2c18766 de32c317
Author: Dom Heinzeller <[email protected]>
Date:   Fri Aug 16 09:22:54 2019 -0600

    Merge branch 'gmtb/develop' of https://github.com/NCAR/NEMSfv3gfs into make_linux_and_macos_modulefiles_flexible

commit de32c31777ff15f0f850bcdc6b7f18e049993794
Merge: ddcf419d d1932bc5
Author: Laurie Carson <[email protected]>
Date:   Fri Aug 16 09:20:59 2019 -0600

    Merge pull request #216 from llpcarson/update_nems

    Update submodule pointers for new NEMS repo

commit d1932bc52876a361c3234b00bb0a18aa6039babd
Author: Laurie Carson <[email protected]>
Date:   Fri Aug 16 09:15:13 2019 -0600

    Update submodule pointers for new NEMS repo

commit a8e8fa58e812f2d0fe88e05c8d278af55ff5a4bf
Author: Laurie Carson <[email protected]>
Date:   Fri Aug 16 09:11:32 2019 -0600

    Update submodule pointers for new NEMS repo, latest ccpp-physics (some SCM updates only)

commit a2c1876603ea7b6e959d85d27566f7350a8398a2
Author: Dom Heinzeller <[email protected]>
Date:   Thu Aug 15 17:20:11 2019 -0600

    Update submodule pointers for FV3, NEMS, ccpp-framework, ccpp-physics

commit 30a41c84d22b9f8bf03d086017c2e5d420531c59
Merge: 42be0321 ddcf419d
Author: Dom Heinzeller <[email protected]>
Date:   Thu Aug 15 17:17:29 2019 -0600

    Merge branch 'gmtb/develop' of https://github.com/NCAR/NEMSfv3gfs into make_linux_and_macos_modulefiles_flexible

commit 42be032139a3e79533fd78f22fedac23efacd056
Author: Dom Heinzeller <[email protected]>
Date:   Wed Aug 14 16:49:50 2019 -0600

    Add support for hera with juno (hera test system) modifications

commit ddcf419d388d2d181aa0a9a5a2f6a9e19d41539e
Merge: 1b5d9cc1 9884db3d
Author: Dom Heinzeller <[email protected]>
Date:   Thu Aug 15 12:01:13 2019 -0600

    Merge pull request #213 from climbfuji/update_macosx_20190811

    Update MacOSX build configuration 2019/08/11

commit 9884db3d03067c73ec6f66e23769d8af25fef7a4
Author: Dom Heinzeller <[email protected]>
Date:   Wed Aug 14 08:33:39 2019 -0600

    Update submodule pointer for FV3

commit 450819fcfcbdb624b33aa4e41fb3efcdccefd727
Author: Dom Heinzeller <[email protected]>
Date:   Wed Aug 14 08:33:28 2019 -0600

    doc/README_MACOSX_clanggfortran.txt, doc/README_MACOSX_gccgfortran.txt: final touches to install/setup documentation

commit e46d3256356a971cbcaa3ce520327115a485a1ad
Author: Dom Heinzeller <[email protected]>
Date:   Tue Aug 13 14:48:14 2019 -0600

    doc/README_MACOSX_*.txt: add missing command for ESMF

commit 730bdfe2286a51857b32da2a999a6f8ab51e5653
Author: Dom Heinzeller <[email protected]>
Date:   Tue Aug 13 10:27:56 2019 -0600

    Update submodule pointer for FV3

commit 99ef65ba88b8964bd6c757dab910710e0f0f7107
Author: Dom Heinzeller <[email protected]>
Date:   Tue Aug 13 10:27:41 2019 -0600

    doc/README_MACOSX_clanggfortran.txt: remove TODO reminders

commit e2c20771b790076a9c932b45ba0d30204914e059
Author: Dom Heinzeller <[email protected]>
Date:   Mon Aug 12 21:03:30 2019 -0600

    modulefiles/macosx.gnu/fv3: update library versions, adjust to work with both clang+gfortran and gcc+gfortran

commit 1b3302e8cdc59aa302240a0b03ecb34141202ac5
Author: Dom Heinzeller <[email protected]>
Date:   Mon Aug 12 21:02:43 2019 -0600

    doc/README_MACOSX_*: work in progress, saving progress

commit 35d02b6f2842d5951122777ddb6630f4ea68959d
Author: Dom Heinzeller <[email protected]>
Date:   Mon Aug 12 21:01:46 2019 -0600

    Update submodule pointer for FV3

commit f86243fce831151f943d64b7bba62391a03f89af
Merge: c7dc843e a0596c86
Author: Dom Heinzeller <[email protected]>
Date:   Mon Aug 12 20:30:08 2019 -0600

    Merge pull request #215 from climbfuji/update_submodule_pointer_20190812

    Update submodule pointer for ccpp/physics

commit a0596c864d8936f39a05ff314eb593211aae0b65
Author: Dom Heinzeller <[email protected]>
Date:   Mon Aug 12 20:28:33 2019 -0600

    Update submodule pointer for ccpp/physics

commit c7dc843eb975f33368037556d0ecf6a55649584f
Merge: 1ebaea29 3d01b44f
Author: Dom Heinzeller <[email protected]>
Date:   Mon Aug 12 20:26:50 2019 -0600

    Merge pull request #214 from climbfuji/bugfix_ruclsm_ice_temp_landmask_landuse_inconsistencies

    Bugfix RUC LSM ice temperature initialization, appbuild bugfix

commit ceef93aa0a0f5815bbca4a5fbf281bb4d9adefbe
Author: Dom Heinzeller <[email protected]>
Date:   Mon Aug 12 09:23:58 2019 -0600

    Work in progress: provide instructions for both clang+gfortran and gcc+gfortran on MacOSX

commit 3d01b44f5b316ca5bfeb0dfa8e570074921bdfec
Author: Dom Heinzeller <[email protected]>
Date:   Mon Aug 12 08:18:45 2019 -0600

    Update submodule pointer for ccpp/physics

commit d4773492a63cfe26bcb84c6d50c78e86855f4b05
Author: Dom Heinzeller <[email protected]>
Date:   Mon Aug 12 08:17:43 2019 -0600

    tests/tests/fv3_ccpp_appbuild: bugfix, add back description/control dir that were unintentially removed in a previous commit

commit ae4795908cfba0955c7aed812f717cf54562cb62
Author: dom <[email protected]>
Date:   Sun Aug 11 16:07:59 2019 -0600

    conf/configure.fv3.macosx.gnu, modulefiles/macosx.gnu/fv3: add flexibility by using user-defined compiler commands and library locations (through environment variables)

commit 619382c41818dfee04a5ab1f5c3cafde7fab821b
Author: Dom Heinzeller <[email protected]>
Date:   Fri Aug 9 10:35:07 2019 -0600

    modulefiles/linux.gnu/fv3: allow user to specify own locations for compilers and libraries

commit 1b5d9cc192f1032d899468379569be64a7b13bd1
Merge: e8a4da85 03553110
Author: Dom Heinzeller <[email protected]>
Date:   Thu Aug 1 16:27:31 2019 -0600

    Merge pull request #211 from climbfuji/github_gmtb_develop_update_submodule_pointers_20190801

    GitHub gmtb/develop update submodule pointers 2019/08/01

commit 0355311007afd0231f93bf5039e72d6df4125292
Author: Dom Heinzeller <[email protected]>
Date:   Thu Aug 1 16:25:10 2019 -0600

    Update submodule pointers for FV3, ccpp/framework, ccpp/physics

commit e8a4da8588c7495408a931cce323ca3c6f5cd190
Merge: b06eeb95 ab06c106
Author: Dom Heinzeller <[email protected]>
Date:   Thu Aug 1 14:46:17 2019 -0600

    Merge pull request #210 from climbfuji/new_metadata_format_step1_20190725

    Migrate to new metadata format step 1

commit bd4cb27b7cc39d246f1aa5c055beb2506427f601
Merge: 1ebaea29 b06eeb95
Author: Dom Heinzeller <[email protected]>
Date:   Thu Aug 1 08:22:28 2019 -0600

    Merge branch 'gmtb/develop' of https://github.com/NCAR/NEMSfv3gfs into HEAD

commit ab06c106e05811faf6e7ae7107769657c3e8b9e0
Author: Dom Heinzeller <[email protected]>
Date:   Wed Jul 31 16:26:46 2019 -0600

    Update submodule pointers for ccpp/framework and ccpp/physics

commit 74d98ff31771e50553e2eb31e6b8cafcfc305008
Merge: b77f1871 13619a43
Author: Dom Heinzeller <[email protected]>
Date:   Wed Jul 31 15:49:32 2019 -0600

    Merge branch 'master' of gerrit:NEMSfv3gfs into HEAD

commit b77f187103fbbcf25c0871a328f094f609761193
Author: Dom Heinzeller <[email protected]>
Date:   Wed Jul 31 13:39:37 2019 -0600

    ccpp/config/ccpp_prebuild_config.py: add new type definition files and TYPEDEFS_NEW_METADATA configuration entry

commit 73d2015f97e3730a334c532eef46f4ae2a3e4193
Author: Dom Heinzeller <[email protected]>
Date:   Mon Jul 22 20:49:37 2019 -0400

    tests/rt.sh, tests/fv3_conf/fv3_qsub.IN_jet -> tests/fv3_conf/fv3_slurm.IN_jet: change scheduler on jet from Moab/Torque to Slurm

commit 2312a74d685f0c348b71c41af86eefc6606fd196
Author: Dom Heinzeller <[email protected]>
Date:   Fri Jul 19 15:14:44 2019 -0600

    conf/before_components.mk: enable jet.intel

commit e6cda943e2adc3b54005a64805c667a9185f02c4
Author: Dom Heinzeller <[email protected]>
Date:   Fri Jul 19 14:51:30 2019 -0600

    Update submodule pointer for FV3

commit 0642cc07aa0808fc86a538e9b2ae8f8295e569ae
Author: Dom Heinzeller <[email protected]>
Date:   Fri Jul 19 14:51:04 2019 -0600

    tests/rt.conf: add jet.intel similar to gaea.intel

commit 8607bea695293c71f879e70d910853b81b492ea5
Author: Dom Heinzeller <[email protected]>
Date:   Fri Jul 19 14:49:21 2019 -0600

    Update modulefiles/jet.intel/fv3 to use modules created by GMTB

commit 1ebaea2974ba4f06f4c1c114faa03fde94c3e293
Merge: 2605f0b7 72c10102
Author: Dom Heinzeller <[email protected]>
Date:   Fri Jul 19 11:47:22 2019 -0600

    Merge pull request #209 from climbfuji/update_submodule_pointer_20190719

    Update submodule pointer for ccpp/physics

commit 72c10102ce07ac8c465f571bea6a6cd5aa5fe088
Author: Dom Heinzeller <[email protected]>
Date:   Fri Jul 19 11:45:57 2019 -0600

    Update submodule pointer for ccpp/physics

commit 2605f0b773ab7a862f9152a0c214f8da372e3d1e
Merge: cad0f201 820e0cc2
Author: Dom Heinzeller <[email protected]>
Date:   Fri Jul 19 11:45:20 2019 -0600

    Merge pull request #208 from climbfuji/gsd_develop_ruc_bugfixes_sar_update_surface_flux_bugfix_inactive_20190718

    Gsd develop ruc bugfixes sar update surface flux bugfix inactive 20190718

commit 820e0cc2a94c9599b4a48e0f7c54900792e8cfbe
Author: Dom Heinzeller <[email protected]>
Date:   Thu Jul 18 21:28:28 2019 -0600

    Update submodule pointer for ccpp/physics

commit d954359f8d36cda0306e9f51a8f5f3797945cd2f
Author: Dom Heinzeller <[email protected]>
Date:   Thu Jul 18 21:28:08 2019 -0600

    ccpp/suites/suite_FV3_GSD_v0.xml: add calls to temporary interstitial schemes for RUC-sfc_sice interaction

commit c3993781d192fa061dc9484342b923e149597b0f
Author: Dom Heinzeller <[email protected]>
Date:   Thu Jul 18 21:27:13 2019 -0600

    ccpp/config/ccpp_prebuild_config.py: add temporary interstitial schemes in lsm_ruc_sfc_sice_interstitial.F90

commit a8d3ec08a9ed65de0bf45d7bdd4974b7a9ae6c83
Author: Dom Heinzeller <[email protected]>
Date:   Thu Jul 18 15:27:48 2019 -0600

    conf/configure.fv3.wcoss_cray, ccpp/build_ccpp.sh: set correct REPRO compiler flags for Intel 16, pass legacy-Intel flag to cmake

commit e0b77299c83e21bd38a1cf0a9cb4617e461c4af9
Author: Dom Heinzeller <[email protected]>
Date:   Thu Jul 18 11:13:16 2019 -0600

    tests/rt_gnu.conf: initial set of GNU-based regression tests, Cheyenne only

commit 35c2ef3f34ca3ca4086d3e71b4c610b56e2659b8
Author: Dom Heinzeller <[email protected]>
Date:   Thu Jul 18 11:12:47 2019 -0600

    tests/rt_utils.sh: update of regular expression for Cheyenne only

commit 4aa66b51bde51c9ec2bb3ca0b240a8bfd71bf2af
Author: Dom Heinzeller <[email protected]>
Date:   Thu Jul 18 11:12:03 2019 -0600

    tests/rt.sh: temorary, change location of regression test baseline to ESRL-GSD-GMTB project directory

commit ed13c0038d6bd15da146321b8ca04ab632a07abc
Author: Dom Heinzeller <[email protected]>
Date:   Thu Jul 18 11:10:58 2019 -0600

    modulefiles/gaea.intel/fv3, modulefiles/gaea.intel/fv3.intel-16.0.3.210: update of modulefiles on gaea for both Intel 16 and 18 for CCPP

commit b99c6e2a1058fea4ea7c083bcf7ba54cd9fc62b6
Author: Dom Heinzeller <[email protected]>
Date:   Thu Jul 18 11:09:38 2019 -0600

    Update of Intel and GNU modulefiles for Cheyenne following the major OS upgrades beginning of July 2019

commit b74241682499f54e474d087fdc6797932534f459
Author: Dom Heinzeller <[email protected]>
Date:   Thu Jul 18 11:08:31 2019 -0600

    conf/configure.fv3.gaea.intel: add option to use Intle 16, adjust repro flags if Intel 16 is used

commit 89024d4dd5034aedec7ed197e84388e888c01a7f
Author: Dom Heinzeller <[email protected]>
Date:   Thu Jul 18 11:03:32 2019 -0600

    conf/before_components.mk: enable gaea.intel, add switch to use Intel 16 compiler on this machine (default is Intel 18)

commit 9909cd1c5711baf1a59595f3e4047aaa8b960606
Author: Dom Heinzeller <[email protected]>
Date:   Thu Jul 18 11:01:42 2019 -0600

    ccpp/CMakeLists.txt, ccpp/build_ccpp.sh: add option INTEL16=Y to build_ccpp.sh, which triggers a cmake flag LEGACY_INTEL to use a different fp-model option in REPRO mode

commit cad0f201bf3de40b3e6b4f24a491ff18d654ac5b
Author: Dom Heinzeller <[email protected]>
Date:   Thu Jul 18 10:46:02 2019 -0600

    Update CODEOWNERS to enable branch protection

commit 89a0f8eaf60b393739658b9605de3194d8159a9d
Merge: 259b675c c7c692c0
Author: Dom Heinzeller <[email protected]>
Date:   Wed Jul 17 20:50:49 2019 -0600

    Merge pull request #207 from climbfuji/gsd_develop_update_submodule_pointers_20190717

    gsd/develop: update submodule pointers for FV3 and ccpp/physics

commit c7c692c02b8a680c6e8f7c03f0a46eee1afdbd4b
Author: Dom Heinzeller <[email protected]>
Date:   Wed Jul 17 20:49:18 2019 -0600

    Update submodule pointers for FV3 and ccpp/physics

commit 259b675cfd32286d7995b7c5a2483344f0e81213
Merge: cb0d5e30 75906385
Author: Dom Heinzeller <[email protected]>
Date:   Wed Jul 17 20:48:13 2019 -0600

    Merge pull request #206 from climbfuji/add_regression_tests_20190717

    gsd/develop: add regression tests for Thompson only, GF only, MYNN only & cleanup

commit 759063853f2cae2173afe9234c7d04424baaafb7
Author: Dom Heinzeller <[email protected]>
Date:   Wed Jul 17 14:47:10 2019 -0600

    tests/fv3_conf/ccpp_gsd_run.IN: add logic to use correct field table, depending on choice of microphysics schemes and whether a TKE PBL scheme is used

commit 835265e06943d297959cc45645d0d5c516e675e2
Author: Dom Heinzeller <[email protected]>
Date:   Wed Jul 17 14:31:47 2019 -0600

    Update submodule pointers for FV3 and ccpp/physics

commit bb7f8838d76dba2f3b8ef85a3015f3fa51b8dd2f
Author: Dom Heinzeller <[email protected]>
Date:   Wed Jul 17 14:31:32 2019 -0600

    parm/ccpp_gsd.nml.IN, tests/fv3_conf/ccpp_gsd_run.IN, tests/rt_ccpp_gsd.conf: use additional namelist variables for more flexibility

commit d454c1cff7d9cf740f2129620f09eddb8fe0cd30
Author: Dom Heinzeller <[email protected]>
Date:   Wed Jul 17 14:29:19 2019 -0600

    tests/tests/fv3_ccpp_*: add new tests for individual schemes of the GSD suite, remove increase of wallclock time on Cheyenne

commit b93828f3c8f338f9bc5a56f0c67a9b88765efa74
Author: Dom Heinzeller <[email protected]>
Date:   Wed Jul 17 14:28:07 2019 -0600

    tests/default_vars.sh: add additional default variables for microphysics

commit b06eeb95aeba76e3ad3e6210cb8b4ff81707086b
Merge: f4790463 4e253679
Author: Dom Heinzeller <[email protected]>
Date:   Wed Jul 17 09:42:39 2019 -0600

    Merge pull request #205 from climbfuji/gitmodules_add_default_branches_20190716

    gmtb/develop: add default branches for submodules to .gitmodules

commit 4e25367977bbfb401e604e867cafd9e3b09892f0
Author: Dom Heinzeller <[email protected]>
Date:   Tue Jul 16 15:15:37 2019 -0600

    .gitmodules: set default branches for submodules

commit cb0d5e30c0a244e1bc14b57b9dbc331b7781319d
Author: Dom Heinzeller <[email protected]>
Date:   Tue Jul 16 15:04:21 2019 -0600

    Update submodule pointers for all submodules

commit 22fd642f89cd516f177e160ecb486d6ed179a56c
Author: Dom Heinzeller <[email protected]>
Date:   Tue Jul 16 15:04:06 2019 -0600

    .gitmodules: set default branches of submodules to gsd/develop

commit f1b1245a9f566a13b1d2dabca46a12729fd1c418
Merge: f9330c32 f4790463
Author: Dom Heinzeller <[email protected]>
Date:   Tue Jul 16 15:03:24 2019 -0600

    Update CODEOWNERS to GSD t…
@junwang-noaa junwang-noaa deleted the upp8011 branch October 30, 2020 14:38
pjpegion pushed a commit to NOAA-PSL/ufs-weather-model.p7b that referenced this pull request Jul 20, 2021
@climbfuji @SMoorthi-emc
Final-final GFS v16 updates / restart reproducibility bugfixes (ufs-c…
3aad398 
…ommunity#212)

* updates the submodule pointer for ccpp-physics for the final-final (!) GFS v16 physics updates
* fixes a bug in io/FV3GFS_io.F90 to obtain restart reproducibility for uncoupled and coupled runs - contributed by @SMoorthi-emc

Co-authored-by: Shrinivas.Moorthi <[email protected]>
epic-cicd-jenkins pushed a commit that referenced this pull request Apr 17, 2023
@gsketefian
Improve the way fixed files are handled. (#212)
a78b7d6 
* Improve the way fixed files are handled.

* Improve the way fixed files are handled so that what is happening with the fixed files is clearer.
* Reduce the number of symlinks needed in each cycle directory.
* Fix the EMC_CONUS_3km grid settings so that it has a valid BLOCKSIZE.
* Add new WE2E test for EMC_CONUS_3km grid.

List of modifications common to more than one file (these are referenced below for each file as needed):
-------------------------------------------------------------------------------------------------------

(A) Instead of passing in NPROCS as an environment variable via the rocoto XML, calculate nprocs locally using workflow variables.
(B) Use new function check_for_preexist_dir_file() that replaces old function check_for_preexist_dir().  The new one works for both directories and files.
(C) Use the new function set_FV3_sfc_climo_filenames() to set those variables in the FV3 namelist file that specify the paths to the surface climatology files on the native grid.
(D) Improve and/or clean up comments and/or informational and/or error messages.
(E) Put in a check to make sure that the executable chgres_cube.exe exists.
(F) Do not export variables when not necessary.
(G) Remove commented-out (unused) code.
(H) Remove blank lines.

File-by-file modifications:
--------------------------

jobs/JREGIONAL_GET_EXTRN_FILES:
* For clarity, rename this to JREGIONAL_GET_EXTRN_MDL_FILES.

jobs/JREGIONAL_MAKE_ICS: (A)

jobs/JREGIONAL_MAKE_LBCS: (A)

jobs/JREGIONAL_MAKE_SFC_CLIMO: (B)

jobs/JREGIONAL_RUN_POST: (B)
* Add check to make sure forecast hour contains exactly two digits.  That's all EMC_post seems to be able to handle at the moment.

scripts/exregional_make_grid.sh: (C), (D), (F)
* Remove topo_dir as a variable since it is not needed in this script.

scripts/exregional_make_ics.sh: (D), (E)

scripts/exregional_make_lbcs.sh: (D), (E)

scripts/exregional_make_orog.sh: (B), (F)
* Remove definition of topo_dir in CHEYENNE stanza since this variable gets redefined later on anyway (for all machines).

scripts/exregional_make_sfc_climo.sh: (A)

scripts/exregional_run_fcst.sh: (A), (D)
* Do not use the (now removed) workflow arrays FIXgsm_FILENAMES and FIXam_FILENAMES to create links in the cycle directory to fixed files in the FIXam directory.  Instead, use the (new) workflow array CYCLEDIR_LINKS_TO_FIXam_FILES_MAPPING that specifies the mapping between the symlinks that need to be created in the cycle directory and the corresponding files in the FIXam directory.
* In community mode, make the symlinks that are created in the cycle directory to the certain files relative.  These files consist of the data_table, field_table, the FV3 namelist, nems_configure, CCPP physics suite, and (for some physics suites) the CCN_ACTIVATE.BIN files.
* Do not copy the forecast executable to the cycle directory.  Instead, just call it using its full path in its original location (in EXECDIR).  This saves disk space and reduces clutter.

scripts/exregional_run_post.sh: (A), (G), (H)

tests/baseline_configs/config.EMC_CONUS_3km.sh:
* Add this new WE2E test configuration.  This tests the workflow on EMC's 3km CONUS grid in NCO mode (i.e. RUN_ENVIR set to "nco").

tests/baseline_configs/config.regional_005.sh:
* Specify non-default computational resources for this WE2E test (the 50km Alaska domain) such that the make_ics, make_lbcs, and run_post tasks do not crash (the number of MPI processes cannot be too high; otherwise, these tasks will fail).

tests/baseline_configs/config.regional_009.sh:
* Uncomment lines specific for Hera and comment out lines specific for Jet.  This has to be changed to include an if-statement that sets parameters based on the machine.

tests/baseline_configs/config.regional_015.sh:
* Add this new WE2E test configuration.  This tests the workflow on a coarse GFDLgrid type of grid that has a stretch factor of almost 1 (it cannot be exactly 1 because then the ufs_weather_model code assumes that it is a global grid and thus expects 6 tiles; this is a bug in the ufs_weather_model code that needs to be fixed).  A more refined version of this grid (refine_ratio changed from 2 as it is here to 36) is being tested by GSL until issues with the JPgrid verification are worked out.  Thus, we make sure capability of the workflow to run on this GFDLgrid doesn't break.

tests/baselines_list.txt:
* Update the list to include the new WE2E tests that are included in this commit (regional_014, regional_015, and EMC_CONUS_3km).  This file isn't really necessary and should be deleted at some point (e.g. in a different commit that fixes up various deficiencies with the WE2E tests).

ush/bash_utils/check_for_preexist_dir.sh:
* Modify the function this file defines such that it now works for files as well as directories.
* Rename the file and the function that it defines from check_for_preexist_dir(.sh) to check_for_preexist_dir_file(.sh).

ush/compare_config_scripts.sh:
* Change variable DEFAULT_EXPT_CONFIG_FN to EXPT_DEFAULT_CONFIG_FN (this variable is renamed in ush/setup.sh).

ush/config_defaults.sh:
* Put the variable SCHED specifying the job scheduler to use in this file so that it becomes user-specifiable (and avaiable as a workflow variable).  This may be needed when adding the capability to run outside of the scheduler (in a future commit/PR).
* Include detailed descriptions of the NCO mode variables (COMINgfs, STMP, NET, envir, RUN, PTMP) are used.
* Add new variable (EXPT_CONFIG_FN) that specifies the name of the experiment/workflow configuration file containing user-specified values.
* Add new variable (FV3_EXEC_FN) that specifies the name to use for the forecast model executable if/when it is copied from the directory in which the build step creates it to the directory in which the workflow scripts look for it (EXECDIR).  This copy step should really be done in during the build.
* For clarity and consistency, rename variables FV3_NML_FN and FV3_NML_CONFIG to FV3_NML_BASE_FN and FV3_NML_YAML_CONFIG_FN, respectively.
* Add new array SFC_CLIMO_FIELDS containing the names of the surface climatology fields used by the sfc_climo_gen code.
* Remove some variables (FNALBC, FNALBC2, etc) whose names are now specified in the new array FV3_NML_VARNAME_TO_SFC_CLIMO_FIELD_MAPPING.
* Add new array FIXgsm_FILES_TO_COPY_TO_FIXam that specifies the fixed files that must be copied from the system directory FIXgsm to the local directory FIXam (which is under the experiment directory).
* Add new array FV3_NML_VARNAME_TO_FIXam_FILES_MAPPING that specifies the FV3 namelist variable names and the corresponding files in the FIXam directory that they are associated with (the variables need to be set to the relative or full paths to these files).
* Add new array FV3_NML_VARNAME_TO_SFC_CLIMO_FIELD_MAPPING that specifies the FV3 namelist variable names and the corresponding surface climatology fields in the sfc_climo_gen code (the variables need to be set to the relative or full paths to the files containing the corresponding fields).
* Add new array CYCLEDIR_LINKS_TO_FIXam_FILES_MAPPING specifying the symlinks that need to be created in each cycle directory and their corresponding target files in the FIXam directory.
* Add new variables specifying the workflow task names and the corresponding values of the number of nodes, logical processes per node, and maximum wallclock time for each task.

ush/generate_FV3SAR_wflow.sh: (D)
* Remove variables PROC_RUN_FCST and NPROCS_RUN_FCST since they are now superceded by the variables NNODES_RUN_FCST and PPN_RUN_FCST.
* Remove the variable CYCLE_DIR since it is being replaced by CYCLE_BASEDIR (which is set in ush/setup.sh; see description of changes to that file).
* For clarity, reorder (and include comments between) the lines in the variable "settings" that specifies how to set the variables (entities, etc) in the template rocoto XML.
* Replace code that copies files from the FIXgsm directory with code that does the same thing using (looping over) the new workflow array variable FIXgsm_FILES_TO_COPY_TO_FIXam.
* Put in new code that copies the forecast model executable from the directoy in which the forecast model repository was cloned into the executable directory (EXECDIR) but ONLY IF such a copy does not
already exist in EXECDIR or, if it does exist, it is older than the source executable.  This copying should really be done in the build step, but it currently is not.
* For clarity, reorder some of the lines in the variable "settings" that is used to specify values in the FV3 fortran namelist file (input.nml) such that they are grouped by type (e.g. grid-related, computational, etc).  Also, include some comments between the lines.
* Remove the hard-coded settings of the "FN..." variables from the "settings" variable.  Instead:
  * For the "FN..." variables that specify paths to fixed files in the FIXam directory, loop through the new workflow array variable FV3_NML_VARNAME_TO_FIXam_FILES_MAPPING and, for each element of this array, create an appropriate entry in "settings".
  * For the "FN..." variables that specify the paths to surface climatology files:  (1) If not running the make_grid task, call the new function set_FV3nml_sfc_climo_filenames that sets those FV3 namelist variables that specify the paths to surface climatology files to the names of the surface climatology files on the FV3SAR grid (these files are either pregenerated or are created by the make_sfc_climo task).  (2) If running the make_grid task, call this same funtion in the make_grid task.  This is because the names of the surface climatology files depend on the CRES parameter (the C-resolution of the grid), and usually, that is not known until the grid is created (by the make_grid task).

ush/link_fix.sh:
* Replace local variable sfc_climo_fields with new workflow array variable SFC_CLIMO_FIELDS.
* Add code that creates symlinks in the FIXsar directory to surface climatology files with names that end in "tile7.nc" that have names that end in "tile1.nc".  This is apparently necessary for FV3 to be able to use the surface climatology files on the native FV3SAR grid (which are either pregenerated or created by the make_sfc_climo task).

ush/load_modules_run_task.sh: (D)
* Fix indentation.

ush/set_FV3nml_sfc_climo_filenames.sh:
* This is a new file that defines a function (set_FV3nml_sfc_climo_filenames) that sets the values of the variables in the forecast model's namelist file that specify the paths to the surface climatology files on the FV3SAR native grid (which are either regenerated or created by the MAKE_SFC_CLIMO_TN task).  We make this a function because it is needed in a couple of places in the scripts.

ush/set_fix_filenames.sh:
* Remove this file since the fixed files are being handled differently now using the new workflow array variables defined in ush/config_defaults.sh.

ush/set_ozone_param.sh:
* This is a new file that defines a function (set_ozone_param) that sets various workflow parameters depending on which ozone parameterization is being used by the experiment.  The ozone parameterization is determined by inspecting the CCPP suite defintion file.

ush/set_predef_grid_params.sh: (G)
* Set BLOCKSIZE to an allowed value for the EMC_CONUS_3km grid.

ush/setup.sh: (D), (G)
* For clarity, rename variable DEFAULT_EXPT_CONFIG_FN to EXPT_DEFAULT_CONFIG_FN.
* Remove variable EXPT_CONFIG_FN since that is now defined in config_defaults.sh.
* Set SCHED to a machine-dependent value only if it is not already specified by the user in config.sh.
* Remove stanza for machine THEIA (macine no longer exists).
* Change SCHED to lowercase (since that is how rocoto accepts it) and Test to make sure that SCHED is set to a valid value.
* Remove setting of variables that specify the task names since those are now in config_defaults.sh.
* Set the new workflow variable CYCLE_BASEDIR that contains the base directory under which the cycle directories are created.  This replaces CYCLE_DIR.  Note that CYCLE_BASEDIR is the old CYCLE_DIR but without the last directory (which is the CDATE of the cycle).  Now, CYCLE_DIR is formed within the XML by combining CYCLE_BASEDIR and the current cycle's date information from rocoto.  Also, add CYCLE_BASEDIR to the variable definitions file (var_defns.sh).
* For clarity, rename variable FV3_NML_CONFIG_FP to FV3_NML_YAML_CONFIG_FP.
* For convenience, create variable FV3_NML_FN (that is set to the name of the namelist file that FV3 expects to read in).  Then use this to set FV3_NML_FP, which is the full path to that namelist file.
* Create new workflow variable FV3_EXEC_FP that conains the full path to the copy of the FV3SAR executable in the executables directory EXECDIR.  Add this new variable to the variable definitions file (in a "cat" statement towards the end of the file).
* Set the new variable LOAD_MODULES_RUN_TASK_FP that specifies the full path to the script that loads the modules for each task and then runs the task.  This used to be hard-coded inside of the rocoto XML but is now needed as a workflow variable (because it is needed in at least a couple of places).  Add this variable to the variable definitions file (var_defns.sh).
* Fix bug (the way the local variable glob_pattern is set) to allow for the use of a dot as the fixed file name separator when in NCO mode (i.e. when RUN_ENVIR is set to "nco").
* Rename the variable NUM_NODES to NNODES_RUN_FCST to be consistent with the naming convention for number of nodes used for other tasks.
* Remove call to function set_fix_filenames since it is no longer needed.
* Remove recording of the array variables FIXgsm_FILENAMES and FIXam_FILENAMES in the variable defintions file since these arrays are no longer needed (they were previously set by the function set_fix_filenames).
* Call the new function set_ozone_param that sets various workflow parameters depending on which ozone parameterization is being used (which is specified in the CCPP suite defintion file).
* Clean up comments.

ush/source_util_funcs.sh: (D)
* Change the file name check_for_preexist_dir.sh to check_for_preexist_dir_file.sh.

ush/templates/FV3SAR_wflow.xml:
* For clarity, reorder the entities that contain jinja parameters by grouping similar items together.
* Make LOAD_MODULES_RUN_TASK_FP an ENTITY that is set externally by the workflow generation scripts (as opposed to hard-coded in the XML) because this is now a variable that is needed in another file (the script that is used to run a task outside of rocoto).
* Replace CYCLE_DIR as an externally specified ENTITY with CYCLE_BASEDIR.
* Create new externally-specified ENTITYs for the number of nodes, the number of MPI processes per node, and the wallclock time for each task.  These will get set by the workflow generation scripts.
* Make LOAD_MODULES_RUN_TASK_FP an ENTITY that is set externally by the workflow generation scripts (as opposed to hard-coded in the XML) because this is now a variable that is needed in another file (the script that is used to run a task outside of rocoto).
* Use the ENTITYs for the number of nodes, the number of MPI processes per node, and the wallclock time for each task to set the PROC_... and RSRC_... ENTITYs for each task (as opposed to hard-coding the latter).
* For brevity, rename ENTITYs containing the strings "MAKE_ICS_SURF_LBC0" and "MAKE_LBC1_TO_LBCN" to "MAKE_ICS" and "MAKE_LBCS", respectively.
* For consistency with other tasks, rename PROC_POST and RSRC_POST to PROC_RUN_POST and RSRC_RUN_POST.
* Change name of ENTITY RSRV_RUN_FCST to RSRV_FCST to be consistent with name of ENTITY QUEUE_FCST.  Names of ENTITYs related to the job scheduler need to be revisited (e.g. maybe change QUEUE_FCST to QUEPART_FCST since in slurm, there are partitions, not queues).
* Remove variable PDY from pre-processing tasks (which are cycle-independent).
* Fix bug where the task name "make_grid" is hard-coded as opposed to using the ENTITY MAKE_GRID_TN.
* For clarity, rename file JREGIONAL_GET_EXTRN_FILES to JREGIONAL_GET_EXTRN_MDL_FILES.
* Reorder environment variable settings in each task to go from least to most general.
* Within each cycle-dependent task, set CYCLE_DIR based on CYCLE_BASEDIR.

ush/valid_param_vals.sh:
* Add valid values for the new user-specifiable workflow variable SCHED.
* Remove the unused predefined grid GSD_HRRR_AK_3km since it has been renamed GSD_RRFSAK_3km.

* Manual merge of Mike's changes for file ush/load_modules_run_task.sh.

* Improve/streamline generation of rocoto workflow XML.

Modifications made to more than one file:
----------------------------------------
(A) For uniformity among workflow tasks, split the worfklow variables that specify the number of nodes, logical processes per node, and wall clock time for the generic GET_EXTRN_MDL_FILES task into two variables each -- one for the GET_EXTRN_ICS_TN and another for the GET_EXTRN_LBCS_TN task.
(B) Remove USHDIR as an ENTITY in the XML since it is not used after being set.

ush/config_defaults.sh: (A)

ush/generate_FV3SAR_wflow.xml: (B)

ush/templates/FV3SAR_wflow.xml: (A), (B)
* Clean up comments.
* To reduce the number of entities rocoto has to deal with, remove the ENTITY definitions for various entities that only appear in one location in the file and replace them with the corresponding jinja2 placeholders or combinations of such placeholders.  Do this for the following entities:
  * The per-task entities NNODES_..., PPN_..., WTIME_..., PROC_..., and RSRC_....
  * EXTRN_MDL_NAME_ICS and EXTRN_MDL_NAME_LBCS

* Get EMC's changes into community workflow (Ratko's PR plus other necessary changes).

File-by-file description of modifications:
-----------------------------------------

fix/fix_upp/nam_nests.hiresf_nn.txt:
* Deleted as requested by EMC (from Ratko's original PR).

fix/fix_upp/postxconfig-NT-fv3sar.txt
* Modified as requested by EMC (from Ratko's original PR).

scripts/exregional_get_extrn_files.sh:
* If extracting the external model files from one or more HPSS archive files, put in a check to ensure that each external model file was extracted exactly once from one of the archive files.

scripts/exregional_make_ics.sh:
* Add parameter settings for the FV3_GFS_2017_gfdlmp_regional physics suite (from Ratko's original PR).
* The FV3_GFS_2017_gfdlmp_regional physics suite requires a variable called tref (reference temperature) in the initial conditions.  That variable is available only for external files coming from the FV3GFS model.  tref can be extracted from the surface analysis file if the namelist variable convert_nst is set to .true. in the chgres_cube namelist file.  Thus, create a new local variable named convert_nst that gets set to true for the FV3GFS model but is false otherwise and pass it to the chgres_cube namelist.

scripts/exregional_make_lbcs.sh:
* Add parameter settings for the FV3_GFS_2017_gfdlmp_regional physics suite (from Ratko's original PR).

tests/baseline_configs/config.EMC_CONUS_3km.sh:
* Rename to tests/baseline_configs/config.nco_conus.sh (see that file).

tests/baseline_configs/config.nco_conus.sh:
* Renamed from tests/baseline_configs/config.EMC_CONUS_3km.sh.
* Change queue from debug to batch since we always want batch as the default (so that if running many tests, they are not limited by the max number of jobs a user can have in the debug queue, which on hera is just 2).
* Set the physics suite for this WE2E test to FV3_GFS_2017_gfdlmp_regional because that's the one EMC runs with.
* Reset the RUN variable (which is used in creating output directory if in NCO mode) to a value that is unique to this WE2E test so that it doesn't interfere with other tests that are running at the same time (and vice versa).

tests/baseline_configs/config.nco_conus_c96.sh:
* New WE2E test for the coarse EMC CONUS grid.  For rapid testing, this can be run instead of config.nco_conus.sh.

tests/baseline_configs/config.new_JPgrid.sh
* New WE2E test for specifying a new (i.e. not predefined) JPgrid.

tests/baseline_configs/config.regional_009.sh:
* Reset the RUN variable (which is used in creating output directory if in NCO mode) to a value that is unique to this WE2E test so that it doesn't interfere with other tests that are running at the same time (and vice versa).

tests/baselines_list.txt:
* Update the full list of WE2E test names to the latest set.

ush/config_defaults.sh:
* Edit comments.

ush/generate_FV3SAR_wflow.sh:
* Remove FNZORC, FNTSFA, FNACNA, and FNSNOA from the "settings" variable used in generating the FV3 input namelist file.  These variables are now instead specified in the base namelist file (ush/templates/input.nml.FV3) because they are the same for all physics suites considered thus far.

ush/get_extrn_mdl_file_dir_info.sh:
* Remove unused code.

ush/link_fix.sh:
* Bug fix:  Change the targets of the "...tile1..." symlinks so that they point to the tile7.halo0.nc files, not the tile7.nc symlinks (which in turn point to the tile7.halo4.nc files, which is not what we want).

ush/set_namelist.py:
* Add option "sort=True" to nml.write method to sort the resulting FV3 namelist file in alphabetical order.  This makes it easier to compare (diff) namelists.

ush/setup.sh:
* Add a temporary section of code that makes fixes to the suite defintion file for the FV3_GFS_2017_gfdlmp_regional physics suite.  This is necessary because the version of the file in the dtc/develop branch of the NCAR fork of the fv3atm repo has not yet been updated to have these corrections.  This code should be removed once that branch/repo have been updated.

ush/templates/FV3.input.yml:
* Add section for the new FV3_GFS_2017_gfdlmp_regional physics suite.  Note that this references the preexisting FV3_GFS_2017_gfdlmp physics suite.

ush/templates/input.nml.FV3:
* Reorder lines so that both namelists and the variables within each namelist are lower case and in alphabetical order.  This makes it easier to compare this base FV3 namelist file to other FV3 namelist files generated using the set_namelist.py script.

ush/templates/diag_table.FV3_GFS_2017_gfdlmp_regional:
* New diag_table file for FV3_GFS_2017_gfdlmp_regional suite.

ush/templates/field_table.FV3_GFS_2017_gfdlmp_regional:
* New field_table file for FV3_GFS_2017_gfdlmp_regional suite.

ush/templates/model_configure.FV3_GFS_2017_gfdlmp_regional:
* New model_configure file for FV3_GFS_2017_gfdlmp_regional suite.

ush/valid_param_vals.sh:
* Add FV3_GFS_2017_gfdlmp_regional to the list of valid physic suites (as requested by EMC).

* Change compile options for EMC as requested by Ratko.
epic-cicd-jenkins pushed a commit that referenced this pull request Apr 17, 2023
@BenjaminBlake-NOAA @mkavulich
@gsketefian @christinaholtNOAA
Feature/ufs sr wx app Version of regional workflow to use with ufs-sr…
5b74245 
…weather-app (#231)

* Modify paths in setup.sh to work in ufs-srweather-app directory structure.

* Remove Externals.cfg, build scripts and manage_externals, since these
are now part of the ufs-srweather-app.

* Add path for SFC_CLIMO_INPUT_DIR for jet.

* Update exregional_make_grid.sh with community_develop.

* Feature/ufs sr wx app Merge code changes from develop branch (#229)

* Add Python plotting script for FV3SAR grib2 output (#215)

Co-authored-by: Benjamin.Blake EMC <[email protected]>

* Adding pull request template file, moving special files to .github directory (#217)

* Move CODOWNERS file to .github directory; make draft pull request template

* Change format a bit, I like this better since it looks nicer on github

* Edit README.md. (#220)

Print newlines before step 4 and step 5 so that they show up as separate entries in the list.

* Improve the way fixed files are handled. (#212)

* Improve the way fixed files are handled.

* Improve the way fixed files are handled so that what is happening with the fixed files is clearer.
* Reduce the number of symlinks needed in each cycle directory.
* Fix the EMC_CONUS_3km grid settings so that it has a valid BLOCKSIZE.
* Add new WE2E test for EMC_CONUS_3km grid.

List of modifications common to more than one file (these are referenced below for each file as needed):
-------------------------------------------------------------------------------------------------------

(A) Instead of passing in NPROCS as an environment variable via the rocoto XML, calculate nprocs locally using workflow variables.
(B) Use new function check_for_preexist_dir_file() that replaces old function check_for_preexist_dir().  The new one works for both directories and files.
(C) Use the new function set_FV3_sfc_climo_filenames() to set those variables in the FV3 namelist file that specify the paths to the surface climatology files on the native grid.
(D) Improve and/or clean up comments and/or informational and/or error messages.
(E) Put in a check to make sure that the executable chgres_cube.exe exists.
(F) Do not export variables when not necessary.
(G) Remove commented-out (unused) code.
(H) Remove blank lines.

File-by-file modifications:
--------------------------

jobs/JREGIONAL_GET_EXTRN_FILES:
* For clarity, rename this to JREGIONAL_GET_EXTRN_MDL_FILES.

jobs/JREGIONAL_MAKE_ICS: (A)

jobs/JREGIONAL_MAKE_LBCS: (A)

jobs/JREGIONAL_MAKE_SFC_CLIMO: (B)

jobs/JREGIONAL_RUN_POST: (B)
* Add check to make sure forecast hour contains exactly two digits.  That's all EMC_post seems to be able to handle at the moment.

scripts/exregional_make_grid.sh: (C), (D), (F)
* Remove topo_dir as a variable since it is not needed in this script.

scripts/exregional_make_ics.sh: (D), (E)

scripts/exregional_make_lbcs.sh: (D), (E)

scripts/exregional_make_orog.sh: (B), (F)
* Remove definition of topo_dir in CHEYENNE stanza since this variable gets redefined later on anyway (for all machines).

scripts/exregional_make_sfc_climo.sh: (A)

scripts/exregional_run_fcst.sh: (A), (D)
* Do not use the (now removed) workflow arrays FIXgsm_FILENAMES and FIXam_FILENAMES to create links in the cycle directory to fixed files in the FIXam directory.  Instead, use the (new) workflow array CYCLEDIR_LINKS_TO_FIXam_FILES_MAPPING that specifies the mapping between the symlinks that need to be created in the cycle directory and the corresponding files in the FIXam directory.
* In community mode, make the symlinks that are created in the cycle directory to the certain files relative.  These files consist of the data_table, field_table, the FV3 namelist, nems_configure, CCPP physics suite, and (for some physics suites) the CCN_ACTIVATE.BIN files.
* Do not copy the forecast executable to the cycle directory.  Instead, just call it using its full path in its original location (in EXECDIR).  This saves disk space and reduces clutter.

scripts/exregional_run_post.sh: (A), (G), (H)

tests/baseline_configs/config.EMC_CONUS_3km.sh:
* Add this new WE2E test configuration.  This tests the workflow on EMC's 3km CONUS grid in NCO mode (i.e. RUN_ENVIR set to "nco").

tests/baseline_configs/config.regional_005.sh:
* Specify non-default computational resources for this WE2E test (the 50km Alaska domain) such that the make_ics, make_lbcs, and run_post tasks do not crash (the number of MPI processes cannot be too high; otherwise, these tasks will fail).

tests/baseline_configs/config.regional_009.sh:
* Uncomment lines specific for Hera and comment out lines specific for Jet.  This has to be changed to include an if-statement that sets parameters based on the machine.

tests/baseline_configs/config.regional_015.sh:
* Add this new WE2E test configuration.  This tests the workflow on a coarse GFDLgrid type of grid that has a stretch factor of almost 1 (it cannot be exactly 1 because then the ufs_weather_model code assumes that it is a global grid and thus expects 6 tiles; this is a bug in the ufs_weather_model code that needs to be fixed).  A more refined version of this grid (refine_ratio changed from 2 as it is here to 36) is being tested by GSL until issues with the JPgrid verification are worked out.  Thus, we make sure capability of the workflow to run on this GFDLgrid doesn't break.

tests/baselines_list.txt:
* Update the list to include the new WE2E tests that are included in this commit (regional_014, regional_015, and EMC_CONUS_3km).  This file isn't really necessary and should be deleted at some point (e.g. in a different commit that fixes up various deficiencies with the WE2E tests).

ush/bash_utils/check_for_preexist_dir.sh:
* Modify the function this file defines such that it now works for files as well as directories.
* Rename the file and the function that it defines from check_for_preexist_dir(.sh) to check_for_preexist_dir_file(.sh).

ush/compare_config_scripts.sh:
* Change variable DEFAULT_EXPT_CONFIG_FN to EXPT_DEFAULT_CONFIG_FN (this variable is renamed in ush/setup.sh).

ush/config_defaults.sh:
* Put the variable SCHED specifying the job scheduler to use in this file so that it becomes user-specifiable (and avaiable as a workflow variable).  This may be needed when adding the capability to run outside of the scheduler (in a future commit/PR).
* Include detailed descriptions of the NCO mode variables (COMINgfs, STMP, NET, envir, RUN, PTMP) are used.
* Add new variable (EXPT_CONFIG_FN) that specifies the name of the experiment/workflow configuration file containing user-specified values.
* Add new variable (FV3_EXEC_FN) that specifies the name to use for the forecast model executable if/when it is copied from the directory in which the build step creates it to the directory in which the workflow scripts look for it (EXECDIR).  This copy step should really be done in during the build.
* For clarity and consistency, rename variables FV3_NML_FN and FV3_NML_CONFIG to FV3_NML_BASE_FN and FV3_NML_YAML_CONFIG_FN, respectively.
* Add new array SFC_CLIMO_FIELDS containing the names of the surface climatology fields used by the sfc_climo_gen code.
* Remove some variables (FNALBC, FNALBC2, etc) whose names are now specified in the new array FV3_NML_VARNAME_TO_SFC_CLIMO_FIELD_MAPPING.
* Add new array FIXgsm_FILES_TO_COPY_TO_FIXam that specifies the fixed files that must be copied from the system directory FIXgsm to the local directory FIXam (which is under the experiment directory).
* Add new array FV3_NML_VARNAME_TO_FIXam_FILES_MAPPING that specifies the FV3 namelist variable names and the corresponding files in the FIXam directory that they are associated with (the variables need to be set to the relative or full paths to these files).
* Add new array FV3_NML_VARNAME_TO_SFC_CLIMO_FIELD_MAPPING that specifies the FV3 namelist variable names and the corresponding surface climatology fields in the sfc_climo_gen code (the variables need to be set to the relative or full paths to the files containing the corresponding fields).
* Add new array CYCLEDIR_LINKS_TO_FIXam_FILES_MAPPING specifying the symlinks that need to be created in each cycle directory and their corresponding target files in the FIXam directory.
* Add new variables specifying the workflow task names and the corresponding values of the number of nodes, logical processes per node, and maximum wallclock time for each task.

ush/generate_FV3SAR_wflow.sh: (D)
* Remove variables PROC_RUN_FCST and NPROCS_RUN_FCST since they are now superceded by the variables NNODES_RUN_FCST and PPN_RUN_FCST.
* Remove the variable CYCLE_DIR since it is being replaced by CYCLE_BASEDIR (which is set in ush/setup.sh; see description of changes to that file).
* For clarity, reorder (and include comments between) the lines in the variable "settings" that specifies how to set the variables (entities, etc) in the template rocoto XML.
* Replace code that copies files from the FIXgsm directory with code that does the same thing using (looping over) the new workflow array variable FIXgsm_FILES_TO_COPY_TO_FIXam.
* Put in new code that copies the forecast model executable from the directoy in which the forecast model repository was cloned into the executable directory (EXECDIR) but ONLY IF such a copy does not
already exist in EXECDIR or, if it does exist, it is older than the source executable.  This copying should really be done in the build step, but it currently is not.
* For clarity, reorder some of the lines in the variable "settings" that is used to specify values in the FV3 fortran namelist file (input.nml) such that they are grouped by type (e.g. grid-related, computational, etc).  Also, include some comments between the lines.
* Remove the hard-coded settings of the "FN..." variables from the "settings" variable.  Instead:
  * For the "FN..." variables that specify paths to fixed files in the FIXam directory, loop through the new workflow array variable FV3_NML_VARNAME_TO_FIXam_FILES_MAPPING and, for each element of this array, create an appropriate entry in "settings".
  * For the "FN..." variables that specify the paths to surface climatology files:  (1) If not running the make_grid task, call the new function set_FV3nml_sfc_climo_filenames that sets those FV3 namelist variables that specify the paths to surface climatology files to the names of the surface climatology files on the FV3SAR grid (these files are either pregenerated or are created by the make_sfc_climo task).  (2) If running the make_grid task, call this same funtion in the make_grid task.  This is because the names of the surface climatology files depend on the CRES parameter (the C-resolution of the grid), and usually, that is not known until the grid is created (by the make_grid task).

ush/link_fix.sh:
* Replace local variable sfc_climo_fields with new workflow array variable SFC_CLIMO_FIELDS.
* Add code that creates symlinks in the FIXsar directory to surface climatology files with names that end in "tile7.nc" that have names that end in "tile1.nc".  This is apparently necessary for FV3 to be able to use the surface climatology files on the native FV3SAR grid (which are either pregenerated or created by the make_sfc_climo task).

ush/load_modules_run_task.sh: (D)
* Fix indentation.

ush/set_FV3nml_sfc_climo_filenames.sh:
* This is a new file that defines a function (set_FV3nml_sfc_climo_filenames) that sets the values of the variables in the forecast model's namelist file that specify the paths to the surface climatology files on the FV3SAR native grid (which are either regenerated or created by the MAKE_SFC_CLIMO_TN task).  We make this a function because it is needed in a couple of places in the scripts.

ush/set_fix_filenames.sh:
* Remove this file since the fixed files are being handled differently now using the new workflow array variables defined in ush/config_defaults.sh.

ush/set_ozone_param.sh:
* This is a new file that defines a function (set_ozone_param) that sets various workflow parameters depending on which ozone parameterization is being used by the experiment.  The ozone parameterization is determined by inspecting the CCPP suite defintion file.

ush/set_predef_grid_params.sh: (G)
* Set BLOCKSIZE to an allowed value for the EMC_CONUS_3km grid.

ush/setup.sh: (D), (G)
* For clarity, rename variable DEFAULT_EXPT_CONFIG_FN to EXPT_DEFAULT_CONFIG_FN.
* Remove variable EXPT_CONFIG_FN since that is now defined in config_defaults.sh.
* Set SCHED to a machine-dependent value only if it is not already specified by the user in config.sh.
* Remove stanza for machine THEIA (macine no longer exists).
* Change SCHED to lowercase (since that is how rocoto accepts it) and Test to make sure that SCHED is set to a valid value.
* Remove setting of variables that specify the task names since those are now in config_defaults.sh.
* Set the new workflow variable CYCLE_BASEDIR that contains the base directory under which the cycle directories are created.  This replaces CYCLE_DIR.  Note that CYCLE_BASEDIR is the old CYCLE_DIR but without the last directory (which is the CDATE of the cycle).  Now, CYCLE_DIR is formed within the XML by combining CYCLE_BASEDIR and the current cycle's date information from rocoto.  Also, add CYCLE_BASEDIR to the variable definitions file (var_defns.sh).
* For clarity, rename variable FV3_NML_CONFIG_FP to FV3_NML_YAML_CONFIG_FP.
* For convenience, create variable FV3_NML_FN (that is set to the name of the namelist file that FV3 expects to read in).  Then use this to set FV3_NML_FP, which is the full path to that namelist file.
* Create new workflow variable FV3_EXEC_FP that conains the full path to the copy of the FV3SAR executable in the executables directory EXECDIR.  Add this new variable to the variable definitions file (in a "cat" statement towards the end of the file).
* Set the new variable LOAD_MODULES_RUN_TASK_FP that specifies the full path to the script that loads the modules for each task and then runs the task.  This used to be hard-coded inside of the rocoto XML but is now needed as a workflow variable (because it is needed in at least a couple of places).  Add this variable to the variable definitions file (var_defns.sh).
* Fix bug (the way the local variable glob_pattern is set) to allow for the use of a dot as the fixed file name separator when in NCO mode (i.e. when RUN_ENVIR is set to "nco").
* Rename the variable NUM_NODES to NNODES_RUN_FCST to be consistent with the naming convention for number of nodes used for other tasks.
* Remove call to function set_fix_filenames since it is no longer needed.
* Remove recording of the array variables FIXgsm_FILENAMES and FIXam_FILENAMES in the variable defintions file since these arrays are no longer needed (they were previously set by the function set_fix_filenames).
* Call the new function set_ozone_param that sets various workflow parameters depending on which ozone parameterization is being used (which is specified in the CCPP suite defintion file).
* Clean up comments.

ush/source_util_funcs.sh: (D)
* Change the file name check_for_preexist_dir.sh to check_for_preexist_dir_file.sh.

ush/templates/FV3SAR_wflow.xml:
* For clarity, reorder the entities that contain jinja parameters by grouping similar items together.
* Make LOAD_MODULES_RUN_TASK_FP an ENTITY that is set externally by the workflow generation scripts (as opposed to hard-coded in the XML) because this is now a variable that is needed in another file (the script that is used to run a task outside of rocoto).
* Replace CYCLE_DIR as an externally specified ENTITY with CYCLE_BASEDIR.
* Create new externally-specified ENTITYs for the number of nodes, the number of MPI processes per node, and the wallclock time for each task.  These will get set by the workflow generation scripts.
* Make LOAD_MODULES_RUN_TASK_FP an ENTITY that is set externally by the workflow generation scripts (as opposed to hard-coded in the XML) because this is now a variable that is needed in another file (the script that is used to run a task outside of rocoto).
* Use the ENTITYs for the number of nodes, the number of MPI processes per node, and the wallclock time for each task to set the PROC_... and RSRC_... ENTITYs for each task (as opposed to hard-coding the latter).
* For brevity, rename ENTITYs containing the strings "MAKE_ICS_SURF_LBC0" and "MAKE_LBC1_TO_LBCN" to "MAKE_ICS" and "MAKE_LBCS", respectively.
* For consistency with other tasks, rename PROC_POST and RSRC_POST to PROC_RUN_POST and RSRC_RUN_POST.
* Change name of ENTITY RSRV_RUN_FCST to RSRV_FCST to be consistent with name of ENTITY QUEUE_FCST.  Names of ENTITYs related to the job scheduler need to be revisited (e.g. maybe change QUEUE_FCST to QUEPART_FCST since in slurm, there are partitions, not queues).
* Remove variable PDY from pre-processing tasks (which are cycle-independent).
* Fix bug where the task name "make_grid" is hard-coded as opposed to using the ENTITY MAKE_GRID_TN.
* For clarity, rename file JREGIONAL_GET_EXTRN_FILES to JREGIONAL_GET_EXTRN_MDL_FILES.
* Reorder environment variable settings in each task to go from least to most general.
* Within each cycle-dependent task, set CYCLE_DIR based on CYCLE_BASEDIR.

ush/valid_param_vals.sh:
* Add valid values for the new user-specifiable workflow variable SCHED.
* Remove the unused predefined grid GSD_HRRR_AK_3km since it has been renamed GSD_RRFSAK_3km.

* Manual merge of Mike's changes for file ush/load_modules_run_task.sh.

* Improve/streamline generation of rocoto workflow XML.

Modifications made to more than one file:
----------------------------------------
(A) For uniformity among workflow tasks, split the worfklow variables that specify the number of nodes, logical processes per node, and wall clock time for the generic GET_EXTRN_MDL_FILES task into two variables each -- one for the GET_EXTRN_ICS_TN and another for the GET_EXTRN_LBCS_TN task.
(B) Remove USHDIR as an ENTITY in the XML since it is not used after being set.

ush/config_defaults.sh: (A)

ush/generate_FV3SAR_wflow.xml: (B)

ush/templates/FV3SAR_wflow.xml: (A), (B)
* Clean up comments.
* To reduce the number of entities rocoto has to deal with, remove the ENTITY definitions for various entities that only appear in one location in the file and replace them with the corresponding jinja2 placeholders or combinations of such placeholders.  Do this for the following entities:
  * The per-task entities NNODES_..., PPN_..., WTIME_..., PROC_..., and RSRC_....
  * EXTRN_MDL_NAME_ICS and EXTRN_MDL_NAME_LBCS

* Get EMC's changes into community workflow (Ratko's PR plus other necessary changes).

File-by-file description of modifications:
-----------------------------------------

fix/fix_upp/nam_nests.hiresf_nn.txt:
* Deleted as requested by EMC (from Ratko's original PR).

fix/fix_upp/postxconfig-NT-fv3sar.txt
* Modified as requested by EMC (from Ratko's original PR).

scripts/exregional_get_extrn_files.sh:
* If extracting the external model files from one or more HPSS archive files, put in a check to ensure that each external model file was extracted exactly once from one of the archive files.

scripts/exregional_make_ics.sh:
* Add parameter settings for the FV3_GFS_2017_gfdlmp_regional physics suite (from Ratko's original PR).
* The FV3_GFS_2017_gfdlmp_regional physics suite requires a variable called tref (reference temperature) in the initial conditions.  That variable is available only for external files coming from the FV3GFS model.  tref can be extracted from the surface analysis file if the namelist variable convert_nst is set to .true. in the chgres_cube namelist file.  Thus, create a new local variable named convert_nst that gets set to true for the FV3GFS model but is false otherwise and pass it to the chgres_cube namelist.

scripts/exregional_make_lbcs.sh:
* Add parameter settings for the FV3_GFS_2017_gfdlmp_regional physics suite (from Ratko's original PR).

tests/baseline_configs/config.EMC_CONUS_3km.sh:
* Rename to tests/baseline_configs/config.nco_conus.sh (see that file).

tests/baseline_configs/config.nco_conus.sh:
* Renamed from tests/baseline_configs/config.EMC_CONUS_3km.sh.
* Change queue from debug to batch since we always want batch as the default (so that if running many tests, they are not limited by the max number of jobs a user can have in the debug queue, which on hera is just 2).
* Set the physics suite for this WE2E test to FV3_GFS_2017_gfdlmp_regional because that's the one EMC runs with.
* Reset the RUN variable (which is used in creating output directory if in NCO mode) to a value that is unique to this WE2E test so that it doesn't interfere with other tests that are running at the same time (and vice versa).

tests/baseline_configs/config.nco_conus_c96.sh:
* New WE2E test for the coarse EMC CONUS grid.  For rapid testing, this can be run instead of config.nco_conus.sh.

tests/baseline_configs/config.new_JPgrid.sh
* New WE2E test for specifying a new (i.e. not predefined) JPgrid.

tests/baseline_configs/config.regional_009.sh:
* Reset the RUN variable (which is used in creating output directory if in NCO mode) to a value that is unique to this WE2E test so that it doesn't interfere with other tests that are running at the same time (and vice versa).

tests/baselines_list.txt:
* Update the full list of WE2E test names to the latest set.

ush/config_defaults.sh:
* Edit comments.

ush/generate_FV3SAR_wflow.sh:
* Remove FNZORC, FNTSFA, FNACNA, and FNSNOA from the "settings" variable used in generating the FV3 input namelist file.  These variables are now instead specified in the base namelist file (ush/templates/input.nml.FV3) because they are the same for all physics suites considered thus far.

ush/get_extrn_mdl_file_dir_info.sh:
* Remove unused code.

ush/link_fix.sh:
* Bug fix:  Change the targets of the "...tile1..." symlinks so that they point to the tile7.halo0.nc files, not the tile7.nc symlinks (which in turn point to the tile7.halo4.nc files, which is not what we want).

ush/set_namelist.py:
* Add option "sort=True" to nml.write method to sort the resulting FV3 namelist file in alphabetical order.  This makes it easier to compare (diff) namelists.

ush/setup.sh:
* Add a temporary section of code that makes fixes to the suite defintion file for the FV3_GFS_2017_gfdlmp_regional physics suite.  This is necessary because the version of the file in the dtc/develop branch of the NCAR fork of the fv3atm repo has not yet been updated to have these corrections.  This code should be removed once that branch/repo have been updated.

ush/templates/FV3.input.yml:
* Add section for the new FV3_GFS_2017_gfdlmp_regional physics suite.  Note that this references the preexisting FV3_GFS_2017_gfdlmp physics suite.

ush/templates/input.nml.FV3:
* Reorder lines so that both namelists and the variables within each namelist are lower case and in alphabetical order.  This makes it easier to compare this base FV3 namelist file to other FV3 namelist files generated using the set_namelist.py script.

ush/templates/diag_table.FV3_GFS_2017_gfdlmp_regional:
* New diag_table file for FV3_GFS_2017_gfdlmp_regional suite.

ush/templates/field_table.FV3_GFS_2017_gfdlmp_regional:
* New field_table file for FV3_GFS_2017_gfdlmp_regional suite.

ush/templates/model_configure.FV3_GFS_2017_gfdlmp_regional:
* New model_configure file for FV3_GFS_2017_gfdlmp_regional suite.

ush/valid_param_vals.sh:
* Add FV3_GFS_2017_gfdlmp_regional to the list of valid physic suites (as requested by EMC).

* Change compile options for EMC as requested by Ratko.

* Allow names of external model files to be different on disk vs. from HPSS (#221)

Allow names of external model files to be different on disk vs. from HPSS.

Motivation:
----------
At least on jet, the names of the external model files can be different depending on whether they are being copied from disk or are being extracted from an archive file in HPSS (for the same external model).  Currently, the workflow assumes that these file names are the same regardless of where/how they are being obtained.  Thus, the workflow breaks on jet.  This PR generalizes the workflow to allow for the external model file names to be different (if necessary) depending on where/how they're being obtained.

Summary of Modifications:
------------------------
* Allow the names of the external model files copied from or linked to on disk to be different than the ones extracted from an archive file in HPSS.  This is the case on jet.
* In each of the get_extrn_ics and get_extrn_lbcs workflow tasks, after obtaining the external model files (either from disk or from HPSS), create a new file and save in it certain parameters that depend on this process of obtaining the external model files (e.g. the names of the external model files that were obtained, since now these names depend on whether they are obtained from disk or HPSS).  Note that one file is created for the get_extrn_ics task and another one for the get_extrn_lbcs task.  These two files are actually shell scripts containing variable definitions that get sourced later by the make_ics and make_lbcs tasks, respectively.  This change removes the need to call the get_extrn_mdl_file_dir_info again as part of the make_ics and make_lbcs tasks.
* Modify the way the temporary fix for the suite_FV3_GFS_2017_gfdlmp_regional physics suite works such that the suite definition file (SDF) is not changed in the ufs_weather_model directory where the clone of the ufs-weather-model repo is located (so that no changes to tracked files are reported in the clone of that repo) and instead, the SDF is first copied to the experiment directory and changed there.


Modifications common to more than one file (used below in listing of file-by-file modifications):
------------------------------------------------------------------------------------------------
(A) Rename local variables to be lower case.
(B) Fix/add/delete comments and/or informational and/or error messages.
(C) Remove theia stanza in machine-dependent case-statement.
(D) Remove calls to save_shell_opts and restore_shell_opts in the machine-dependent case-statement because those calls were to suppress output from the "module load" commands that were previously located in this case-statement, but the "module load"s have been moved out of this case-statement and into the ush/load_modules_run_task.sh script.
(E) Remove the call to the get_extrn_mdl_file_dir_info funtion because the information that this call returned is now obtained by sourcing new variable definition files created by the get_extrn_ics and get_extrn_lbcs workflow tasks.  The full paths to these new files (actually shell scripts that just contain the variable definitions) are, respectively:
  ${EXPT_DIR}/${CYCLE_DIR}/${EXTRN_MDL_NAME_ICS}/ICS/extrn_mdl_ics_var_defns.sh
  ${EXPT_DIR}/${CYCLE_DIR}/${EXTRN_MDL_NAME_LBCS}/LBCS/extrn_mdl_ics_var_defns.sh
(F) Remove some of the arguments to the call to scripts/exregional_make_ics.sh since those arguments are now obtained by sourcing the new variable definitions file extrn_mdl_ics_var_defns.sh created by the get_extrn_ics task.
(G) Remove some of the arguments to the call to scripts/exregional_make_lbcs.sh since those arguments are now obtained by sourcing the new variable definitions file extrn_mdl_lbcs_var_defns.sh created by the get_extrn_lbcs task.
(H) For clarity, change name of workflow variable from EXTRN_MDL_LBC_UPDATE_FHRS to EXTRN_MDL_LBC_SPEC_FHRS (where "SPEC_FHRS" stands for specification forecast hours, because the LBCs aren't being updated at these forecast hours, they're being specified).  Also, do same for the local (i.e. lowercase) version of this variable.
(I) For clarity, change name of workflow variable from LBC_UPDATE_INTVL_HRS to LBC_SPEC_INTVL_HRS for the same reason as explained above in (I).
(J) Change LBC_SPEC_INTVL_HRS from 6 to 3 to test having more than one LBC specification time after hour 0 (the forecast goes to hour 6).  For now, do this only for those WE2E tests having both EXTRN_MDL_NAME_ICS and EXTRN_MDL_NAME_LBCS set to "FV3GFS".  Don't do it for the external model being set to "GSMGFS" because that model does not provide output with 3-hour frequency, only 6-hour frequency (I think).

File-by-file modifications:
--------------------------

jobs/JREGIONAL_GET_EXTRN_MDL_FILES: (A), (B), (H)
* Use the function check_var_valid_vals to ensure that the value specified for ICS_OR_LBCS is valid.
* In the call to the function get_extrn_mdl_file_dir_info, instead of returning only one set of external model file names (varname_extrn_mdl_fns), return two sets -- one if the files are on disk (varname_extrn_mdl_fns_on_disk) and another if the files are in archive file on HPSS (varname_extrn_mdl_fns_in_arcv).

jobs/JREGIONAL_MAKE_ICS: (A), (B), (C), (D), (E), (F)

jobs/JREGIONAL_MAKE_LBCS: (A), (B), (C), (D), (E), (G)

jobs/JREGIONAL_RUN_POST: (B)

scripts/exregional_get_extrn_files.sh:
* Rename to scripts/exregional_get_extrn_mdl_files.sh.  See below for other changes.

scripts/exregional_get_extrn_mdl_files.sh: (A), (B), (H)
* Add new input argument ics_or_lbcs instead of assuming that an environment variable named ICS_OR_LBCS exists.
* Add new input argument extrn_mdl_cdate that will be written to the new variable definition file (extrn_mdl_ics_var_defns.sh or extrn_mdl_lbcs_var_defns.sh) that this script now creates (see bullet point below).
* Replace input argument EXTRN_MDL_FNS with the two new input arguments extrn_mdl_fns_on_disk and extrn_mdl_fns_in_arcv that are the external model file names if copying them from disk and if extracting them from an archive file(s), respectively.  Also, replace EXTRN_MDL_FNS throughout the script with either extrn_mdl_fns_on_disk or extrn_mdl_fns_in_arcv as appropriate.  This is part of the modifications to allow the names of the external model files copied from or linked to on disk to be different than the ones extracted from an archive file in HPSS (which is the case on jet).
* Remove the fix for resetting external model file names on jet because this is now done by the use of two different input arguments for the file names (extrn_mdl_fns_on_disk and extrn_mdl_fns_in_arcv).
* Add code that, after the external model files are obtained (copied, linked to, or extracted from archive), creates a new file (a shell script) that contains the definitions of several external-model related variables that will be needed by downstream tasks.  For the ICs (i.e. the get_extrn_ics task), the full path to this file is at
  ${EXPT_DIR}/${CYCLE_DIR}/${EXTRN_MDL_NAME_ICS}/ICS/extrn_mdl_ics_var_defns.sh
For the LBCs (i.e. the get_extrn_ics task), the full path to this file is at
  ${EXPT_DIR}/${CYCLE_DIR}/${EXTRN_MDL_NAME_LBCS}/LBCS/extrn_mdl_ics_var_defns.sh
These files are sourced by the make_ics and make_lbcs tasks, respectively, to obtain the necessary variable definitions.

scripts/exregional_make_ics.sh: (A), (B), (F)

scripts/exregional_make_lbcs.sh: (A), (B), (G), (H)

tests/baseline_configs/config.nco_conus.sh: (I), (J)

tests/baseline_configs/config.nco_conus_c96.sh: (I), (J)

tests/baseline_configs/config.new_JPgrid.sh: (I), (J)

tests/baseline_configs/config.regional_001.sh: (I)

tests/baseline_configs/config.regional_002.sh: (I), (J)

tests/baseline_configs/config.regional_003.sh: (I)

tests/baseline_configs/config.regional_004.sh: (I)

tests/baseline_configs/config.regional_005.sh: (I)

tests/baseline_configs/config.regional_006.sh: (I), (J)

tests/baseline_configs/config.regional_007.sh: (I), (J)

tests/baseline_configs/config.regional_008.sh: (I), (J)

tests/baseline_configs/config.regional_009.sh: (I)

tests/baseline_configs/config.regional_010.sh: (I)

tests/baseline_configs/config.regional_011.sh: (I)

tests/baseline_configs/config.regional_012.sh: (I)

tests/baseline_configs/config.regional_013.sh: (I)

tests/baseline_configs/config.regional_014.sh: (I)

tests/baseline_configs/config.regional_015.sh: (I), (J)

ush/config_defaults.sh: (B), (I)
* Add the new workflow variables EXTRN_MDL_ICS_VAR_DEFNS_FN and EXTRN_MDL_LBCS_VAR_DEFNS_FN that specify the names of the two new external-model-associated variable definitions files that the get_extrn_ics and get_extrn_lbcs now create.
* Remove extra spaces at ends of elements of array CYCLEDIR_LINKS_TO_FIXam_FILES_MAPPING.

ush/generate_FV3SAR_wflow.sh: (I)
* Put in an if-statement that renames the temporary SDF (suite_FV3_GFS_2017_gfdlmp_regional.xml.tmp) created in the experiment directory by the temporay fix in ush/setup.sh (see below) for the FV3_GFS_2017_gfdlmp_regional physics suite to the actual suite file (suite_FV3_GFS_2017_gfdlmp_regional.xml).  This code needs to be removed once the SDF for this physics suite is fixed in the dtc/develop branch of the NCAR fork of the ufs-weather-model repo.

ush/get_extrn_mdl_file_dir_info.sh: (A), (C), (H)
* In the arguments list, instead of returning only one set of external model file names (varname_extrn_mdl_fns), return two sets -- one if the files are on disk (varname_extrn_mdl_fns_on_disk) and another if the files are in archive file on HPSS (varname_extrn_mdl_fns_in_arcv).
* Remove some unused code.
* Add new local variables needed to create sets of file names for both the on-disk case and the in-archive-file case.
* Allow the input argument anl_or_fcst to contain a value that is either upper case or lower case (instead of just upper case as before).
* Change code to specify both the local array variables fns_on_disk and fns_in_arcv instead of the single array variable fns.  These two new variables are (so far) the same on all machines except Jet, in which case they may be different depending on the external model considered.
* Change the way output variables are set so that each is set only if the corresponding input argument that specifies its name is not empty.  This way, if certain output variables are not needed, the user can simply not specify them in the call to this function (in which case the process_args function will simply set them to empty strings) and not obtain any errors.

ush/set_ozone_param.sh: (B)

ush/setup.sh: (A), (I)
* Change the code that is a temporary fix for the fact that the SDF for the FV3_GFS_2017_gfdlmp_regional physics suite in the dtc/develop branch of the NCAR fork of the ufs-weather-model repo is currently incorrect so that it first copies the suite defintion file to the experiment directory and makes fixes to it there (instead of fixing it directly in its original location at sorc/ufs_weather_model/...).  As part of this change:
  a) Move the creation of the experiment directory (EXPTDIR) to before the temporary code.
  b) Move the original code that calls the function set_ozone_param in the else" portion of the "if" statement that runs the temporary code.
  These two changes can be undone later after the temporary code is removed.

* Fix for contrib module handling. (#226)

## DESCRIPTION OF CHANGES:.
All the module files contained in this repository have been updated for the change on Jet/Hera to remove the contrib module, and instead use the `module use -a` method for locating necessary contrib package modulefiles. This will be the ONLY way to use these modules after June 11, 2020, and is already in place to work now. As such, this is a **time sensitive** PR.

## TESTS CONDUCTED:.
Loaded a few different modules on front end node. This included modules that need to be loaded, and those "modules" that are actually shell scripts and must be sourced. Both methods worked as expected.

End-to-end tests completed successfully on Hera: 
new_JPgrid
regional_002
regional_003
regional_004
regional_005
regional_006
regional_007
regional_008
regional_015

* Fix directory overwrite bug in NCO mode. (#222)

Bug fix to get the WE2E tests to work properly in NCO mode.

Summary of modifications:
------------------------
* Modify scripts and WE2E configuration files that run in NCO mode to prevent simultaneously running experiments in NCO mode from writing to the same set of directories.
* Add a WE2E test on the GSD_RAP13km grid.

Modifications common to more than one file (used below in listing of file-by-file modifications):
------------------------------------------------------------------------------------------------
(A) Fix/add/delete comments and/or informational and/or error messages.
(B) Remove unused code and/or fix code indentation.
(C) Set the variables RUN and envir to EXPT_SUBDIR, which when the WE2E test is run will contain the name of the test.  Note that:
  * We can't get the name of the WE2E test from the file name because before running the experiment generation script for this WE2E test, the script that runs the WE2E tests copies this configuration file into ush and renames it config.sh (because that's the name the experiment generation script expects the configuration file to have).  Thus, at the time the experiment generation script is called, the file name no longer contains the name of the test, so it cannot be used to obtain the test name.
  * Even though EXPT_SUBDIR is set to a null string near the beginning of this file, the script that runs the WE2E tests inserts the name of the test into the definition of EXPT_SUBDIR (i.e. it changes the line 'EXPT_SUBDIR=""' into 'EXPT_SUBDIR="test_name"') before calling the experiment generation script.  Thus, EXPT_SUBDIR will be properly set to the name of the test when the experiment generation script is called.
(D) Bug fix - change variable name from LBC_UPDATE_INTVL_HRS to LBC_SPEC_INTVL_HRS.

File-by-file description of modifications:
-----------------------------------------

jobs/JREGIONAL_RUN_POST:
* In the definition of COMOUT, use the new workflow variable COMOUT_BASEDIR that specifies the base directory under which each cycle's UPP output directories will be placed.

scripts/exregional_get_extrn_files.sh:
* Remove this file.  This has been replaced with scripts/exregional_get_extrn_mdl_files.sh.

tests/baseline_configs/config.GSD_RAP13km.sh:
* This is a new WE2E test on the GSD_RAP13km grid using the FV3_GSD_v0 physics suite.

tests/baseline_configs/config.nco_conus.sh: (A), (C)

tests/baseline_configs/config.nco_conus_c96.sh: (A), (C)

tests/baseline_configs/config.regional_001.sh: (A)

tests/baseline_configs/config.regional_002.sh: (A)

tests/baseline_configs/config.regional_003.sh: (A)

tests/baseline_configs/config.regional_004.sh: (A)

tests/baseline_configs/config.regional_005.sh: (A)

tests/baseline_configs/config.regional_006.sh: (A)

tests/baseline_configs/config.regional_007.sh: (A)

tests/baseline_configs/config.regional_008.sh: (A)

tests/baseline_configs/config.regional_009.sh: (A), (B), (C)
* Change the time interval for specifying external model LBCs from 6 hrs to 3 hrs (to match what is done in all the other WE2E tests that use FV3GFS as the external model for LBCs).

tests/baseline_configs/config.regional_010.sh: (A)

tests/baseline_configs/config.regional_011.sh: (A)

tests/baseline_configs/config.regional_012.sh: (A)

tests/baseline_configs/config.regional_013.sh: (A)

tests/baseline_configs/config.regional_014.sh: (A)

tests/baseline_configs/config.regional_015.sh: (A)

tests/baselines_list.txt:
* Add the new GSD_RAP13km test to the full list of WE2E tests.

tests/run_experiments.sh: (A), (B)

ush/config.community.sh: (D)

ush/config.nco.sh: (D)

ush/config_defaults.sh: (A)
* Increase wallclock times for certain tasks since on hera some of the WE2E tests aren't able to complete within the allowed time.

ush/set_predef_grid_params.sh:
* Change the time step on the GSD_RAP13km grid from 90 sec to 50 sec to get a stable forecast.

ush/setup.sh: (A)
* For simplicity, combine two blocks of code that create different directories into a single block (that creates both sets of directories).
* In NCO mode, change defintion of CYCLE_BASEDIR from "$STMP/tmpnwprd/${EMC_GRID_NAME}" to "$STMP/tmpnwprd/$RUN".  This is to make it easier to associate this directory with an experiment (wich can be done by setting RUN to the experiment name in the experiment's configuration file).  That way, two experiments running simultaneously (but having different names) will not end up writing to the same CYCLE_BASEDIR (which currently can happen if the two experiments are using the same grid, i.e. the same value of EMC_GRID_NAME).
* Add COMOUT_BASEDIR as a new secondary workflow variable.  In NCO mode, set this to the base directory under which the output from the RUN_POST_TN task will be placed (i.e. the cycle-independent portion of the RUN_POST_TN task's output directory) and then check whether this directory exists (and deal with it as specified by the variable PREEXISTING_DIR_METHOD).  The goal here is to be able to uniquely associate this directory to an experiment so that simultaneously running experiments do not write their UPP outputs into the same directory (and possibly overwrite each other's results).  This directory is given by COMOUT_BASEDIR="$COMROOT/$NET/$envir", so we can uniquely associate it with an experiment by setting envir to the name of the experiment (which can be done in the experiment's configuration file).  This is why in the WE2E configuration files listed above (config.nco_conus.sh, config.nco_conus_c96.sh, and config.regional_009.sh), we set envir to the name of the experiment.  In community mode, the new variable COMOUT_BASEDIR is not used, so in that case we set it to an empty string.

* WCOSS port changes (#228)

Co-authored-by: Benjamin Blake <[email protected]>

Not tested on hera yet.  Will do soon.

Co-authored-by: BenjaminBlake-NOAA <[email protected]>
Co-authored-by: Benjamin.Blake EMC <[email protected]>
Co-authored-by: Michael Kavulich <[email protected]>
Co-authored-by: gsketefian <[email protected]>
Co-authored-by: Christina Holt <[email protected]>

* Modify link to module file for forecast run.  modules.nems is no longer
created by the cmake, static build for the ufs-community/ufs-weather-model.
Use modulefiles/$machine.intel/fv3.

Co-authored-by: BenjaminBlake-NOAA <[email protected]>
Co-authored-by: Benjamin.Blake EMC <[email protected]>
Co-authored-by: Michael Kavulich <[email protected]>
Co-authored-by: gsketefian <[email protected]>
Co-authored-by: Christina Holt <[email protected]>
epic-cicd-jenkins pushed a commit that referenced this pull request Apr 17, 2023
@gspetro-NOAA @gspetro
Update SRW Documentation (#212)
31dab61 
* updated docs

* added git submodule

* fix formatting

* added new submodule commits

* fixed ref links

* finished Intro

* finish Components & Intro edits

* edited Rocoto workflow section of Quickstart

* added minor hpc submodule commits

* Updates to Rocoto Workflow in Quick Start

* add to HPC-stack intro

* submodule updates

* added submodule docs edits

* hpc-stack updates & formatting fixes

* hpc-stack intro edits

* bibtex attempted fix

* add hpc-stack module edits

* update sphinxcontrib version

* add .readthedocs.yaml file

* update .readthedocs.yaml file

* update .readthedocs.yaml file

* update conf.py

* updates .readthedocs.yaml with submodules

* updates .readthedocs.yaml with submodules

* submodule updates

* submodule updates

* minor Intro edits

* minor Intro edits

* minor Intro edits

* submodule updates

* fixed typos in QS

* QS updates

* QS updates

* QS updates

* updates to InputOutput and QS

* fix I/O doc typos

* pull updates to hpc-stack docs

* pull updates to hpc-stack docs

* fix table wrapping

* updates to QS for cloud

* fix QS export statements

* fix QS export statements

* QS edits on bind, config

* add bullet points to notes

* running without rocoto

* add HPC-Stack submodule w/docs

* split QS into container/non-container approaches

* added filepath changes for running in container on Orion, et al.

* edits to overview and container QS

* moved CodeReposAndDirs.rst info to the Introduction & deleted file

* continued edits to SRWAppOverview

* combine overview w/non-container docs

* finish merging non-container guide & SRWOverview, rename/remove files, update FAQ

* minor edits for Intro & QS

* updates to BuildRun doc through 3.8.1

* edits to Build/Run and Components

* remove .gitignore

* fix Ch 3 title, 4 supported platform levels note

* fix typos, add term links

* other minor fixes/suggestions implemented

* updated Intro based on feedback; changed SRW to SRW App throughout

* update comment to Intro citation

* add user-defined vertical levels to future work

* Add instructions for srw_common module load

* fix typo

* update Intro & BuildRunSRW based on Mark's feedback

* minor intro updates

* 1st round of jwolff's edits

* 2nd round of jwolff updates

* update QS intro

* fix minor physics details

* update citation and physics suite name

* add compute node allocation info to QS

* add authoritative hpc-stack docs to Intro

Co-authored-by: gspetro <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@WenMeng-NOAA WenMeng-NOAA WenMeng-NOAA approved these changes

@yangfanglin yangfanglin Awaiting requested review from yangfanglin

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@junwang-noaa @WenMeng-NOAA