Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[NOAA] NASA fork sync. (#37) #30

Merged
merged 1 commit into from
Nov 7, 2023
Merged

[NOAA] NASA fork sync. (#37) #30

merged 1 commit into from
Nov 7, 2023

Conversation

FlorianDeconinck
Copy link
Collaborator

  • Initialize GeosDycoreWrapper with bdt (timestep)

  • Use GEOS version of constants

    1. Add qcld to the list of tracers beings advected
  1. Made GEOS specific changes to thresholds in saturation adjustment
  • Accumulate diss_est

  • Allow GEOS_WRAPPER to process device data

  • Add clear to collector for 3rd party use. GEOS pass down timings to caller

  • Make kernel analysis run a copy stencil to compute local bandwith Parametrize tool with backend, output format

  • Move constant on a env var Add saturation adjustement threshold to const

  • lint

  • More linting

  • Remove unused if leading to empty code block

  • Restrict dace to 0.14.1 due to a parsing bug

  • Add guard for bdt==0 Fix bad merge for bdt with GEOS_Wrapper

  • Remove unused code

  • Fix theroritical timings Lint

  • Fixed a bug where pkz was being calculated twice, and the second calc was wrong

  • Downgrade DaCe to 0.14.0 pending array aliasing fix

  • Set default cache path for orchestrated DaCe to respect GT_CACHE_* env

  • Remove previous per stencil override of default_build_folder

  • Revert "Set default cache path for orchestrated DaCe to respect GT_CACHE_* env"

This reverts commit 4fc5b4d.

  • Revert "Remove previous per stencil override of default_build_folder"

This reverts commit 2245027.

  • Read cache_root in default dace backend

  • Document faulty behavior with GT_CACHE_DIR_NAME

  • Fix bad requirements syntax

  • Check for the string value of CONST_VERSION directly instead of enum

  • Protect constant selection more rigorusly. Clean abort on unknown constant given

  • Log constants selection

  • Refactor NQ to constants.py

  • Replace all logger with pace_log Introduce PACE_LOGLEVEL to control log level from outside

  • Code guidelines clean up

  • Devops/GitHub actions on (Devops/GitHub actions on #15)

  • Linting on PR

  • Run main unit test

  • Update python to available 3.8.12

  • Remove cd to pace

  • Lint: git submodule recursive

  • Typo

  • Add openmpi to the image

  • Linting

  • Fix unit tests (remove dxa, dya rely on halo ex)

  • typo

  • Change name of jobs

  • Distributed compilation on orchestrated backend for NxN layouts (Distributed compilation on orchestrated backend for NxN layouts #14)

  • Adapt orchestration distribute compile for NxN layout

  • Remove debug code

  • Add a more descriptive string base postfix for cache naming Identify the code path for all cases
    Consistent reload post-compile
    Create a central space for all caches generation logic No more original layout check required

  • Add a test on caches relocatability

  • Verbose todo

  • Linting on PR

  • Run main unit test

  • Update python to available 3.8.12

  • Remove cd to pace

  • Lint: git submodule recursive

  • Typo

  • Add openmpi to the image

  • Linting

  • Fix unit tests (remove dxa, dya rely on halo ex)

  • typo

  • Change name of jobs

  • Missing enum

  • Lint imports

  • Fix unit tests

  • Deactivate relocability test due to Python crash Logged as issyue 16

  • Typo

  • Raise for 1,X and X,1 layouts which requires a new descriptor

  • Added ak, bk for 137 levels in eta.py

  • Add floating point precision to GEOS bridge init

  • lint

  • Add device PCI bus id (for MPS debug)

  • Typo + lint

  • Try to detect MPS reading the "log" pipe

  • Lint

  • Clean up

  • Log info GEOS bridge (Log info GEOS bridge #18)

  • Add floating point precision to GEOS bridge init

  • lint

  • Add device PCI bus id (for MPS debug)

  • Typo + lint

  • Try to detect MPS reading the "log" pipe

  • Lint

  • Clean up

  • Update geos/develop to grab NOAA PR9 results (Update geos/develop to grab NOAA PR9 results #21)

  • Verbose choice of block/grid size

  • added build script for c5

  • updated repo to NOAA

  • GEOS integration ([orch:dace:X] bad reinterpret cast leads to NaN #9)

  • Initialize GeosDycoreWrapper with bdt (timestep)

  • Use GEOS version of constants

    1. Add qcld to the list of tracers beings advected
  1. Made GEOS specific changes to thresholds in saturation adjustment
  • Accumulate diss_est

  • Allow GEOS_WRAPPER to process device data

  • Add clear to collector for 3rd party use. GEOS pass down timings to caller

  • Make kernel analysis run a copy stencil to compute local bandwith Parametrize tool with backend, output format

  • Move constant on a env var Add saturation adjustement threshold to const

  • Remove unused if leading to empty code block

  • Restrict dace to 0.14.1 due to a parsing bug

  • Add guard for bdt==0 Fix bad merge for bdt with GEOS_Wrapper

  • Remove unused code

  • Fix theroritical timings

  • Fixed a bug where pkz was being calculated twice, and the second calc was wrong

  • Downgrade DaCe to 0.14.0 pending array aliasing fix

  • Set default cache path for orchestrated DaCe to respect GT_CACHE_* env

  • Remove previous per stencil override of default_build_folder

  • Revert "Set default cache path for orchestrated DaCe to respect GT_CACHE_* env"

  • Revert "Remove previous per stencil override of default_build_folder"

  • Read cache_root in default dace backend

  • Document faulty behavior with GT_CACHE_DIR_NAME

  • Fix bad requirements syntax

  • Check for the string value of CONST_VERSION directly instead of enum

  • Protect constant selection more rigorusly. Clean abort on unknown constant given

  • Log constants selection

  • Refactor NQ to constants.py

  • Fix or explain inlined import

  • Verbose runtime error when bad dt_atmos

  • Verbose warm up

  • re-initialize heat_source and diss_est each call, add do_skeb check to accumulation



  1. Made GEOS specific changes to thresholds in saturation adjustment
  • Accumulate diss_est

  • Allow GEOS_WRAPPER to process device data

  • Add clear to collector for 3rd party use. GEOS pass down timings to caller

  • Make kernel analysis run a copy stencil to compute local bandwith Parametrize tool with backend, output format

  • Move constant on a env var Add saturation adjustement threshold to const

  • Restrict dace to 0.14.1 due to a parsing bug

  • Add guard for bdt==0

  • Fix theroritical timings

  • Fixed a bug where pkz was being calculated twice, and the second calc was wrong

  • Downgrade DaCe to 0.14.0 pending array aliasing fix

  • Set default cache path for orchestrated DaCe to respect GT_CACHE_* env

  • Remove previous per stencil override of default_build_folder

  • Revert "Set default cache path for orchestrated DaCe to respect GT_CACHE_* env"

  • Read cache_root in default dace backend

  • Document faulty behavior with GT_CACHE_DIR_NAME

  • Check for the string value of CONST_VERSION directly instead of enum

  • Protect constant selection more rigorusly. Clean abort on unknown constant given

  • Log constants selection

  • Refactor NQ to constants.py

  • Introduce PACE_LOGLEVEL to control log level from outside

  • Code guidelines clean up

  • Devops/GitHub actions on (Devops/GitHub actions on #15)

  • Linting on PR

  • Run main unit test

  • Update python to available 3.8.12

  • Fix unit tests (remove dxa, dya rely on halo ex)

  • Update HISTORY.md

  • Adapt log_level in driver.run

  • Verbose the PACE_CONSTANTS

  • Doc log level hierarchical nature


  • Lint

  • Update gt4py, dace, cleanup (Update gt4py, dace, cleanup #19)

  • Update gt4py to top of master on June 21

  • Update DaCe to 0.14.2 Workaround aliasing issue in FiniteVolumeTransport

  • Fix to gt4py storage

  • Downgrade to dace 0.14.1

  • DaCe to 0.14.4 Orchestrating NonHydrostaticPressureGradient
    Adptating code to newer gt4py

  • Regenerate constraints.txt

  • Default constants to GFS Fix snapshot for GPU runs
    Lint on ETA
    Fix log level

  • Remove daint_venv submodule

  • Adding dace as a submodule Removing buildenv as a submodule

  • Update gt4py to latest master

  • Skip ConstantPropagation during Simplify

  • Remove buidlenv

  • Update requirements_dev.txt

  • Add editable util to requirements_dev.txt

  • lint

  • scipy for tests is now needed

  • Pin DaCe to pace-fixes-0 merge

  • Remove logging setup in test_translate

  • Make cupy import robust to device not being available

  • Fix to GEOS bridge MPS detection

  • Up gt4py to August 14th EOD:

    • Hip/ROCm
    • New allocators
  • DaCE module: swap SSH for HTTPS (DaCe module: swap SSH for HTTPS #26)

  • GEOS GridTools stencils build override (GEOS GridTools stencils build override #27)

  • Stencil build override for GEOS

  • Deactivate warnings if PACE_LOGLEVEL is > WARNING

  • Better log level

  • Bad merge (again)

* Initialize GeosDycoreWrapper with bdt (timestep)

* Use GEOS version of constants

* 1. Add qcld to the list of tracers beings advected
2. Made GEOS specific changes to thresholds in saturation adjustment

* Accumulate diss_est

* Allow GEOS_WRAPPER to process device data

* Add clear to collector for 3rd party use. GEOS pass down timings to caller

* Make kernel analysis run a copy stencil to compute local bandwith
Parametrize tool with backend, output format

* Move constant on a env var
Add saturation adjustement threshold to const

* lint

* More linting

* Remove unused if leading to empty code block

* Restrict dace to 0.14.1 due to a parsing bug

* Add guard for bdt==0
Fix bad merge for bdt with GEOS_Wrapper

* Remove unused code

* Fix theroritical timings
Lint

* Fixed a bug where pkz was being calculated twice, and the second calc was wrong

* Downgrade DaCe to 0.14.0 pending array aliasing fix

* Set default cache path for orchestrated DaCe to respect GT_CACHE_* env

* Remove previous per stencil override of default_build_folder

* Revert "Set default cache path for orchestrated DaCe to respect GT_CACHE_* env"

This reverts commit 4fc5b4d.

* Revert "Remove previous per stencil override of default_build_folder"

This reverts commit 2245027.

* Read cache_root in default dace backend

* Document faulty behavior with GT_CACHE_DIR_NAME

* Fix bad requirements syntax

* Check for the string value of CONST_VERSION directly instead of enum

* Protect constant selection more rigorusly.
Clean abort on unknown constant given

* Log constants selection

* Refactor NQ to constants.py

* Replace all logger with pace_log
Introduce PACE_LOGLEVEL to control log level from outside

* Code guidelines clean up

* Devops/GitHub actions on (#15)

* Linting on PR

* Run main unit test

* Update python to available 3.8.12

* Remove cd to pace

* Lint: git submodule recursive

* Typo

* Add openmpi to the image

* Linting

* Fix unit tests (remove dxa, dya rely on halo ex)

* typo

* Change name of jobs

* Distributed compilation on orchestrated backend for NxN layouts (#14)

* Adapt orchestration distribute compile for NxN layout

* Remove debug code

* Add a more descriptive string base postfix for cache naming
Identify the code path for all cases
Consistent reload post-compile
Create a central space for all caches generation logic
No more original layout check required

* Add a test on caches relocatability

* Verbose todo

* Linting on PR

* Run main unit test

* Update python to available 3.8.12

* Remove cd to pace

* Lint: git submodule recursive

* Typo

* Add openmpi to the image

* Linting

* Fix unit tests (remove dxa, dya rely on halo ex)

* typo

* Change name of jobs

* Missing enum

* Lint imports

* Fix unit tests

* Deactivate relocability test due to Python crash
Logged as issyue 16

* Typo

* Raise for 1,X and X,1 layouts which requires a new descriptor

* Added ak, bk for 137 levels in eta.py

* Add floating point precision to GEOS bridge init

* lint

* Add device PCI bus id (for MPS debug)

* Typo + lint

* Try to detect MPS reading the "log" pipe

* Lint

* Clean up

* Log info GEOS bridge (#18)

* Add floating point precision to GEOS bridge init

* lint

* Add device PCI bus id (for MPS debug)

* Typo + lint

* Try to detect MPS reading the "log" pipe

* Lint

* Clean up

* Update geos/develop to grab NOAA PR9 results (#21)

* Verbose choice of block/grid size

* added build script for c5

* updated repo to NOAA

* GEOS integration (#9)

* Initialize GeosDycoreWrapper with bdt (timestep)

* Use GEOS version of constants

* 1. Add qcld to the list of tracers beings advected
2. Made GEOS specific changes to thresholds in saturation adjustment

* Accumulate diss_est

* Allow GEOS_WRAPPER to process device data

* Add clear to collector for 3rd party use. GEOS pass down timings to caller

* Make kernel analysis run a copy stencil to compute local bandwith
Parametrize tool with backend, output format

* Move constant on a env var
Add saturation adjustement threshold to const

* Remove unused if leading to empty code block

* Restrict dace to 0.14.1 due to a parsing bug

* Add guard for bdt==0
Fix bad merge for bdt with GEOS_Wrapper

* Remove unused code

* Fix theroritical timings

* Fixed a bug where pkz was being calculated twice, and the second calc was wrong

* Downgrade DaCe to 0.14.0 pending array aliasing fix

* Set default cache path for orchestrated DaCe to respect GT_CACHE_* env

* Remove previous per stencil override of default_build_folder

* Revert "Set default cache path for orchestrated DaCe to respect GT_CACHE_* env"

* Revert "Remove previous per stencil override of default_build_folder"

* Read cache_root in default dace backend

* Document faulty behavior with GT_CACHE_DIR_NAME

* Fix bad requirements syntax

* Check for the string value of CONST_VERSION directly instead of enum

* Protect constant selection more rigorusly.
Clean abort on unknown constant given

* Log constants selection

* Refactor NQ to constants.py

* Fix or explain inlined import

* Verbose runtime error when bad dt_atmos

* Verbose warm up

* re-initialize heat_source and diss_est each call, add do_skeb check to accumulation

---------

Co-authored-by: Purnendu Chakraborty <[email protected]>
Co-authored-by: Oliver Elbert <[email protected]>

---------

Co-authored-by: Rusty Benson <[email protected]>
Co-authored-by: Oliver Elbert <[email protected]>
Co-authored-by: Purnendu Chakraborty <[email protected]>
Co-authored-by: Oliver Elbert <[email protected]>

* [NOAA:Update] Bring back #15 & doubly periodic domain (#25)

* Feature/dp driver (#13)

* initial commit

* adding test config

* adding the rest of driver and util code

* updating history.md

* move u_max to dycore config

* uncomment assert

* added comment explaining the copy of grid type to dycore config

* Turn main unit test  & lint on PR, logger clean up [NASA:Update]  (#15)

* Initialize GeosDycoreWrapper with bdt (timestep)

* Use GEOS version of constants

* 1. Add qcld to the list of tracers beings advected
2. Made GEOS specific changes to thresholds in saturation adjustment

* Accumulate diss_est

* Allow GEOS_WRAPPER to process device data

* Add clear to collector for 3rd party use. GEOS pass down timings to caller

* Make kernel analysis run a copy stencil to compute local bandwith
Parametrize tool with backend, output format

* Move constant on a env var
Add saturation adjustement threshold to const

* Restrict dace to 0.14.1 due to a parsing bug

* Add guard for bdt==0

* Fix theroritical timings

* Fixed a bug where pkz was being calculated twice, and the second calc was wrong

* Downgrade DaCe to 0.14.0 pending array aliasing fix

* Set default cache path for orchestrated DaCe to respect GT_CACHE_* env

* Remove previous per stencil override of default_build_folder

* Revert "Set default cache path for orchestrated DaCe to respect GT_CACHE_* env"

* Read cache_root in default dace backend

* Document faulty behavior with GT_CACHE_DIR_NAME

* Check for the string value of CONST_VERSION directly instead of enum

* Protect constant selection more rigorusly.
Clean abort on unknown constant given

* Log constants selection

* Refactor NQ to constants.py

* Introduce PACE_LOGLEVEL to control log level from outside

* Code guidelines clean up

* Devops/GitHub actions on (#15)

* Linting on PR

* Run main unit test

* Update python to available 3.8.12

* Fix unit tests (remove dxa, dya rely on halo ex)

* Update HISTORY.md

* Adapt log_level in driver.run

* Verbose the PACE_CONSTANTS

* Doc log level hierarchical nature

---------

Co-authored-by: Purnendu Chakraborty <[email protected]>
Co-authored-by: Purnendu Chakraborty <[email protected]>

* Lint

---------

Co-authored-by: Oliver Elbert <[email protected]>
Co-authored-by: Purnendu Chakraborty <[email protected]>
Co-authored-by: Purnendu Chakraborty <[email protected]>

* Update gt4py, dace, cleanup (#19)

* Update gt4py to top of master on June 21

* Update DaCe to 0.14.2
Workaround aliasing issue in FiniteVolumeTransport

* Fix to gt4py storage

* Downgrade to dace 0.14.1

* DaCe to 0.14.4
Orchestrating NonHydrostaticPressureGradient
Adptating code to newer gt4py

* Regenerate constraints.txt

* Default constants to GFS
Fix snapshot for GPU runs
Lint on ETA
Fix log level

* Remove `daint_venv` submodule

* Adding dace as a submodule
Removing buildenv as a submodule

* Update gt4py to latest master

* Skip ConstantPropagation during `Simplify`

* Remove buidlenv

* Update requirements_dev.txt

* Add editable util to requirements_dev.txt

* lint

* scipy for tests is now needed

* Pin `DaCe` to pace-fixes-0 merge

* Remove logging setup in test_translate

* Make cupy import robust to device not being available

* Fix to GEOS bridge MPS detection

* Up gt4py to August 14th EOD:
  - Hip/ROCm
  - New allocators

* DaCE module: swap SSH for HTTPS (#26)

* GEOS GridTools stencils build override (#27)

* Stencil build override for GEOS

* Deactivate warnings if PACE_LOGLEVEL is > WARNING

* Better log level

* Bad merge (again)

---------

Co-authored-by: Purnendu Chakraborty <[email protected]>
Co-authored-by: Purnendu Chakraborty <[email protected]>
Co-authored-by: Rusty Benson <[email protected]>
Co-authored-by: Oliver Elbert <[email protected]>
Co-authored-by: Oliver Elbert <[email protected]>
@FlorianDeconinck FlorianDeconinck merged commit 425371a into geos/main Nov 7, 2023
2 checks passed
FlorianDeconinck pushed a commit that referenced this pull request Nov 7, 2023
…(continuous issue) (#30)

* Testing changes reflected across branches

* Undoing changes made in build_gaea_c5.sh

* Testing vscode functionality, by adding a change to external_grid branch

* Testing vscode functionality, by adding a change to external_grid branch

* Edited init_utils.py and initialize_tc.py regarding the overlap in initialize_delp and initialize_edge_pressure functions

---------

Co-authored-by: Frank Malatino <[email protected]>
FlorianDeconinck added a commit that referenced this pull request Dec 1, 2023
* Initialize GeosDycoreWrapper with bdt (timestep)

* Use GEOS version of constants

* 1. Add qcld to the list of tracers beings advected
2. Made GEOS specific changes to thresholds in saturation adjustment

* Accumulate diss_est

* Allow GEOS_WRAPPER to process device data

* Add clear to collector for 3rd party use. GEOS pass down timings to caller

* Make kernel analysis run a copy stencil to compute local bandwith
Parametrize tool with backend, output format

* Move constant on a env var
Add saturation adjustement threshold to const

* lint

* More linting

* Remove unused if leading to empty code block

* Restrict dace to 0.14.1 due to a parsing bug

* Add guard for bdt==0
Fix bad merge for bdt with GEOS_Wrapper

* Remove unused code

* Fix theroritical timings
Lint

* Fixed a bug where pkz was being calculated twice, and the second calc was wrong

* Downgrade DaCe to 0.14.0 pending array aliasing fix

* Set default cache path for orchestrated DaCe to respect GT_CACHE_* env

* Remove previous per stencil override of default_build_folder

* Revert "Set default cache path for orchestrated DaCe to respect GT_CACHE_* env"

This reverts commit 4fc5b4d.

* Revert "Remove previous per stencil override of default_build_folder"

This reverts commit 2245027.

* Read cache_root in default dace backend

* Document faulty behavior with GT_CACHE_DIR_NAME

* Fix bad requirements syntax

* Check for the string value of CONST_VERSION directly instead of enum

* Protect constant selection more rigorusly.
Clean abort on unknown constant given

* Log constants selection

* Refactor NQ to constants.py

* Replace all logger with pace_log
Introduce PACE_LOGLEVEL to control log level from outside

* Code guidelines clean up

* Devops/GitHub actions on (#15)

* Linting on PR

* Run main unit test

* Update python to available 3.8.12

* Remove cd to pace

* Lint: git submodule recursive

* Typo

* Add openmpi to the image

* Linting

* Fix unit tests (remove dxa, dya rely on halo ex)

* typo

* Change name of jobs

* Distributed compilation on orchestrated backend for NxN layouts (#14)

* Adapt orchestration distribute compile for NxN layout

* Remove debug code

* Add a more descriptive string base postfix for cache naming
Identify the code path for all cases
Consistent reload post-compile
Create a central space for all caches generation logic
No more original layout check required

* Add a test on caches relocatability

* Verbose todo

* Linting on PR

* Run main unit test

* Update python to available 3.8.12

* Remove cd to pace

* Lint: git submodule recursive

* Typo

* Add openmpi to the image

* Linting

* Fix unit tests (remove dxa, dya rely on halo ex)

* typo

* Change name of jobs

* Missing enum

* Lint imports

* Fix unit tests

* Deactivate relocability test due to Python crash
Logged as issyue 16

* Typo

* Raise for 1,X and X,1 layouts which requires a new descriptor

* Added ak, bk for 137 levels in eta.py

* Add floating point precision to GEOS bridge init

* lint

* Add device PCI bus id (for MPS debug)

* Typo + lint

* Try to detect MPS reading the "log" pipe

* Lint

* Clean up

* Log info GEOS bridge (#18)

* Add floating point precision to GEOS bridge init

* lint

* Add device PCI bus id (for MPS debug)

* Typo + lint

* Try to detect MPS reading the "log" pipe

* Lint

* Clean up

* Update geos/develop to grab NOAA PR9 results (#21)

* Verbose choice of block/grid size

* added build script for c5

* updated repo to NOAA

* GEOS integration (#9)

* Initialize GeosDycoreWrapper with bdt (timestep)

* Use GEOS version of constants

* 1. Add qcld to the list of tracers beings advected
2. Made GEOS specific changes to thresholds in saturation adjustment

* Accumulate diss_est

* Allow GEOS_WRAPPER to process device data

* Add clear to collector for 3rd party use. GEOS pass down timings to caller

* Make kernel analysis run a copy stencil to compute local bandwith
Parametrize tool with backend, output format

* Move constant on a env var
Add saturation adjustement threshold to const

* Remove unused if leading to empty code block

* Restrict dace to 0.14.1 due to a parsing bug

* Add guard for bdt==0
Fix bad merge for bdt with GEOS_Wrapper

* Remove unused code

* Fix theroritical timings

* Fixed a bug where pkz was being calculated twice, and the second calc was wrong

* Downgrade DaCe to 0.14.0 pending array aliasing fix

* Set default cache path for orchestrated DaCe to respect GT_CACHE_* env

* Remove previous per stencil override of default_build_folder

* Revert "Set default cache path for orchestrated DaCe to respect GT_CACHE_* env"

* Revert "Remove previous per stencil override of default_build_folder"

* Read cache_root in default dace backend

* Document faulty behavior with GT_CACHE_DIR_NAME

* Fix bad requirements syntax

* Check for the string value of CONST_VERSION directly instead of enum

* Protect constant selection more rigorusly.
Clean abort on unknown constant given

* Log constants selection

* Refactor NQ to constants.py

* Fix or explain inlined import

* Verbose runtime error when bad dt_atmos

* Verbose warm up

* re-initialize heat_source and diss_est each call, add do_skeb check to accumulation

---------

Co-authored-by: Purnendu Chakraborty <[email protected]>
Co-authored-by: Oliver Elbert <[email protected]>

---------

Co-authored-by: Rusty Benson <[email protected]>
Co-authored-by: Oliver Elbert <[email protected]>
Co-authored-by: Purnendu Chakraborty <[email protected]>
Co-authored-by: Oliver Elbert <[email protected]>

* [NOAA:Update] Bring back #15 & doubly periodic domain (#25)

* Feature/dp driver (#13)

* initial commit

* adding test config

* adding the rest of driver and util code

* updating history.md

* move u_max to dycore config

* uncomment assert

* added comment explaining the copy of grid type to dycore config

* Turn main unit test  & lint on PR, logger clean up [NASA:Update]  (#15)

* Initialize GeosDycoreWrapper with bdt (timestep)

* Use GEOS version of constants

* 1. Add qcld to the list of tracers beings advected
2. Made GEOS specific changes to thresholds in saturation adjustment

* Accumulate diss_est

* Allow GEOS_WRAPPER to process device data

* Add clear to collector for 3rd party use. GEOS pass down timings to caller

* Make kernel analysis run a copy stencil to compute local bandwith
Parametrize tool with backend, output format

* Move constant on a env var
Add saturation adjustement threshold to const

* Restrict dace to 0.14.1 due to a parsing bug

* Add guard for bdt==0

* Fix theroritical timings

* Fixed a bug where pkz was being calculated twice, and the second calc was wrong

* Downgrade DaCe to 0.14.0 pending array aliasing fix

* Set default cache path for orchestrated DaCe to respect GT_CACHE_* env

* Remove previous per stencil override of default_build_folder

* Revert "Set default cache path for orchestrated DaCe to respect GT_CACHE_* env"

* Read cache_root in default dace backend

* Document faulty behavior with GT_CACHE_DIR_NAME

* Check for the string value of CONST_VERSION directly instead of enum

* Protect constant selection more rigorusly.
Clean abort on unknown constant given

* Log constants selection

* Refactor NQ to constants.py

* Introduce PACE_LOGLEVEL to control log level from outside

* Code guidelines clean up

* Devops/GitHub actions on (#15)

* Linting on PR

* Run main unit test

* Update python to available 3.8.12

* Fix unit tests (remove dxa, dya rely on halo ex)

* Update HISTORY.md

* Adapt log_level in driver.run

* Verbose the PACE_CONSTANTS

* Doc log level hierarchical nature

---------

Co-authored-by: Purnendu Chakraborty <[email protected]>
Co-authored-by: Purnendu Chakraborty <[email protected]>

* Lint

---------

Co-authored-by: Oliver Elbert <[email protected]>
Co-authored-by: Purnendu Chakraborty <[email protected]>
Co-authored-by: Purnendu Chakraborty <[email protected]>

* Update gt4py, dace, cleanup (#19)

* Update gt4py to top of master on June 21

* Update DaCe to 0.14.2
Workaround aliasing issue in FiniteVolumeTransport

* Fix to gt4py storage

* Downgrade to dace 0.14.1

* DaCe to 0.14.4
Orchestrating NonHydrostaticPressureGradient
Adptating code to newer gt4py

* Regenerate constraints.txt

* Default constants to GFS
Fix snapshot for GPU runs
Lint on ETA
Fix log level

* Remove `daint_venv` submodule

* Adding dace as a submodule
Removing buildenv as a submodule

* Update gt4py to latest master

* Skip ConstantPropagation during `Simplify`

* Remove buidlenv

* Update requirements_dev.txt

* Add editable util to requirements_dev.txt

* lint

* scipy for tests is now needed

* Pin `DaCe` to pace-fixes-0 merge

* Remove logging setup in test_translate

* Make cupy import robust to device not being available

* Fix to GEOS bridge MPS detection

* Up gt4py to August 14th EOD:
  - Hip/ROCm
  - New allocators

* DaCE module: swap SSH for HTTPS (#26)

* GEOS GridTools stencils build override (#27)

* Stencil build override for GEOS

* Deactivate warnings if PACE_LOGLEVEL is > WARNING

* Better log level

* Bad merge (again)

* NASA fork sync. (#37) (#30)

* Initialize GeosDycoreWrapper with bdt (timestep)

* Use GEOS version of constants

* 1. Add qcld to the list of tracers beings advected
2. Made GEOS specific changes to thresholds in saturation adjustment

* Accumulate diss_est

* Allow GEOS_WRAPPER to process device data

* Add clear to collector for 3rd party use. GEOS pass down timings to caller

* Make kernel analysis run a copy stencil to compute local bandwith
Parametrize tool with backend, output format

* Move constant on a env var
Add saturation adjustement threshold to const

* lint

* More linting

* Remove unused if leading to empty code block

* Restrict dace to 0.14.1 due to a parsing bug

* Add guard for bdt==0
Fix bad merge for bdt with GEOS_Wrapper

* Remove unused code

* Fix theroritical timings
Lint

* Fixed a bug where pkz was being calculated twice, and the second calc was wrong

* Downgrade DaCe to 0.14.0 pending array aliasing fix

* Set default cache path for orchestrated DaCe to respect GT_CACHE_* env

* Remove previous per stencil override of default_build_folder

* Revert "Set default cache path for orchestrated DaCe to respect GT_CACHE_* env"

This reverts commit 4fc5b4d.

* Revert "Remove previous per stencil override of default_build_folder"

This reverts commit 2245027.

* Read cache_root in default dace backend

* Document faulty behavior with GT_CACHE_DIR_NAME

* Fix bad requirements syntax

* Check for the string value of CONST_VERSION directly instead of enum

* Protect constant selection more rigorusly.
Clean abort on unknown constant given

* Log constants selection

* Refactor NQ to constants.py

* Replace all logger with pace_log
Introduce PACE_LOGLEVEL to control log level from outside

* Code guidelines clean up

* Devops/GitHub actions on (#15)

* Linting on PR

* Run main unit test

* Update python to available 3.8.12

* Remove cd to pace

* Lint: git submodule recursive

* Typo

* Add openmpi to the image

* Linting

* Fix unit tests (remove dxa, dya rely on halo ex)

* typo

* Change name of jobs

* Distributed compilation on orchestrated backend for NxN layouts (#14)

* Adapt orchestration distribute compile for NxN layout

* Remove debug code

* Add a more descriptive string base postfix for cache naming
Identify the code path for all cases
Consistent reload post-compile
Create a central space for all caches generation logic
No more original layout check required

* Add a test on caches relocatability

* Verbose todo

* Linting on PR

* Run main unit test

* Update python to available 3.8.12

* Remove cd to pace

* Lint: git submodule recursive

* Typo

* Add openmpi to the image

* Linting

* Fix unit tests (remove dxa, dya rely on halo ex)

* typo

* Change name of jobs

* Missing enum

* Lint imports

* Fix unit tests

* Deactivate relocability test due to Python crash
Logged as issyue 16

* Typo

* Raise for 1,X and X,1 layouts which requires a new descriptor

* Added ak, bk for 137 levels in eta.py

* Add floating point precision to GEOS bridge init

* lint

* Add device PCI bus id (for MPS debug)

* Typo + lint

* Try to detect MPS reading the "log" pipe

* Lint

* Clean up

* Log info GEOS bridge (#18)

* Add floating point precision to GEOS bridge init

* lint

* Add device PCI bus id (for MPS debug)

* Typo + lint

* Try to detect MPS reading the "log" pipe

* Lint

* Clean up

* Update geos/develop to grab NOAA PR9 results (#21)

* Verbose choice of block/grid size

* added build script for c5

* updated repo to NOAA

* GEOS integration (#9)

* Initialize GeosDycoreWrapper with bdt (timestep)

* Use GEOS version of constants

* 1. Add qcld to the list of tracers beings advected
2. Made GEOS specific changes to thresholds in saturation adjustment

* Accumulate diss_est

* Allow GEOS_WRAPPER to process device data

* Add clear to collector for 3rd party use. GEOS pass down timings to caller

* Make kernel analysis run a copy stencil to compute local bandwith
Parametrize tool with backend, output format

* Move constant on a env var
Add saturation adjustement threshold to const

* Remove unused if leading to empty code block

* Restrict dace to 0.14.1 due to a parsing bug

* Add guard for bdt==0
Fix bad merge for bdt with GEOS_Wrapper

* Remove unused code

* Fix theroritical timings

* Fixed a bug where pkz was being calculated twice, and the second calc was wrong

* Downgrade DaCe to 0.14.0 pending array aliasing fix

* Set default cache path for orchestrated DaCe to respect GT_CACHE_* env

* Remove previous per stencil override of default_build_folder

* Revert "Set default cache path for orchestrated DaCe to respect GT_CACHE_* env"

* Revert "Remove previous per stencil override of default_build_folder"

* Read cache_root in default dace backend

* Document faulty behavior with GT_CACHE_DIR_NAME

* Fix bad requirements syntax

* Check for the string value of CONST_VERSION directly instead of enum

* Protect constant selection more rigorusly.
Clean abort on unknown constant given

* Log constants selection

* Refactor NQ to constants.py

* Fix or explain inlined import

* Verbose runtime error when bad dt_atmos

* Verbose warm up

* re-initialize heat_source and diss_est each call, add do_skeb check to accumulation

---------




---------






* [NOAA:Update] Bring back #15 & doubly periodic domain (#25)

* Feature/dp driver (#13)

* initial commit

* adding test config

* adding the rest of driver and util code

* updating history.md

* move u_max to dycore config

* uncomment assert

* added comment explaining the copy of grid type to dycore config

* Turn main unit test  & lint on PR, logger clean up [NASA:Update]  (#15)

* Initialize GeosDycoreWrapper with bdt (timestep)

* Use GEOS version of constants

* 1. Add qcld to the list of tracers beings advected
2. Made GEOS specific changes to thresholds in saturation adjustment

* Accumulate diss_est

* Allow GEOS_WRAPPER to process device data

* Add clear to collector for 3rd party use. GEOS pass down timings to caller

* Make kernel analysis run a copy stencil to compute local bandwith
Parametrize tool with backend, output format

* Move constant on a env var
Add saturation adjustement threshold to const

* Restrict dace to 0.14.1 due to a parsing bug

* Add guard for bdt==0

* Fix theroritical timings

* Fixed a bug where pkz was being calculated twice, and the second calc was wrong

* Downgrade DaCe to 0.14.0 pending array aliasing fix

* Set default cache path for orchestrated DaCe to respect GT_CACHE_* env

* Remove previous per stencil override of default_build_folder

* Revert "Set default cache path for orchestrated DaCe to respect GT_CACHE_* env"

* Read cache_root in default dace backend

* Document faulty behavior with GT_CACHE_DIR_NAME

* Check for the string value of CONST_VERSION directly instead of enum

* Protect constant selection more rigorusly.
Clean abort on unknown constant given

* Log constants selection

* Refactor NQ to constants.py

* Introduce PACE_LOGLEVEL to control log level from outside

* Code guidelines clean up

* Devops/GitHub actions on (#15)

* Linting on PR

* Run main unit test

* Update python to available 3.8.12

* Fix unit tests (remove dxa, dya rely on halo ex)

* Update HISTORY.md

* Adapt log_level in driver.run

* Verbose the PACE_CONSTANTS

* Doc log level hierarchical nature

---------




* Lint

---------





* Update gt4py, dace, cleanup (#19)

* Update gt4py to top of master on June 21

* Update DaCe to 0.14.2
Workaround aliasing issue in FiniteVolumeTransport

* Fix to gt4py storage

* Downgrade to dace 0.14.1

* DaCe to 0.14.4
Orchestrating NonHydrostaticPressureGradient
Adptating code to newer gt4py

* Regenerate constraints.txt

* Default constants to GFS
Fix snapshot for GPU runs
Lint on ETA
Fix log level

* Remove `daint_venv` submodule

* Adding dace as a submodule
Removing buildenv as a submodule

* Update gt4py to latest master

* Skip ConstantPropagation during `Simplify`

* Remove buidlenv

* Update requirements_dev.txt

* Add editable util to requirements_dev.txt

* lint

* scipy for tests is now needed

* Pin `DaCe` to pace-fixes-0 merge

* Remove logging setup in test_translate

* Make cupy import robust to device not being available

* Fix to GEOS bridge MPS detection

* Up gt4py to August 14th EOD:
  - Hip/ROCm
  - New allocators

* DaCE module: swap SSH for HTTPS (#26)

* GEOS GridTools stencils build override (#27)

* Stencil build override for GEOS

* Deactivate warnings if PACE_LOGLEVEL is > WARNING

* Better log level

* Bad merge (again)

---------

Co-authored-by: Purnendu Chakraborty <[email protected]>
Co-authored-by: Purnendu Chakraborty <[email protected]>
Co-authored-by: Rusty Benson <[email protected]>
Co-authored-by: Oliver Elbert <[email protected]>
Co-authored-by: Oliver Elbert <[email protected]>

---------

Co-authored-by: Purnendu Chakraborty <[email protected]>
Co-authored-by: Purnendu Chakraborty <[email protected]>
Co-authored-by: Rusty Benson <[email protected]>
Co-authored-by: Oliver Elbert <[email protected]>
Co-authored-by: Oliver Elbert <[email protected]>
FlorianDeconinck added a commit that referenced this pull request Dec 1, 2023
* Initialize GeosDycoreWrapper with bdt (timestep)

* Use GEOS version of constants

* 1. Add qcld to the list of tracers beings advected
2. Made GEOS specific changes to thresholds in saturation adjustment

* Accumulate diss_est

* Allow GEOS_WRAPPER to process device data

* Add clear to collector for 3rd party use. GEOS pass down timings to caller

* Make kernel analysis run a copy stencil to compute local bandwith
Parametrize tool with backend, output format

* Move constant on a env var
Add saturation adjustement threshold to const

* lint

* More linting

* Remove unused if leading to empty code block

* Restrict dace to 0.14.1 due to a parsing bug

* Add guard for bdt==0
Fix bad merge for bdt with GEOS_Wrapper

* Remove unused code

* Fix theroritical timings
Lint

* Fixed a bug where pkz was being calculated twice, and the second calc was wrong

* Downgrade DaCe to 0.14.0 pending array aliasing fix

* Set default cache path for orchestrated DaCe to respect GT_CACHE_* env

* Remove previous per stencil override of default_build_folder

* Revert "Set default cache path for orchestrated DaCe to respect GT_CACHE_* env"

This reverts commit 4fc5b4d.

* Revert "Remove previous per stencil override of default_build_folder"

This reverts commit 2245027.

* Read cache_root in default dace backend

* Document faulty behavior with GT_CACHE_DIR_NAME

* Fix bad requirements syntax

* Check for the string value of CONST_VERSION directly instead of enum

* Protect constant selection more rigorusly.
Clean abort on unknown constant given

* Log constants selection

* Refactor NQ to constants.py

* Replace all logger with pace_log
Introduce PACE_LOGLEVEL to control log level from outside

* Code guidelines clean up

* Devops/GitHub actions on (#15)

* Linting on PR

* Run main unit test

* Update python to available 3.8.12

* Remove cd to pace

* Lint: git submodule recursive

* Typo

* Add openmpi to the image

* Linting

* Fix unit tests (remove dxa, dya rely on halo ex)

* typo

* Change name of jobs

* Distributed compilation on orchestrated backend for NxN layouts (#14)

* Adapt orchestration distribute compile for NxN layout

* Remove debug code

* Add a more descriptive string base postfix for cache naming
Identify the code path for all cases
Consistent reload post-compile
Create a central space for all caches generation logic
No more original layout check required

* Add a test on caches relocatability

* Verbose todo

* Linting on PR

* Run main unit test

* Update python to available 3.8.12

* Remove cd to pace

* Lint: git submodule recursive

* Typo

* Add openmpi to the image

* Linting

* Fix unit tests (remove dxa, dya rely on halo ex)

* typo

* Change name of jobs

* Missing enum

* Lint imports

* Fix unit tests

* Deactivate relocability test due to Python crash
Logged as issyue 16

* Typo

* Raise for 1,X and X,1 layouts which requires a new descriptor

* Added ak, bk for 137 levels in eta.py

* Add floating point precision to GEOS bridge init

* lint

* Add device PCI bus id (for MPS debug)

* Typo + lint

* Try to detect MPS reading the "log" pipe

* Lint

* Clean up

* Log info GEOS bridge (#18)

* Add floating point precision to GEOS bridge init

* lint

* Add device PCI bus id (for MPS debug)

* Typo + lint

* Try to detect MPS reading the "log" pipe

* Lint

* Clean up

* Update geos/develop to grab NOAA PR9 results (#21)

* Verbose choice of block/grid size

* added build script for c5

* updated repo to NOAA

* GEOS integration (#9)

* Initialize GeosDycoreWrapper with bdt (timestep)

* Use GEOS version of constants

* 1. Add qcld to the list of tracers beings advected
2. Made GEOS specific changes to thresholds in saturation adjustment

* Accumulate diss_est

* Allow GEOS_WRAPPER to process device data

* Add clear to collector for 3rd party use. GEOS pass down timings to caller

* Make kernel analysis run a copy stencil to compute local bandwith
Parametrize tool with backend, output format

* Move constant on a env var
Add saturation adjustement threshold to const

* Remove unused if leading to empty code block

* Restrict dace to 0.14.1 due to a parsing bug

* Add guard for bdt==0
Fix bad merge for bdt with GEOS_Wrapper

* Remove unused code

* Fix theroritical timings

* Fixed a bug where pkz was being calculated twice, and the second calc was wrong

* Downgrade DaCe to 0.14.0 pending array aliasing fix

* Set default cache path for orchestrated DaCe to respect GT_CACHE_* env

* Remove previous per stencil override of default_build_folder

* Revert "Set default cache path for orchestrated DaCe to respect GT_CACHE_* env"

* Revert "Remove previous per stencil override of default_build_folder"

* Read cache_root in default dace backend

* Document faulty behavior with GT_CACHE_DIR_NAME

* Fix bad requirements syntax

* Check for the string value of CONST_VERSION directly instead of enum

* Protect constant selection more rigorusly.
Clean abort on unknown constant given

* Log constants selection

* Refactor NQ to constants.py

* Fix or explain inlined import

* Verbose runtime error when bad dt_atmos

* Verbose warm up

* re-initialize heat_source and diss_est each call, add do_skeb check to accumulation

---------




---------






* [NOAA:Update] Bring back #15 & doubly periodic domain (#25)

* Feature/dp driver (#13)

* initial commit

* adding test config

* adding the rest of driver and util code

* updating history.md

* move u_max to dycore config

* uncomment assert

* added comment explaining the copy of grid type to dycore config

* Turn main unit test  & lint on PR, logger clean up [NASA:Update]  (#15)

* Initialize GeosDycoreWrapper with bdt (timestep)

* Use GEOS version of constants

* 1. Add qcld to the list of tracers beings advected
2. Made GEOS specific changes to thresholds in saturation adjustment

* Accumulate diss_est

* Allow GEOS_WRAPPER to process device data

* Add clear to collector for 3rd party use. GEOS pass down timings to caller

* Make kernel analysis run a copy stencil to compute local bandwith
Parametrize tool with backend, output format

* Move constant on a env var
Add saturation adjustement threshold to const

* Restrict dace to 0.14.1 due to a parsing bug

* Add guard for bdt==0

* Fix theroritical timings

* Fixed a bug where pkz was being calculated twice, and the second calc was wrong

* Downgrade DaCe to 0.14.0 pending array aliasing fix

* Set default cache path for orchestrated DaCe to respect GT_CACHE_* env

* Remove previous per stencil override of default_build_folder

* Revert "Set default cache path for orchestrated DaCe to respect GT_CACHE_* env"

* Read cache_root in default dace backend

* Document faulty behavior with GT_CACHE_DIR_NAME

* Check for the string value of CONST_VERSION directly instead of enum

* Protect constant selection more rigorusly.
Clean abort on unknown constant given

* Log constants selection

* Refactor NQ to constants.py

* Introduce PACE_LOGLEVEL to control log level from outside

* Code guidelines clean up

* Devops/GitHub actions on (#15)

* Linting on PR

* Run main unit test

* Update python to available 3.8.12

* Fix unit tests (remove dxa, dya rely on halo ex)

* Update HISTORY.md

* Adapt log_level in driver.run

* Verbose the PACE_CONSTANTS

* Doc log level hierarchical nature

---------




* Lint

---------





* Update gt4py, dace, cleanup (#19)

* Update gt4py to top of master on June 21

* Update DaCe to 0.14.2
Workaround aliasing issue in FiniteVolumeTransport

* Fix to gt4py storage

* Downgrade to dace 0.14.1

* DaCe to 0.14.4
Orchestrating NonHydrostaticPressureGradient
Adptating code to newer gt4py

* Regenerate constraints.txt

* Default constants to GFS
Fix snapshot for GPU runs
Lint on ETA
Fix log level

* Remove `daint_venv` submodule

* Adding dace as a submodule
Removing buildenv as a submodule

* Update gt4py to latest master

* Skip ConstantPropagation during `Simplify`

* Remove buidlenv

* Update requirements_dev.txt

* Add editable util to requirements_dev.txt

* lint

* scipy for tests is now needed

* Pin `DaCe` to pace-fixes-0 merge

* Remove logging setup in test_translate

* Make cupy import robust to device not being available

* Fix to GEOS bridge MPS detection

* Up gt4py to August 14th EOD:
  - Hip/ROCm
  - New allocators

* DaCE module: swap SSH for HTTPS (#26)

* GEOS GridTools stencils build override (#27)

* Stencil build override for GEOS

* Deactivate warnings if PACE_LOGLEVEL is > WARNING

* Better log level

* Bad merge (again)

* NASA fork sync. (#37) (#30)

* Initialize GeosDycoreWrapper with bdt (timestep)

* Use GEOS version of constants

* 1. Add qcld to the list of tracers beings advected
2. Made GEOS specific changes to thresholds in saturation adjustment

* Accumulate diss_est

* Allow GEOS_WRAPPER to process device data

* Add clear to collector for 3rd party use. GEOS pass down timings to caller

* Make kernel analysis run a copy stencil to compute local bandwith
Parametrize tool with backend, output format

* Move constant on a env var
Add saturation adjustement threshold to const

* lint

* More linting

* Remove unused if leading to empty code block

* Restrict dace to 0.14.1 due to a parsing bug

* Add guard for bdt==0
Fix bad merge for bdt with GEOS_Wrapper

* Remove unused code

* Fix theroritical timings
Lint

* Fixed a bug where pkz was being calculated twice, and the second calc was wrong

* Downgrade DaCe to 0.14.0 pending array aliasing fix

* Set default cache path for orchestrated DaCe to respect GT_CACHE_* env

* Remove previous per stencil override of default_build_folder

* Revert "Set default cache path for orchestrated DaCe to respect GT_CACHE_* env"

This reverts commit 4fc5b4d.

* Revert "Remove previous per stencil override of default_build_folder"

This reverts commit 2245027.

* Read cache_root in default dace backend

* Document faulty behavior with GT_CACHE_DIR_NAME

* Fix bad requirements syntax

* Check for the string value of CONST_VERSION directly instead of enum

* Protect constant selection more rigorusly.
Clean abort on unknown constant given

* Log constants selection

* Refactor NQ to constants.py

* Replace all logger with pace_log
Introduce PACE_LOGLEVEL to control log level from outside

* Code guidelines clean up

* Devops/GitHub actions on (#15)

* Linting on PR

* Run main unit test

* Update python to available 3.8.12

* Remove cd to pace

* Lint: git submodule recursive

* Typo

* Add openmpi to the image

* Linting

* Fix unit tests (remove dxa, dya rely on halo ex)

* typo

* Change name of jobs

* Distributed compilation on orchestrated backend for NxN layouts (#14)

* Adapt orchestration distribute compile for NxN layout

* Remove debug code

* Add a more descriptive string base postfix for cache naming
Identify the code path for all cases
Consistent reload post-compile
Create a central space for all caches generation logic
No more original layout check required

* Add a test on caches relocatability

* Verbose todo

* Linting on PR

* Run main unit test

* Update python to available 3.8.12

* Remove cd to pace

* Lint: git submodule recursive

* Typo

* Add openmpi to the image

* Linting

* Fix unit tests (remove dxa, dya rely on halo ex)

* typo

* Change name of jobs

* Missing enum

* Lint imports

* Fix unit tests

* Deactivate relocability test due to Python crash
Logged as issyue 16

* Typo

* Raise for 1,X and X,1 layouts which requires a new descriptor

* Added ak, bk for 137 levels in eta.py

* Add floating point precision to GEOS bridge init

* lint

* Add device PCI bus id (for MPS debug)

* Typo + lint

* Try to detect MPS reading the "log" pipe

* Lint

* Clean up

* Log info GEOS bridge (#18)

* Add floating point precision to GEOS bridge init

* lint

* Add device PCI bus id (for MPS debug)

* Typo + lint

* Try to detect MPS reading the "log" pipe

* Lint

* Clean up

* Update geos/develop to grab NOAA PR9 results (#21)

* Verbose choice of block/grid size

* added build script for c5

* updated repo to NOAA

* GEOS integration (#9)

* Initialize GeosDycoreWrapper with bdt (timestep)

* Use GEOS version of constants

* 1. Add qcld to the list of tracers beings advected
2. Made GEOS specific changes to thresholds in saturation adjustment

* Accumulate diss_est

* Allow GEOS_WRAPPER to process device data

* Add clear to collector for 3rd party use. GEOS pass down timings to caller

* Make kernel analysis run a copy stencil to compute local bandwith
Parametrize tool with backend, output format

* Move constant on a env var
Add saturation adjustement threshold to const

* Remove unused if leading to empty code block

* Restrict dace to 0.14.1 due to a parsing bug

* Add guard for bdt==0
Fix bad merge for bdt with GEOS_Wrapper

* Remove unused code

* Fix theroritical timings

* Fixed a bug where pkz was being calculated twice, and the second calc was wrong

* Downgrade DaCe to 0.14.0 pending array aliasing fix

* Set default cache path for orchestrated DaCe to respect GT_CACHE_* env

* Remove previous per stencil override of default_build_folder

* Revert "Set default cache path for orchestrated DaCe to respect GT_CACHE_* env"

* Revert "Remove previous per stencil override of default_build_folder"

* Read cache_root in default dace backend

* Document faulty behavior with GT_CACHE_DIR_NAME

* Fix bad requirements syntax

* Check for the string value of CONST_VERSION directly instead of enum

* Protect constant selection more rigorusly.
Clean abort on unknown constant given

* Log constants selection

* Refactor NQ to constants.py

* Fix or explain inlined import

* Verbose runtime error when bad dt_atmos

* Verbose warm up

* re-initialize heat_source and diss_est each call, add do_skeb check to accumulation

---------




---------






* [NOAA:Update] Bring back #15 & doubly periodic domain (#25)

* Feature/dp driver (#13)

* initial commit

* adding test config

* adding the rest of driver and util code

* updating history.md

* move u_max to dycore config

* uncomment assert

* added comment explaining the copy of grid type to dycore config

* Turn main unit test  & lint on PR, logger clean up [NASA:Update]  (#15)

* Initialize GeosDycoreWrapper with bdt (timestep)

* Use GEOS version of constants

* 1. Add qcld to the list of tracers beings advected
2. Made GEOS specific changes to thresholds in saturation adjustment

* Accumulate diss_est

* Allow GEOS_WRAPPER to process device data

* Add clear to collector for 3rd party use. GEOS pass down timings to caller

* Make kernel analysis run a copy stencil to compute local bandwith
Parametrize tool with backend, output format

* Move constant on a env var
Add saturation adjustement threshold to const

* Restrict dace to 0.14.1 due to a parsing bug

* Add guard for bdt==0

* Fix theroritical timings

* Fixed a bug where pkz was being calculated twice, and the second calc was wrong

* Downgrade DaCe to 0.14.0 pending array aliasing fix

* Set default cache path for orchestrated DaCe to respect GT_CACHE_* env

* Remove previous per stencil override of default_build_folder

* Revert "Set default cache path for orchestrated DaCe to respect GT_CACHE_* env"

* Read cache_root in default dace backend

* Document faulty behavior with GT_CACHE_DIR_NAME

* Check for the string value of CONST_VERSION directly instead of enum

* Protect constant selection more rigorusly.
Clean abort on unknown constant given

* Log constants selection

* Refactor NQ to constants.py

* Introduce PACE_LOGLEVEL to control log level from outside

* Code guidelines clean up

* Devops/GitHub actions on (#15)

* Linting on PR

* Run main unit test

* Update python to available 3.8.12

* Fix unit tests (remove dxa, dya rely on halo ex)

* Update HISTORY.md

* Adapt log_level in driver.run

* Verbose the PACE_CONSTANTS

* Doc log level hierarchical nature

---------




* Lint

---------





* Update gt4py, dace, cleanup (#19)

* Update gt4py to top of master on June 21

* Update DaCe to 0.14.2
Workaround aliasing issue in FiniteVolumeTransport

* Fix to gt4py storage

* Downgrade to dace 0.14.1

* DaCe to 0.14.4
Orchestrating NonHydrostaticPressureGradient
Adptating code to newer gt4py

* Regenerate constraints.txt

* Default constants to GFS
Fix snapshot for GPU runs
Lint on ETA
Fix log level

* Remove `daint_venv` submodule

* Adding dace as a submodule
Removing buildenv as a submodule

* Update gt4py to latest master

* Skip ConstantPropagation during `Simplify`

* Remove buidlenv

* Update requirements_dev.txt

* Add editable util to requirements_dev.txt

* lint

* scipy for tests is now needed

* Pin `DaCe` to pace-fixes-0 merge

* Remove logging setup in test_translate

* Make cupy import robust to device not being available

* Fix to GEOS bridge MPS detection

* Up gt4py to August 14th EOD:
  - Hip/ROCm
  - New allocators

* DaCE module: swap SSH for HTTPS (#26)

* GEOS GridTools stencils build override (#27)

* Stencil build override for GEOS

* Deactivate warnings if PACE_LOGLEVEL is > WARNING

* Better log level

* Bad merge (again)

---------







---------

Co-authored-by: Purnendu Chakraborty <[email protected]>
Co-authored-by: Purnendu Chakraborty <[email protected]>
Co-authored-by: Rusty Benson <[email protected]>
Co-authored-by: Oliver Elbert <[email protected]>
Co-authored-by: Oliver Elbert <[email protected]>
FlorianDeconinck added a commit that referenced this pull request Dec 7, 2023
* NASA commits sync (#31)

* Initialize GeosDycoreWrapper with bdt (timestep)

* Use GEOS version of constants

* 1. Add qcld to the list of tracers beings advected
2. Made GEOS specific changes to thresholds in saturation adjustment

* Accumulate diss_est

* Allow GEOS_WRAPPER to process device data

* Add clear to collector for 3rd party use. GEOS pass down timings to caller

* Make kernel analysis run a copy stencil to compute local bandwith
Parametrize tool with backend, output format

* Move constant on a env var
Add saturation adjustement threshold to const

* lint

* More linting

* Remove unused if leading to empty code block

* Restrict dace to 0.14.1 due to a parsing bug

* Add guard for bdt==0
Fix bad merge for bdt with GEOS_Wrapper

* Remove unused code

* Fix theroritical timings
Lint

* Fixed a bug where pkz was being calculated twice, and the second calc was wrong

* Downgrade DaCe to 0.14.0 pending array aliasing fix

* Set default cache path for orchestrated DaCe to respect GT_CACHE_* env

* Remove previous per stencil override of default_build_folder

* Revert "Set default cache path for orchestrated DaCe to respect GT_CACHE_* env"

This reverts commit 4fc5b4d.

* Revert "Remove previous per stencil override of default_build_folder"

This reverts commit 2245027.

* Read cache_root in default dace backend

* Document faulty behavior with GT_CACHE_DIR_NAME

* Fix bad requirements syntax

* Check for the string value of CONST_VERSION directly instead of enum

* Protect constant selection more rigorusly.
Clean abort on unknown constant given

* Log constants selection

* Refactor NQ to constants.py

* Replace all logger with pace_log
Introduce PACE_LOGLEVEL to control log level from outside

* Code guidelines clean up

* Devops/GitHub actions on (#15)

* Linting on PR

* Run main unit test

* Update python to available 3.8.12

* Remove cd to pace

* Lint: git submodule recursive

* Typo

* Add openmpi to the image

* Linting

* Fix unit tests (remove dxa, dya rely on halo ex)

* typo

* Change name of jobs

* Distributed compilation on orchestrated backend for NxN layouts (#14)

* Adapt orchestration distribute compile for NxN layout

* Remove debug code

* Add a more descriptive string base postfix for cache naming
Identify the code path for all cases
Consistent reload post-compile
Create a central space for all caches generation logic
No more original layout check required

* Add a test on caches relocatability

* Verbose todo

* Linting on PR

* Run main unit test

* Update python to available 3.8.12

* Remove cd to pace

* Lint: git submodule recursive

* Typo

* Add openmpi to the image

* Linting

* Fix unit tests (remove dxa, dya rely on halo ex)

* typo

* Change name of jobs

* Missing enum

* Lint imports

* Fix unit tests

* Deactivate relocability test due to Python crash
Logged as issyue 16

* Typo

* Raise for 1,X and X,1 layouts which requires a new descriptor

* Added ak, bk for 137 levels in eta.py

* Add floating point precision to GEOS bridge init

* lint

* Add device PCI bus id (for MPS debug)

* Typo + lint

* Try to detect MPS reading the "log" pipe

* Lint

* Clean up

* Log info GEOS bridge (#18)

* Add floating point precision to GEOS bridge init

* lint

* Add device PCI bus id (for MPS debug)

* Typo + lint

* Try to detect MPS reading the "log" pipe

* Lint

* Clean up

* Update geos/develop to grab NOAA PR9 results (#21)

* Verbose choice of block/grid size

* added build script for c5

* updated repo to NOAA

* GEOS integration (#9)

* Initialize GeosDycoreWrapper with bdt (timestep)

* Use GEOS version of constants

* 1. Add qcld to the list of tracers beings advected
2. Made GEOS specific changes to thresholds in saturation adjustment

* Accumulate diss_est

* Allow GEOS_WRAPPER to process device data

* Add clear to collector for 3rd party use. GEOS pass down timings to caller

* Make kernel analysis run a copy stencil to compute local bandwith
Parametrize tool with backend, output format

* Move constant on a env var
Add saturation adjustement threshold to const

* Remove unused if leading to empty code block

* Restrict dace to 0.14.1 due to a parsing bug

* Add guard for bdt==0
Fix bad merge for bdt with GEOS_Wrapper

* Remove unused code

* Fix theroritical timings

* Fixed a bug where pkz was being calculated twice, and the second calc was wrong

* Downgrade DaCe to 0.14.0 pending array aliasing fix

* Set default cache path for orchestrated DaCe to respect GT_CACHE_* env

* Remove previous per stencil override of default_build_folder

* Revert "Set default cache path for orchestrated DaCe to respect GT_CACHE_* env"

* Revert "Remove previous per stencil override of default_build_folder"

* Read cache_root in default dace backend

* Document faulty behavior with GT_CACHE_DIR_NAME

* Fix bad requirements syntax

* Check for the string value of CONST_VERSION directly instead of enum

* Protect constant selection more rigorusly.
Clean abort on unknown constant given

* Log constants selection

* Refactor NQ to constants.py

* Fix or explain inlined import

* Verbose runtime error when bad dt_atmos

* Verbose warm up

* re-initialize heat_source and diss_est each call, add do_skeb check to accumulation

---------

Co-authored-by: Purnendu Chakraborty <[email protected]>
Co-authored-by: Oliver Elbert <[email protected]>

---------

Co-authored-by: Rusty Benson <[email protected]>
Co-authored-by: Oliver Elbert <[email protected]>
Co-authored-by: Purnendu Chakraborty <[email protected]>
Co-authored-by: Oliver Elbert <[email protected]>

* [NOAA:Update] Bring back #15 & doubly periodic domain (#25)

* Feature/dp driver (#13)

* initial commit

* adding test config

* adding the rest of driver and util code

* updating history.md

* move u_max to dycore config

* uncomment assert

* added comment explaining the copy of grid type to dycore config

* Turn main unit test  & lint on PR, logger clean up [NASA:Update]  (#15)

* Initialize GeosDycoreWrapper with bdt (timestep)

* Use GEOS version of constants

* 1. Add qcld to the list of tracers beings advected
2. Made GEOS specific changes to thresholds in saturation adjustment

* Accumulate diss_est

* Allow GEOS_WRAPPER to process device data

* Add clear to collector for 3rd party use. GEOS pass down timings to caller

* Make kernel analysis run a copy stencil to compute local bandwith
Parametrize tool with backend, output format

* Move constant on a env var
Add saturation adjustement threshold to const

* Restrict dace to 0.14.1 due to a parsing bug

* Add guard for bdt==0

* Fix theroritical timings

* Fixed a bug where pkz was being calculated twice, and the second calc was wrong

* Downgrade DaCe to 0.14.0 pending array aliasing fix

* Set default cache path for orchestrated DaCe to respect GT_CACHE_* env

* Remove previous per stencil override of default_build_folder

* Revert "Set default cache path for orchestrated DaCe to respect GT_CACHE_* env"

* Read cache_root in default dace backend

* Document faulty behavior with GT_CACHE_DIR_NAME

* Check for the string value of CONST_VERSION directly instead of enum

* Protect constant selection more rigorusly.
Clean abort on unknown constant given

* Log constants selection

* Refactor NQ to constants.py

* Introduce PACE_LOGLEVEL to control log level from outside

* Code guidelines clean up

* Devops/GitHub actions on (#15)

* Linting on PR

* Run main unit test

* Update python to available 3.8.12

* Fix unit tests (remove dxa, dya rely on halo ex)

* Update HISTORY.md

* Adapt log_level in driver.run

* Verbose the PACE_CONSTANTS

* Doc log level hierarchical nature

---------

Co-authored-by: Purnendu Chakraborty <[email protected]>
Co-authored-by: Purnendu Chakraborty <[email protected]>

* Lint

---------

Co-authored-by: Oliver Elbert <[email protected]>
Co-authored-by: Purnendu Chakraborty <[email protected]>
Co-authored-by: Purnendu Chakraborty <[email protected]>

* Update gt4py, dace, cleanup (#19)

* Update gt4py to top of master on June 21

* Update DaCe to 0.14.2
Workaround aliasing issue in FiniteVolumeTransport

* Fix to gt4py storage

* Downgrade to dace 0.14.1

* DaCe to 0.14.4
Orchestrating NonHydrostaticPressureGradient
Adptating code to newer gt4py

* Regenerate constraints.txt

* Default constants to GFS
Fix snapshot for GPU runs
Lint on ETA
Fix log level

* Remove `daint_venv` submodule

* Adding dace as a submodule
Removing buildenv as a submodule

* Update gt4py to latest master

* Skip ConstantPropagation during `Simplify`

* Remove buidlenv

* Update requirements_dev.txt

* Add editable util to requirements_dev.txt

* lint

* scipy for tests is now needed

* Pin `DaCe` to pace-fixes-0 merge

* Remove logging setup in test_translate

* Make cupy import robust to device not being available

* Fix to GEOS bridge MPS detection

* Up gt4py to August 14th EOD:
  - Hip/ROCm
  - New allocators

* DaCE module: swap SSH for HTTPS (#26)

* GEOS GridTools stencils build override (#27)

* Stencil build override for GEOS

* Deactivate warnings if PACE_LOGLEVEL is > WARNING

* Better log level

* Bad merge (again)

* NASA fork sync. (#37) (#30)

* Initialize GeosDycoreWrapper with bdt (timestep)

* Use GEOS version of constants

* 1. Add qcld to the list of tracers beings advected
2. Made GEOS specific changes to thresholds in saturation adjustment

* Accumulate diss_est

* Allow GEOS_WRAPPER to process device data

* Add clear to collector for 3rd party use. GEOS pass down timings to caller

* Make kernel analysis run a copy stencil to compute local bandwith
Parametrize tool with backend, output format

* Move constant on a env var
Add saturation adjustement threshold to const

* lint

* More linting

* Remove unused if leading to empty code block

* Restrict dace to 0.14.1 due to a parsing bug

* Add guard for bdt==0
Fix bad merge for bdt with GEOS_Wrapper

* Remove unused code

* Fix theroritical timings
Lint

* Fixed a bug where pkz was being calculated twice, and the second calc was wrong

* Downgrade DaCe to 0.14.0 pending array aliasing fix

* Set default cache path for orchestrated DaCe to respect GT_CACHE_* env

* Remove previous per stencil override of default_build_folder

* Revert "Set default cache path for orchestrated DaCe to respect GT_CACHE_* env"

This reverts commit 4fc5b4d.

* Revert "Remove previous per stencil override of default_build_folder"

This reverts commit 2245027.

* Read cache_root in default dace backend

* Document faulty behavior with GT_CACHE_DIR_NAME

* Fix bad requirements syntax

* Check for the string value of CONST_VERSION directly instead of enum

* Protect constant selection more rigorusly.
Clean abort on unknown constant given

* Log constants selection

* Refactor NQ to constants.py

* Replace all logger with pace_log
Introduce PACE_LOGLEVEL to control log level from outside

* Code guidelines clean up

* Devops/GitHub actions on (#15)

* Linting on PR

* Run main unit test

* Update python to available 3.8.12

* Remove cd to pace

* Lint: git submodule recursive

* Typo

* Add openmpi to the image

* Linting

* Fix unit tests (remove dxa, dya rely on halo ex)

* typo

* Change name of jobs

* Distributed compilation on orchestrated backend for NxN layouts (#14)

* Adapt orchestration distribute compile for NxN layout

* Remove debug code

* Add a more descriptive string base postfix for cache naming
Identify the code path for all cases
Consistent reload post-compile
Create a central space for all caches generation logic
No more original layout check required

* Add a test on caches relocatability

* Verbose todo

* Linting on PR

* Run main unit test

* Update python to available 3.8.12

* Remove cd to pace

* Lint: git submodule recursive

* Typo

* Add openmpi to the image

* Linting

* Fix unit tests (remove dxa, dya rely on halo ex)

* typo

* Change name of jobs

* Missing enum

* Lint imports

* Fix unit tests

* Deactivate relocability test due to Python crash
Logged as issyue 16

* Typo

* Raise for 1,X and X,1 layouts which requires a new descriptor

* Added ak, bk for 137 levels in eta.py

* Add floating point precision to GEOS bridge init

* lint

* Add device PCI bus id (for MPS debug)

* Typo + lint

* Try to detect MPS reading the "log" pipe

* Lint

* Clean up

* Log info GEOS bridge (#18)

* Add floating point precision to GEOS bridge init

* lint

* Add device PCI bus id (for MPS debug)

* Typo + lint

* Try to detect MPS reading the "log" pipe

* Lint

* Clean up

* Update geos/develop to grab NOAA PR9 results (#21)

* Verbose choice of block/grid size

* added build script for c5

* updated repo to NOAA

* GEOS integration (#9)

* Initialize GeosDycoreWrapper with bdt (timestep)

* Use GEOS version of constants

* 1. Add qcld to the list of tracers beings advected
2. Made GEOS specific changes to thresholds in saturation adjustment

* Accumulate diss_est

* Allow GEOS_WRAPPER to process device data

* Add clear to collector for 3rd party use. GEOS pass down timings to caller

* Make kernel analysis run a copy stencil to compute local bandwith
Parametrize tool with backend, output format

* Move constant on a env var
Add saturation adjustement threshold to const

* Remove unused if leading to empty code block

* Restrict dace to 0.14.1 due to a parsing bug

* Add guard for bdt==0
Fix bad merge for bdt with GEOS_Wrapper

* Remove unused code

* Fix theroritical timings

* Fixed a bug where pkz was being calculated twice, and the second calc was wrong

* Downgrade DaCe to 0.14.0 pending array aliasing fix

* Set default cache path for orchestrated DaCe to respect GT_CACHE_* env

* Remove previous per stencil override of default_build_folder

* Revert "Set default cache path for orchestrated DaCe to respect GT_CACHE_* env"

* Revert "Remove previous per stencil override of default_build_folder"

* Read cache_root in default dace backend

* Document faulty behavior with GT_CACHE_DIR_NAME

* Fix bad requirements syntax

* Check for the string value of CONST_VERSION directly instead of enum

* Protect constant selection more rigorusly.
Clean abort on unknown constant given

* Log constants selection

* Refactor NQ to constants.py

* Fix or explain inlined import

* Verbose runtime error when bad dt_atmos

* Verbose warm up

* re-initialize heat_source and diss_est each call, add do_skeb check to accumulation

---------




---------






* [NOAA:Update] Bring back #15 & doubly periodic domain (#25)

* Feature/dp driver (#13)

* initial commit

* adding test config

* adding the rest of driver and util code

* updating history.md

* move u_max to dycore config

* uncomment assert

* added comment explaining the copy of grid type to dycore config

* Turn main unit test  & lint on PR, logger clean up [NASA:Update]  (#15)

* Initialize GeosDycoreWrapper with bdt (timestep)

* Use GEOS version of constants

* 1. Add qcld to the list of tracers beings advected
2. Made GEOS specific changes to thresholds in saturation adjustment

* Accumulate diss_est

* Allow GEOS_WRAPPER to process device data

* Add clear to collector for 3rd party use. GEOS pass down timings to caller

* Make kernel analysis run a copy stencil to compute local bandwith
Parametrize tool with backend, output format

* Move constant on a env var
Add saturation adjustement threshold to const

* Restrict dace to 0.14.1 due to a parsing bug

* Add guard for bdt==0

* Fix theroritical timings

* Fixed a bug where pkz was being calculated twice, and the second calc was wrong

* Downgrade DaCe to 0.14.0 pending array aliasing fix

* Set default cache path for orchestrated DaCe to respect GT_CACHE_* env

* Remove previous per stencil override of default_build_folder

* Revert "Set default cache path for orchestrated DaCe to respect GT_CACHE_* env"

* Read cache_root in default dace backend

* Document faulty behavior with GT_CACHE_DIR_NAME

* Check for the string value of CONST_VERSION directly instead of enum

* Protect constant selection more rigorusly.
Clean abort on unknown constant given

* Log constants selection

* Refactor NQ to constants.py

* Introduce PACE_LOGLEVEL to control log level from outside

* Code guidelines clean up

* Devops/GitHub actions on (#15)

* Linting on PR

* Run main unit test

* Update python to available 3.8.12

* Fix unit tests (remove dxa, dya rely on halo ex)

* Update HISTORY.md

* Adapt log_level in driver.run

* Verbose the PACE_CONSTANTS

* Doc log level hierarchical nature

---------




* Lint

---------





* Update gt4py, dace, cleanup (#19)

* Update gt4py to top of master on June 21

* Update DaCe to 0.14.2
Workaround aliasing issue in FiniteVolumeTransport

* Fix to gt4py storage

* Downgrade to dace 0.14.1

* DaCe to 0.14.4
Orchestrating NonHydrostaticPressureGradient
Adptating code to newer gt4py

* Regenerate constraints.txt

* Default constants to GFS
Fix snapshot for GPU runs
Lint on ETA
Fix log level

* Remove `daint_venv` submodule

* Adding dace as a submodule
Removing buildenv as a submodule

* Update gt4py to latest master

* Skip ConstantPropagation during `Simplify`

* Remove buidlenv

* Update requirements_dev.txt

* Add editable util to requirements_dev.txt

* lint

* scipy for tests is now needed

* Pin `DaCe` to pace-fixes-0 merge

* Remove logging setup in test_translate

* Make cupy import robust to device not being available

* Fix to GEOS bridge MPS detection

* Up gt4py to August 14th EOD:
  - Hip/ROCm
  - New allocators

* DaCE module: swap SSH for HTTPS (#26)

* GEOS GridTools stencils build override (#27)

* Stencil build override for GEOS

* Deactivate warnings if PACE_LOGLEVEL is > WARNING

* Better log level

* Bad merge (again)

---------

Co-authored-by: Purnendu Chakraborty <[email protected]>
Co-authored-by: Purnendu Chakraborty <[email protected]>
Co-authored-by: Rusty Benson <[email protected]>
Co-authored-by: Oliver Elbert <[email protected]>
Co-authored-by: Oliver Elbert <[email protected]>

---------

Co-authored-by: Purnendu Chakraborty <[email protected]>
Co-authored-by: Purnendu Chakraborty <[email protected]>
Co-authored-by: Rusty Benson <[email protected]>
Co-authored-by: Oliver Elbert <[email protected]>
Co-authored-by: Oliver Elbert <[email protected]>

* Update DaCe to 0.15.1 RC (#35)

- Update: DaCe to 0.15.1 RC and GT4Py to latest main 
- Minor: orchestration build logging 
- Minor: dead code clean up

---------

Co-authored-by: Purnendu Chakraborty <[email protected]>
Co-authored-by: Purnendu Chakraborty <[email protected]>
Co-authored-by: Rusty Benson <[email protected]>
Co-authored-by: Oliver Elbert <[email protected]>
Co-authored-by: Oliver Elbert <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant