Updates on 04_21 #2

Merge in latest dev/gfdl updates

Merge in latest MOM6 dev/gfdl updates

Merge in latest commits

Merge in latest MOM dev/gfdl updates

Merge in latest MOM6 updates

Merge in latest dev/gfdl updates

merge latest updates into dev/gfdl

Merge in latest dev/gfdl commits

Merge in dev/gfdl updates

Merge in latest dev/gfdl updates

merge in latest dev/gfdl changes

merge in latest dev/gfdl updates

Merge in latest MOM6 updates

Merge in latest dev/gfdl updates

Merge in latest round of dev/gfdl updates

merge in latest dev/gfdl updates

Merge in latest dev/gfdl updates

Merge in latest updates from dev/gfdl

Merge in latest dev/gfdl updates

Merge in updates to remap_all_state_vars

Merge in latest updates

Merge in dev/gfdl updates

merge dev/emc

* TC4 integration into test suite This patch renames the tc4 test to activate it in the test suite. It also modifies the Makefile to build the input field test scripts. It also modifies the Python build scripts to be PEP8-conformant. We temporarily disable tc4 in the restart tests, since they currently fail. This needs to be addressed before we can merge this into the main branch. The patch does not enable the necessary Python modules for running on Travis, that will also be addressed later. * Travis python support; tc4 Makefile The custom TC4 Makefile has been added (oops), and the presumed Python Ubuntu packages have been added for Travis. * Verify ENABLE_THERMODYNAMICS is True before posting C_p diagnostic * Make tc4 faster * remove trailing whitespace * add unit scaling * fix restart fail for tc4 and some cleanup * remove trailiny ws * Enable tc4.restart test * +Pass timeesteps to tracer diagnostics in [T] Pass timeesteps to the tracer diagnistics routines post_tracer_diagnostics and postALE_tracer_diagnostics and to adiabatic in units of [T}. All answers are bitwise identical. * +Rescaled tracer advective flux diagnostics Rescaled the internal units of the tracer advective flux diagnostics to units of [conc H L2 T-1] for code simplicity and dimensional consistency testing. Also corrected the units of some tracer fluxes as documented in comments and commented out unused elements of the tracer_type. All answers are bitwise identical. * +Pass timesteps to ALE_main in [T] Pass the timesteps to ALE_main, ALE_main_offline, and ALE_main_accelerated in units of [T] for code simplicity and dimensional consistency testing. This also includes the rescaling of remapping-driven tracer tendencies. All answers and diagnostics are bitwise identical. * +Pass timesteps to tracer column_physics in [T] Pass timesteps to the various tracer column_physics routines in [T] for dimensional consistency testing. Also added a new unit_scale_type argument to these routines. All answers are bitwise identical, but there are minor interface changes to 13 subroutines. * +Pass timesteps to applyTracerBoundaryFluxesInOut in [T] Pass timesteps to applyTracerBoundaryFluxesInOut in [T], and use units of [T-1] for internal source and decay rates for the oil tracer and in fluxes of CFCs. Also modified extract_offline_main to return timesteps as real values with units of [T]. Also there is a new unit_scale_type argument to register_oil_tracer. All answers in the MOM6_examples test cases and regression tests are bitwise identical. * Simplified expressions in MOM_PointAccel Simplified expressions inside of MOM_PointAccel, taking into account that all velocities use the same units of [L T-1]. All answers are bitwise identical. * Corrected dimensional epsilons in downscaling Added distinct negligible volumes, face areas, horizonal areas and lengths with proper dimensional rescaling in the downsample field routines. With these changes, downscaled diagnostics should now pass the dimensional rescaling tests, whereas previously there would have been a problem when the numbers used to represent lengths are smaller than about 1e-8 times their MKS values. All answers are bitwise identical without dimensional rescaling. * Simplified expressions in MOM_offline_aux Simplified expressions in distribute_residual_uh_barotropic. All answers are bitwise identical. * Revised wave_speed to return speed in [L T-1] Revised wave_speed to return the internal wave speed in units of [L T-1] and to use mono_N2_depth in units of [Z] for code simplification and expanded dimensional consistency testing. Also revised the internal units of some related diagnostics in calculate_diagnostic_fields. All answers and diagnostics are bitwise identical. * Rescaled internal variables in wave_speed Rescale internal calculations in wave_speed and wave_speeds for greater robustness and dimensional consistency testing. All answers are bitwise identical and pass dimensional scaling tests. * +Changed the units of minimum_forcing_depth to [H] Changed the units of minimum_forcing_depth passed to applyBoundaryFluxesInOut and applyTracerBoundaryFluxesInOut to [H]. All answers are bitwise identical. * Correction of documented units in comments Corrected some units in comments and eliminated some unused variables. All answers are bitwise identical. * Adiabatic clock ID bugfix This patch fixes an initialization bug of the diabatic timer, which was being used to measure adiabatic time but was never initialized if the experiment was configured as adiabatic. We fix this by introducing a separate timer for the adiabatic solver. Although we could have reused the diabatic timer, the addition of a new variable should not add any overhead on modern compilers. * Corrected an OMP declaration Added a variable to an OMP declaration. All answers are bitwise identical, and a recently added compile-time error with openMP was fixed. * Update MOM.F90 Fixed Alistair's embarrassing error. * Dimensional rescaling in MOM_open_boundary.F90 Added rescaling for dimensional consistency testing in MOM_open_boundary.F90, including splitting variables with different units that had previously shared the same variable and adding more extensive documentation of variables. Also changed the dimensions of the timesteps passed to radiation_open_bdry_conds and update_segment_tracer_reservoirs to [T] and added vertical_grid_type and unit_scale_type arguments to open_boundary_init and open_boundary_test_extern_h. All answers are bitwise identical, although some probably bugs have been noted in comments and there are new or altered arguments to several routines. * (*)Fixed invariance bugs in MOM_open_boundary.F90 Corrected dimensional consistency bugs in update_segment_tracer_reservoirs and horizontal indexing and related bugs in gradient_at_q_points with oblique_grad OBCs. These will both change answers in test cases that use some open boundary condition options, but not in any of the MOM6-examples test cases.

This reverts commit 855d706.

Revert "Merge in dev/gfdl updates"

…GFDL-dev/gfdl Conflicts: src/parameterizations/lateral/MOM_MEKE.F90

…/gfdl

merge dev/emc

* (*)Fixed dimensional inconsistency in P3M_functions Corrected dimensionally inconsistent expressions in P3M_functions.F90, notably in P3M_limiter and monotonize_cubic and a complete rewrite and simplification of is_cubic_monotonic. Also added comments documenting the units of all real variables in this module, and changed the code to use logical variables in place of integer "booleans", including in the return value from is_cubic_monotonic. These changes will change (fix) the answers when remapping variables with small numerical values, but no answers change in the MOM6-examples test cases. * +Added REMAPPING_2018 runtime option Added a new runtime option, REMAPPING_2018, which if set to false triggers the use of new, more accurate expressions in various parts of the ALE remapping code. By default, the older expressions are used, and all answers are bitwise identical, but there are new optional arguments to various routines related to remapping to trigger the use of new mathematically equivalent expressions. By default all answers are bitwise identical, but there are new and reordered entries in the MOM6_parameter_doc files. * Corrected the formatting of a doxygen comment * Added conversion factors to forcing diagnostics Added conversion factors to 4 mass-flux diagnostics and comments to 4 others on why no conversion factors are needed. All answers are bitwise identical. * Added correct scaling factors to chksum calls Added scale arguments to 5 chksum calls and grouped another two chksum calls while also adding the right scaling argument. All answers are bitwise identical. * +Unscales area before taking global sum Undoes the dimensional scaling of the cell areas before taking their global sum, so that the reproducing sum does not overflow when there is dimensional rescaling. All answers are bitwise identical when there is no rescaling, but this eliminates a source of inadvertent overflows or underflows in the global sums, and there is a new optional argument to compute_global_grid_integrals. * (*)Correct dimensionally inconsistent advective CFL Corrects the dimensionally inconsistent expressions for the CFL number in the tracer advection code, in which a negligible thickness had been added to the cell volume to avoid division by zero. This change does not alter the solutions in the MOM6-examples test cases, but now it permits dimensional rescaling of lengths over a much larger range, and it could change answers if the minimum layer thicknesses are small enough. * Unscale sea level before averaging Unscale interface heights before taking a global average via a reproducing sum in non-Boussinesq mode global diagnostics to permit dimensional consistency testing over a larger range. All answers are bitwise identical. * +Added an optional tmp_scale arg to global_i_mean Added an optional tmp_scale argument to global_i_mean and global_j_mean to specify an internal rescaling of variables being averaged before the reproducing sum. All answers are bitwise identical, but there are new optional arguments to two public interfaces. * Expand consistency testing with i-mean sponges Use tmp_scale when taking the i-mean interface heights for i-mean sponges, to give a greatly expanded range of dimensional consistency testing. All answers are bitwise identical.

Merge in dev/gfdl updates

Merge in latest dev/gfdl updates

merge to latest dev/emc

Merge in latest dev/gfdl updates

merge in latest dev/gfdl updates

Merge in updates from dev/gfdl

Merge in latest dev/gfdl updates

Merge in dev/gfdl updates

Merge in latest dev/gfdl updates

Merge in dev/gfdl updates

merge in dev/gfdl updates

Feature/logcleanup (#20)

nuopc_driver updates (#22)

Merge in latest updates from dev/gfdl

Merge in latest dev/gfdl updates

merge in dev/gfdl updates

Sync with NOAA-GFDL dev/gfdl branch

* 'dev/gfdl' of github.com:NOAA-GFDL/MOM6: Fix to out-of-bounds error Slight improvement in writeMessageAndDesc Corrected recent bug in writeMessageAndDesc Included H_subroundoff in a denominator +Added code to handle tab lengths in documentation Added explicit form of cpu_clock_id (*)Improve ePBL when EPBL_ORIGINAL_PE_CALC=False Alternate fix for divide-by-zero. Clean up some divide-by-zero issues in OBCs. Added a space to a description. Codecov: Set base as parent (target) branch Fixes an integer-kind mismatch in MOM_random, seed_from_time() (#1113)

Merge in dev/gfdl updates

Sync with MOM6 dev/gfdl

Sync with NOAA-GFDL dev/gfdl

sync with dev/gfdl

Sync with NOAA-GFDL dev/gfdl

This reverts commit e4ca1dc, reversing changes made to 40bfb4b.

Sync with NOAA-GFDL

Sync with NOAA-GFDL dev/gfdl

…ons of the routines with the fms-io or fms2-io interfaces added module use statments for fms2_io and MOM_io helper routines to MOM_restart added use_fms2=.true. arguments to save_restart and restore_state calls added write_ic=.true. to the save_restart call in MOM.F90 added module MOM_axis with routines to define and register axes and their metadata added module MOM_read_data_fms2.F90 with wrappers for fms2_io read_data interfaces and required routines added module MOM_write_field_fms2.F90 with wrappers for fms2_io write_data interfaces updated module use statments in MOM_io and MOM_restart to reference routines in MOM_read_data_fms2, MOM_write_field_fms2, and MOM_axis made write_field and create_file interfaces in MOM_io added create_file routines to MOM_io that accept file names or file objects to create/overwrite netcdf files that will be written to via write_field calls fixed compile-time errors added new MOM_read_data routines to MOM_read_data interface in MOM_io added placeholder call for new write_field routines to MOM_io changed use_fms2 to a required first argument in save_restart_fms2 and restore_state_fms2 changed write_ic to a required argument in write_initial_conditions fixed the layer and interface checks in MOM_get_diagnostic_axis_data commented out manual checksum registration in save_restart_fms2 so that internal fms2-io checksum computation is used added checks for time units to restore_state and save_restart added logic to make the restart time 1 to save_restart_fms2 if there is an abnormally large value passed to the routine added interface routine file_exists_FMS2 that uses the fms2_io file_exists call added subroutine get_num_restart_files to MOM_restart that searches for known variants of the input file names and returns then number of restart files available for querying, and the optional list of filepaths added loop to search the files for all mandatory variables in the list of file paths returned to by cal to get_num_restart_files added calls to get the variable dimension names and pass them as arguments to register_restart_field in restore_state_fms2 removed exit from inner variable loop in restore_state_fms2 code cleanup moved missing_fields=0 outside of the CS loop in restore_state_fms2 moved missing_fields=0 outside of the CS loop in restore_state_fms2 converted save_restart and restore_state to interface that call versions of the routines with the fms-io or fms2-io interfaces added module use statments for fms2_io and MOM_io helper routines to MOM_restart added use_fms2=.true. arguments to save_restart and restore_state calls added write_ic=.true. to the save_restart call in MOM.F90 added module MOM_axis with routines to define and register axes and their metadata added module MOM_read_data_fms2.F90 with wrappers for fms2_io read_data interfaces and required routines added module MOM_write_field_fms2.F90 with wrappers for fms2_io write_data interfaces updated module use statments in MOM_io and MOM_restart to reference routines in MOM_read_data_fms2, MOM_write_field_fms2, and MOM_axis made write_field and create_file interfaces in MOM_io added create_file routines to MOM_io that accept file names or file objects to create/overwrite netcdf files that will be written to via write_field calls fixed compile-time errors added new MOM_read_data routines to MOM_read_data interface in MOM_io added placeholder call for new write_field routines to MOM_io changed use_fms2 to a required first argument in save_restart_fms2 and restore_state_fms2 changed write_ic to a required argument in write_initial_conditions fixed the layer and interface checks in MOM_get_diagnostic_axis_data commented out manual checksum registration in save_restart_fms2 so that internal fms2-io checksum computation is used added checks for time units to restore_state and save_restart added logic to make the restart time 1 to save_restart_fms2 if there is an abnormally large value passed to the routine added interface routine file_exists_FMS2 that uses the fms2_io file_exists call added subroutine get_num_restart_files to MOM_restart that searches for known variants of the input file names and returns then number of restart files available for querying, and the optional list of filepaths added loop to search the files for all mandatory variables in the list of file paths returned to by cal to get_num_restart_files added calls to get the variable dimension names and pass them as arguments to register_restart_field in restore_state_fms2 removed exit from inner variable loop in restore_state_fms2 code cleanup removed test workflow directory moved missing_fields=0 outside of the CS loop in restore_state_fms2 converted save_restart and restore_state to interface that call versions of the routines with the fms-io or fms2-io interfaces added module use statments for fms2_io and MOM_io helper routines to MOM_restart added use_fms2=.true. arguments to save_restart and restore_state calls added write_ic=.true. to the save_restart call in MOM.F90 added module MOM_axis with routines to define and register axes and their metadata added module MOM_read_data_fms2.F90 with wrappers for fms2_io read_data interfaces and required routines added module MOM_write_field_fms2.F90 with wrappers for fms2_io write_data interfaces updated module use statments in MOM_io and MOM_restart to reference routines in MOM_read_data_fms2, MOM_write_field_fms2, and MOM_axis made write_field and create_file interfaces in MOM_io added create_file routines to MOM_io that accept file names or file objects to create/overwrite netcdf files that will be written to via write_field calls fixed compile-time errors added new MOM_read_data routines to MOM_read_data interface in MOM_io added placeholder call for new write_field routines to MOM_io changed use_fms2 to a required first argument in save_restart_fms2 and restore_state_fms2 changed write_ic to a required argument in write_initial_conditions fixed the layer and interface checks in MOM_get_diagnostic_axis_data commented out manual checksum registration in save_restart_fms2 so that internal fms2-io checksum computation is used added checks for time units to restore_state and save_restart added logic to make the restart time 1 to save_restart_fms2 if there is an abnormally large value passed to the routine added interface routine file_exists_FMS2 that uses the fms2_io file_exists call added subroutine get_num_restart_files to MOM_restart that searches for known variants of the input file names and returns then number of restart files available for querying, and the optional list of filepaths added loop to search the files for all mandatory variables in the list of file paths returned to by cal to get_num_restart_files added calls to get the variable dimension names and pass them as arguments to register_restart_field in restore_state_fms2 removed exit from inner variable loop in restore_state_fms2 code cleanup removed test workflow directory removed white space updated FMS tag in .testing Makefile fixed argument comments to use doxygen style in MOM_write_field_fms2 code cleanup removed \TODO from MOM_restart fixed documentation for module variables in MOM_read_data_fms2 and MOM_write_field_fms2 removed doxygenized TODO statements removed new MOM_read_data routines from interface, and commented out calls in MOM_io fixed doxygen definitions more doxygen fixes changed save_restart and restore_state to wrapper routines with opitional use_fms2 and write_ic arguments added use_fms2=.true. and write_ic=.true. to save_restart and restore_state calls removed the error messages from append_substring tried reordering the write_ic and use_fms2 checks in save_restart to fix error with invalid memory reference in the MOM.F90 call to save_restart-write_initial_conditions

…ing the use_fms2 and write_ic flags if present to avoid invalid memory reference error added str_len argument to register_variable_attribute calls added support to for rotated fields to write_initial_conditions and save_restart_fms2 removed whitespace

removed use_fms2 optional arguments in save_restart, restore_state, and create_file made use_fms2 a MOM_restart module variable Update MOM_state_initialization.F90 Remove space. Update MOM_ice_shelf.F90 Add space. Update MOM_driver.F90 Add space Update mom_surface_forcing_mct.F90 remove whitespace Update ocean_model_MOM.F90 Add space Update mom_ocean_model_nuopc.F90 Add space Update MOM_surface_forcing.F90 Add space Changes needed work ESM4 to run with new io

MOM_ice_shelf_dynamics.F90 modified initialize_ice_shelf_boundary_channel in MOM_ice_shelf_initialze.F90

Merge branch 'revert_drivers' of https://github.com/MJHarrison-GFDL/MOM6 into matts-revert-drivers

…e ice-shelf_fields consistent with diag_table Modifications to MOM_ice_shelf.F90 to apply melting to the case of a dynamic ice shelf.

- In A and B grid configuration halos were never updated after taux/tauy were populated. - This propogated through to the ustar_gustless field, hence caused a restart issue when using ustar_gustless in parameterizations. - This appears to correct the restart issue by updating the halos at the end of the A and B grid taux/tauy loops.

Add two halo updates for taux and tauy in mom_surface_forcing_nuopc

…20210120 This is corresponding GFDL 20210120 main branch commit (hash # fe5e605)

skip-ci

…120' into bugfix/li2016fix skip-ci

This patch fixes a sign bug, in both MCT and NUOPC, when accounting for the latent heat from fprec and frunnoff. Following MOM6's definition, both fprec and frunoff are > 0 into the ocean. Therefore, the latent heat associated with these terms should be negative.

Bugfix: sign error on fprec for nuopc and mct caps

Feature/update to gfdl 20210120

* read config variable 'use_mommesh' in ufs; default is false to have mom cap run on grid

Allow MOM6 to use a mesh optionally

initialized in MOM_barotropic

v_bdry_val through halo

due to sub-ice-shelf melting

2. Split the loop that addes Rayleigh drag term to [uv]_accel_bt, so that the if-block is now outside the loop

…update-to-GFDL-20210224 this is corresponding to GFDL 20210224 main branch commit (hash # 00c2819)

Created the new config_src directory tree structure, as agreed upon at https://github.com/NOAA-GFDL/MOM6/discussions/1286, to eventually accomodate the selection of different infrastructures. No .F90 files are changed but there are small changes to ac/configure.ac to accomodate the new structure while also allowing for the old target to use the old structure. All answers are bitwise identical, and all MOM6-examples and TC tests are passing.

Fix the bug that Rayleigh_[uv] was not initialized in MOM_barotropic

Updated the MOM6/docs files and .gitlab-ci.yml to reflect the new config_src directory structure.

…barotropic

- Use a new branch of MRS in the gitlab pipelines

- The job within the gitlab pipeline gnu:ice-ocean-nolibs had a modified search path but a "," instead of a "/" led to al caps being compiled and thus failure.

This patch removes the AC_CHECK_FILE autoconf macro which assigns a default path to the MOM_memory.h file path. This path was defaulting back to the old directory, and was added to support the regression verification testing, but this is a regression problem with should be handled by the regression, not the autoconf configuration. This will produce a regression test fail, but it not a cause for concern.

Remove AC_CHECK_FILE for legacy MOM_memory.h

This patch removes two lines where the target build (used for regression testing) were using the active branch's configure.ac, which caused path mismatches. We now use our local autoconf configuration files. This patch also fixes a typo in the source code dependencies of the target repository. The AC_CHECK_FILE for the solo driver has also beed removed, for the same reason as the MOM_memory.h check. Some comments were also added or reformatted to undocumented flags.

Testing: Target uses local autoconf

Hallberg noaa gfdl fms2

- Tag v0.93-beta of CVMix is the last tag on their "master" branch before an API change within CVMix. - Answers reproduce with the three-year prior commit being used, as tested in MOM6-examples single_column cases. These do not cover all CVMix code.

- This tag involves an API change that requires changes to MOM_tidal_mixing.F90. - Changes are dropped arguments so presumably that data is not needed. - Answers reproduce in so far as they are covered by MOM6-examples. Not all of CVMix is covered in these tests and this code in particular is not.

- v0.98-beta is the latest tag of CVMix that is available a.t.t. - There were no API changes since the v0.94b-beta that affected MOM6. - No answer changes for MOM6-examples, but these do not exercise much of CVMix.

Fix a missing line regarding vel_underflow

Depth-integrated momentum budget diagnostics

…90; MOM_ice_shelf_initialize.F90

Feature/update to gfdl 20210224

Typo correction

Corrected three module use statements in infra/FMS1 to eliminate any dependencies of code in config_src/infra/FMS1 on code in src/framework. These use statements eventually point to the same place as before, but with less indirection. This change should facilitate later steps to compile everything in and under the config_src/infra directories as libraries. All answers are bitwise identical.

Duplicated all of the modules in infra/FMS1 into new copies in infra/FMS2, in preparation for merging in the FMS2 i/o changes to the new infra/FMS2 directory. All answers are bitwise identical, regardless of which config_src/infra directory is used.

This patch fixes an issue with the vertical array bounds of the Ferrari et al. streamfunction. The array is bounded across interfaces, from 1 to nz+1, but only the interior values need to be determined due to the arbitrary boundary value (set here to zero). In the current source, the streamfunction is rescaled before calling streamfn_solver, but need not be applied to the boundary values. This is unlikely to cause errors in production, since the values are later reset to zero, but the rescaling can raise errors in more aggressive debugging builds, such as when the arrays are initialized with NaN values.

Added code to explicitly set a (1x1) io_domain when no other io_layout is specified, complying with changing requirements for 2020 and later versions of FMS, and following the default behavior of previous versions. All answers are bitwise identical.

…om/wrongkindofdoctor/MOM6 into FMS2_io Merged in PR #1165, "Add fms2-io to MOM restart interfaces", but with modifications to move the new files to config_src/infra/FMS2, where they belong, and to avoid inappropriate changes to code in the src/framework directory. All existing tests are passing with config_src/infra/FMS1, but the new changes require that the version of FMS used with config_src/infra/FMS2 to at least 2020.03, and this disables all of our AM2-based coupled test cases (due to the deliberate decision from the GFDL Modeling Systems group to drop support for this older atmospheric model). Although the new code reproduces the previous answers in ocean_only and ice_ocean_SIS2 test cases with the intel and gnu compilers, with the pgi (19.10.0) compiler there are fatal errors in all ice_ocean_SIS2 cases and in ocean_only cases driven by data override. The last version of FMS where the pgi compiler works in all cases is 2019.01. The problem in some ocean_only cases appears to be related to the use of FMS2 i/o calls in the data_override routines. With the ice_ocean_SIS2 cases, the error message in with the pgi compiler is "mpp_broadcast_text: broadcasting from invalid PE.". (Note that with the newer versions of FMS (2020.03 and later), OM_1deg requires that the namelist file for data_override_nml has the added line {reproduce_null_char_bug = .true.} to reproduce the previous answers, but it is a fatal error to add this setting with FMS versions before 2020.03.)

Eliminated scale_data from MOM_read_data_fms2, as it was duplicative of rescale_comp_data from MOM_domain_infra.F90. Also eliminated the optional scale argument from the fms2 write_field routines, because this scaling is already handled in the framework/MOM_io.F90 routines, and because the way the scaling was implemented with a pointer to defeat the intent(in) of the data array to the write routines was a deviously terrible idea that would cause endless problems. Because the mpp_domain in the MOM_domain_types have been modified to always have an io_domain set, the test and calls setting an io_domain in all of the fms2 read_data and fms2 write_data routines. In addition, there is no longer a need for a routine to read specifically from the supergrid, so the fms2 MOM_read_data_2d_supergrid routine was eliminated. The code compiles with these changes, but none of this fms2 I/O code is exercised yet in MOM6, so of course all answers are bitwise identical.

FMS1 Framework integration

…20210308 this is corresponding to GFDL 20210308 main branch commit (hash # a4049e7)

This commit fixes the issue that dirs%restart_input_dir is not initiated before "ocean_solo.res" and "ice_solo.res" are read.

Replaced the two optional arguments xPosition and yPosition to the two routines wrapped by the interface MOM_register_variable_axes in MOM_axis.F90 with a single optional argument, position. Also dramatically refactored the code in read_data_fms2.F90 to eliminate unused variants and adding more granular subroutines to dramatically reduce duplicative code, eliminating several (unexercised) bugs in the process; that file went from 1441 lines long to 924. The code compiles with these changes, but none of this fms2 I/O code is exercised yet in MOM6, so of course all answers are bitwise identical.

Bugfix: empty restart_input_dir for *_solo.res

Cleaned up MOM_restart.F90 in the wake of the recent merge of FMS2 I/O related changes into MOM6. The changes include calling the recently added functions get_variable_byte_size and get_num_restart_files, and revising the names of several variables in open_restart_units and the order in which the count of files is incremented for greater clarity. There are also a number of duplicative comments that were removed while others were reformatted. All answers and output are bitwise identical.

Merge from main

Added code to explicitly set a (1x1) io_domain when no other io_layout is specified, complying with changing requirements for 2020 and later versions of FMS, and following the default behavior of previous versions. Also corrected three module use statements in infra/FMS1 to eliminate any dependencies of code in config_src/infra/FMS1 on code in src/framework. These use statements eventually point to the same place as before, but with less indirection. This change should facilitate later steps to compile everything in and under the config_src/infra directories as libraries. All answers are bitwise identical.

* if true, allow the topog overrides to change the land mask

Explicitly set (1x1) io_domain as a default

Added the option to use FMS2 interfaces to read variables in the FMS2 version of MOM_io_infra.F90. For now, this is controlled by a hard-coded module variable, FMS2_reads, in MOM_io.F90. Also extensively revised MOM_read_fms2.F90 to add the new public subroutine prepare_to_read_var that is used from MOM_io_infra.F90, to systematically determine the directions that axes in files using the cartesian_axis attribute before resorting to using the axis names or units. All of the subroutines from MOM_read_fms2.F90 whose functionality has been merged into MOM_io_infra.F90 have now been removed. All answers are bitwise identical and reproduce the FMS1 test cases, but there is a new publicly visible routine.

- Units of isoneutral or interface slope were recorded as "nondim". While true in SI units, not so for MOM6 units. MOM6 distinguishes between units of length in the vertical (Z) and horizontal (L) the slopes should have units of "Z L-1 ~> nondim". - This has consequences for other variables in calc_isoneutral_slopes(). - An internal constant, G_Rho0, was defined differently from elsewhere in the code. "g" has units of "L2 Z-1 T-2 ~ m s-2" because it is the vertical component of the gradient of geopotential in "L2 T-2 ~ m2 s-2". Everywhere else `G_Rho0 = g_Earth/Rho0` but in this routine it was different in order render N2 (the Brunt-Vaisala frequency) in units of "T-2" (s-2). - N2 is a quantity associated with dispersion relations and defined N2 = - g/Rho0 d/dz rho and either way acquires units of "L2 Z-2 T-2" and not just "T-2". In SI units L2 Z-2 = 1. So I have also changed the units of N2 in this, and connected, modules. - The changes also propagate to MOM_lateral_mixing_coeffs.F90 and MOM_thickness_diffuse.F90. - Changing the definition of G_Rho0 in calc_isoneutral_slopes(), and its units to "L2 Z-1 T-2", the slope and N2 calculations then require many less inline conversions. Many of the one-line changes in this commit remove factors like US%Z_to_L.There is one exception: - In the calculation of slope, we use in the denominator a mostly non-vanishing replacement for d/dz rho, the magnitude of grad rho from mag_grad2 = ( d/dx rho )^2 + ( d/dz rho )^2. In code this had `mag_grad2 = drdy**2 + (L_to_Z*drdz)**2` since this is mixing gradients in the horizontal and vertical. The result should be in "R2 Z-2" so now `mag_grad2 = (Z_to_L*drdy)**2 + drdz**2` - A few diagnostics needed new, or changed, conversion factors. - One run-time parameter needed a conversion parameter. - For the most part this commit moves inline conversions of units to the I/O stage, which is an indicator that it is the right thing to do.

Update CVMix submodule

Change units of slope returned from calc_isoneutral_slopes() to "Z L-1"

Bugfix: FGNV streamfunction vertical bounds

Modified the routines that write out the checksums of various types related to the surface ocean state so that only the root PE writes the checksums. The modified routines are ocn_ice_bnd_type_chksum and ocean_public_type_chksum in config_src/infra/FMS_cap. The analogous routines with the nuopc_cap and mct_cap have not been modified; although doing so would be a very good idea, it should be done by someone who is actively testing them. In some large-PE tests cases this reduces the volume of output to stdout by over 90% without any loss of information. All solutions are bitwise identical, but there may be minor white-space reformatting of the output in some cases.

Feature/update to gfdl 20210308

Use axistype for MOM_interp_infra directly from mpp_io_mod and add a copy of get_axis_data to both copies of infra/FMS[12]/MOM_interp_infra.F90, and then use these in framework/MOM_horizontal_regridding.F90, to permit the MOM6 I/O calls to use the FMS2 interfaces without simultaneously requiring changes to the horizontal interpolation code. All answers are bitwise identical, but there are changes to the interfaces offered by a public module.

Added a new optional logical argument to write_metadata_axis to indicate when an axis is staggered at the edges of the tracer grid, and changed calls to get_file_info to stop requesting the number of global attributes. Also eliminated some unused optional arguments to the FMS1 version of write_metadata field. All answers are bitwise identical, but there are minor changes to some I/O related interfaces.

Corrected the logic of a warning message in categorize_axis by adding parentheses. All answers are bitwise identical, and spurious warnings are no longer being issued.

Added a large number of calls to handle all of the writes via the FMS2 interfaces to infra/FMS2/MOM_io_infra.F90. There are newly defined private types in MOM_io_infra to wrap the axistype and fieldtype that had previously been offered from mpp_io_mod. All answers are bitwise identical and it has been verified that output files do not change and the restarts are still working.

Deleted the unused MOM_axis.F90 and MOM_write_field_fms2.F90 modules, now that MOM_io_infra.F90 has been updated to be able to use FMS2 interfaces for both reading and writing to files. As these modules had never been used, they do not change any answers.

dynamic ice shelf

Added variants of MOM_read_data_0d, MOM_read_data_0d_int, MOM_read_data_1d, MOM_read_data_1d_int, and MOM_read_data_2d_region that use the FMS2 interfaces to read data. Also altered prepare_to_read_var so that it does not open a file (which had been an option before); one argument to prepare_to_read_var was no longer needed and so it was removed. Also added a public interface to find_varname_in_file that does the same as prepare_to_read_var but works on the other FMS2 file type without domain decomposition. Several unused and now redundant routines were removed from MOM_read_data_fms2.F90. Comments describing a number of variables were also added. All of these changes are confined to config_src/infra/FMS2, and all answers are bitwise identical.

Only write checksums from root PE MCT/NUOPC caps

Moved the routines prepare_to_read_var and find_varname_in_file, along with four other routines that prepare_to_read_var calls from MOM_read_data_fms2 to MOM_io_infra, and eliminated the file MOM_read_data_fms2.F90. All answers are bitwise identical, but this rearrangement of identical code does eliminate one public module, so that the file structure of infra/FMS2 is identical to that of infra/FMS1.

Two minor changes to the .testing build: - We now apply `--recurse-submodules` to the target build in the regression test. This is required after an update to the submodules, when the target submodule is out of sync with the main branch (e.g. dev/gfdl at NOAA-GFDL). - A LDFLAGS_USER hook was added to the `.testing/Makefile` configuration, similar to the FCFLAGS_* hooks. This is required if the library dependencies do not reside in the default directores. For example, this may be needed for a custom netCDF or MPI library.

Testing: Recurse target submodules, LDFLAGS hook

…before getting currents in A- or B-grid staggering. Now added.

Add a missing ".nc" suffix to the output filename with FMS2_io, while also issuing a warning, following the practice of FMS1. Also reordered the calls to add the longname and axis attributes to FMS2 files, to follow the order used in MOM6 calls to FMS1. All answers are bitwise identical, but there are some changes to output filenames and orders of attributes in files (to revert to traditional behavior).

EMC update 20210322

FMS2 restart routines expect axes to be domain-decomposed. However, the domain_write_1d function does not apply this decomposition and instead routes this operation to compressed_write_1d. In order to accommodate this, we explicitly slice the 1d arrays of any axes into its domain-decomposed segment before passing to write_data. We have also introduced a control flag to MOM's FMS2 axistype to direct MOM_write_axis when this needs to be applied. We currently apply the domain decomposition flag to all horizontal axes regardless of circumstances. For now this is probably sufficient, but may need further testing (e.g. cube sphere).

Explicit domain decomposition of horizontal axes

Corrects reads of hexadecimal checksum attributes from distributed files when using the FMS2 IO interfaces. All answers are bitwise identical, and reads and writes of distributed sets of restart files are now working with the FMS2 IO interfaces.

+Enable FMS2 interface reads (includes merge of PR #1165)

Only write checksums from root PE for FMS_cap

…urrents Bugfix in getting surface currents [Additions from GMAO (3)]

Remove timelevel arguments from MOM_read_data calls for variables that should never have multiple time-levels. All answers are bitwise identical, and this will eliminate some warning messages in output.

New diagnostics to horizontal_viscosity were causing issues with stack memory on some platforms, causing the runtime to more than double. Two of the diagnostics were allocatables and the other two were local variables. By redefining the two allocatables as locals (and presumably moving to stack), the faster performance was restored. While the underlying cause is unclear, this is almost certainly due to stack spill in this function, which happens to have a large number of local arrays - including many 3d arrays used to gather diagnostics - and any new variable is going to have volatile consequences. This should be seen as a short term fix. In the future, we need better tools to detect this problem and better guidance on how to responsibly use stack. Also note that two variables were removed: `max_diss_rate_[qh]`. Neither variable was used in the function.

+Enable FMS2 interface writes

Added additional wrappers to the MOM6 framework and MOM_couplertype_infra for coupler_types routines and types that are needed by SIS2. These include support for the use of a coupler_3d_bc_type, including as overloads to the existing coupler_type_spawn, coupler_type_copy_data, coupler_type_increment_data, coupler_type_initialized and coupler_type_write_chksums. There are also new overloaded wrappers in all three files for coupler_type_redistribute_data, coupler_type_data_override, coupler_type_rescale_data. All answers are bitwise identical.

Added additional domain routine interfaces that are needed by SIS2, including the new function same_domain, which tests whether two domains use the same layout and conforming computational domain sizes, and a new 4d-array variant of redistribute_array because SIS2 uses thickness categories as a 4th dimension. All answers are bitwise identical.

Remove inappropriate timelevel arguments

- previous code had averaging instead of summation for SPP (x:sum,y:point,z:point) diagnostics - corrects an issue where these diagnostics were incorrect by approximately a factor of 2. - Orginially found when analyzing the depth-integrated temperature advection diagnostic (T_ady_2d)

MOM_hor_visc: Variables moved to stack

This patch restructures the CorAdCalc function so that the loops are more easily vectorized on a broader range of systems. The number of memory access has also been slightly reduced. We observed a 1.75x speedup on a modern consumer AMD processor (Ryzen 5 2600) and a 1.24x speedup on Gaea's Intel Xeons (E5-2697 v4). Description There are two major changes: - An if-block testing for `Area_q` was removed, and the `h_neglect * Area_q` term was replaced with a new `vol_neglect` term. This term is intended to prevent division by zero when the hArea_q is zero. Otherwise, it is meant to be below roundoff and have no impact on the calculation. Previously, a zero value of Area_q would force a division by zero. Using vol_neglect ensures that the denominator is always nonzero. The value is set to use `H_subroundoff` times an area of 0.1 mm2, suggested by Robert Hallberg as a hypothetical Kolmogorov scale. Numerical results are intended to be independent of this choice. - Two separated loops associated with the bounded Coriolis term were combined into a single loop, which reduced both the number of internal if-blocks and avoided redundant memory load/stores. Other if-blocks inside of do-loops were moved outside of the loops. I can provide two potential explanations for the difference in Intel and AMD performance: * Masking instructions have a lower latency on Intel CPUs, which permit limited vectorization of if-blocks. Similar vectorization is not possible on AMD CPUs. So Intel is less likely to benefit from if-block re-ordering. * Our Intel nodes on Gaea have a lower RAM bandwidth, and see a smaller benefit from vectorization, which must required greater bandwidth. This speedup may be greater on a more modern Intel platform. Although the code has been vectorized on both Intel and AMD platforms, there are still many memory accesses per operation, which is limiting performance. The changes below are not expected to change any answers, and none were detected. But since we are changing a core component, I'd suggest reviewing this carefully. Sample timings are provided below. Runtime measurements -------------------- AMD Before: (Ocean Coriolis & mom advection) 1.091571 (Ocean Coriolis & mom advection) 1.086183 (Ocean Coriolis & mom advection) 1.091197 (Ocean Coriolis & mom advection) 1.087709 (Ocean Coriolis & mom advection) 1.086990 AMD After: (Ocean Coriolis & mom advection) 0.619346 (Ocean Coriolis & mom advection) 0.624106 (Ocean Coriolis & mom advection) 0.625438 (Ocean Coriolis & mom advection) 0.630169 (Ocean Coriolis & mom advection) 0.621736 ---- Intel Before: (Ocean Coriolis & mom advection) 0.981367 (Ocean Coriolis & mom advection) 0.982316 (Ocean Coriolis & mom advection) 0.986633 (Ocean Coriolis & mom advection) 0.981260 (Ocean Coriolis & mom advection) 0.982810 Intel After: (Ocean Coriolis & mom advection) 0.788747 (Ocean Coriolis & mom advection) 0.797684 (Ocean Coriolis & mom advection) 0.786874 (Ocean Coriolis & mom advection) 0.792120 (Ocean Coriolis & mom advection) 0.795373

…03-26 dev/gfdl main candidate 2021-03-26

- Fixes for SSP (x:sum;y:sum,z:point) and PSP (x:point,y:sum,z:point) diagnostics - Removed unused `total_weight` arrays in these cases

Corrected comments describing the various CT_copy_data routines, following suggestions in a review by Keith Lindsay. All answers are bitwise identical.

…to SIS2_infra_support

+Add infrastructure interfaces needed by SIS2

Fixed downsampling for x:sum y:point z:point diags

Coriolis: Improved coradcalc vectorization

Added support for new IO capabilities that are needed by SIS2 to use the MOM6 framework and infrastructure code, but should also be useful within MOM6 itself. These new capabilities include writing global attributes to files, using create_file named axes that are not derived from a MOM6 grid type, and new options and elements in the vardesc type to support a wider array of axes and to provide the position of the grid staggering via an integer position variable instead of the short character strings that had been used. As a part of this commit, there are the new opaques type axis_info and attribute_info, and the new routines set_axis_info, delete_axis_info, set_attribute_info and delete_attribute_info to facilitate these new capabilities, as well as the publicly visible function position_from_horgrid to translate the vardesc%hor_grid character strings into the integer position flag used elsewhere in the MOM6 and FMS codes. Within the MOM_io_infra, there is a new variant of the overloaded interface write_meta to handle writing global attributes. There are also two new optional arguments to create_file and reopen_file, and two new optional arguments to var_desc, modify_vardesc, and query_vardesc. All answers and output are bitwise identical.

Restructured the code slightly so that the output files that are generated when input_filename = 'F' is exactly the same as if it is 'n' if there are no restart files in the restart_input_dir, or as if it is 'r' if the restart files are there. Previously, the solutions with 'F' worked this way, but no ocean_geometry.nc or Vertical_grid.nc files were written when WRITE_GEOM=1, regardless of the presence or absence of the restart files, and the MOM_parameter_doc.all files differed slightly between the 'n' and 'F' or 'r' cases. As a part of these changes, the determination of whether this is a new run is moved earlier in the algorithm, and now sits outside of MOM_initialize_state. All solutions are bitwise identical, but there are changes in the position of the PARALLEL_RESTARTFILES and REFERENCE_HEIGHT entries in most MOM_parameter_doc.all files.

+Enhanced support for novel axes in MOM_io

Change the types returned from the 5 rotated_field_chksum from integer to integer(kind=int64), so that the full 64-byte checksums are returned. Without this change, the checksums that are written to MOM6 restart files or interpreted from them are truncated to the latter half of their length. This changes the checksums that are written to the restart files, but both before and after this change the values that are written are the same as those that are generated after reading the restart with the same version of the code. The code can run across this change by setting RESTART_CHECKSUMS_REQUIRED = False for the run segment where the transition occurs. The solutions themselves are bitwise identical.

Changed the declarations of the vardesc and fields arrays to allocatable in write_ocean_geometry_files, primarily to get one of the TC test cases to run properly with the gcc compiler by shifting the memory for these arrays from stack to heap. The reason why this change works is not clear. Some comments describing these variables were also added. All answers are bitwise identical.

…e_file_output

+More consistent treatment of input_filename = 'F'

(*)Write full checksums to restarts

Commits on Jul 17, 2020

Create hola_tierra.yml

wrongkindofdoctor authored Jul 17, 2020

Configuration menu

View commit details

Copy full SHA for cbdcf8a

Browse repository at this point

Copy the full SHA

cbdcf8a View commit details

Browse the repository at this point in the history

Commits on Feb 10, 2021

fixed ice-shelf advection

OlgaSergienko committed Feb 10, 2021

Configuration menu

View commit details

Copy full SHA for 7752052

Browse repository at this point

Copy the full SHA

7752052 View commit details

Browse the repository at this point in the history

Commits on Apr 5, 2021

Merge branch 'main' into main_to_gfdl

marshallward committed Apr 5, 2021

Configuration menu

View commit details

Copy full SHA for fae570d

Browse repository at this point

Copy the full SHA

fae570d View commit details

Browse the repository at this point in the history

Updates on 04_21 #2

Updates on 04_21 #2

Commits on Feb 5, 2019

Commits on Feb 27, 2019

Commits on Mar 21, 2019

Commits on Mar 28, 2019

Commits on Apr 8, 2019

Commits on Apr 15, 2019

Commits on Apr 18, 2019

Commits on Apr 25, 2019

Commits on Apr 26, 2019

Commits on May 9, 2019

Commits on May 17, 2019

Commits on May 24, 2019

Commits on Jun 5, 2019

Commits on Jun 20, 2019

Commits on Jul 3, 2019

Commits on Jul 23, 2019

Commits on Jul 29, 2019

Commits on Jul 31, 2019

Commits on Aug 12, 2019

Commits on Aug 28, 2019

Commits on Sep 26, 2019

Commits on Oct 2, 2019

Commits on Oct 4, 2019

Commits on Oct 22, 2019

Commits on Nov 12, 2019

Commits on Nov 19, 2019

Commits on Nov 26, 2019

Commits on Nov 28, 2019

Commits on Dec 2, 2019

Commits on Dec 3, 2019

Commits on Dec 6, 2019

Commits on Dec 9, 2019

Commits on Dec 17, 2019

Commits on Jan 6, 2020

Commits on Jan 13, 2020

Commits on Jan 16, 2020

Commits on Jan 17, 2020

Commits on Jan 24, 2020

Commits on Feb 3, 2020

Commits on Feb 10, 2020

Commits on Feb 14, 2020

Commits on Feb 26, 2020

Commits on Mar 9, 2020

Commits on Mar 16, 2020

Commits on Mar 30, 2020

Commits on Apr 6, 2020

Commits on Apr 8, 2020

Commits on Apr 9, 2020

Commits on Apr 16, 2020

Commits on Apr 20, 2020

Commits on May 1, 2020

Commits on May 7, 2020

Commits on May 26, 2020

Commits on Jun 5, 2020

Commits on Jun 9, 2020

Commits on Jul 7, 2020

Commits on Jul 10, 2020

Commits on Jul 13, 2020

Commits on Jul 17, 2020

Commits on Jul 22, 2020

Commits on Jul 28, 2020

Commits on Jul 31, 2020

Commits on Aug 1, 2020

Commits on Aug 6, 2020

Commits on Aug 7, 2020

Commits on Aug 17, 2020

Commits on Aug 24, 2020

Commits on Sep 1, 2020

Commits on Sep 9, 2020

Commits on Oct 19, 2020

Commits on Nov 25, 2020

Commits on Dec 16, 2020

Commits on Dec 23, 2020

Commits on Dec 29, 2020

Commits on Jan 6, 2021

Commits on Jan 15, 2021

Commits on Jan 17, 2021

Commits on Jan 21, 2021