Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update NetCDF comp/link #336

Merged
merged 7 commits into from
Mar 25, 2021
Merged

Update NetCDF comp/link #336

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
e3d24a5
updates for NetCDF comp/linking Issue #332
JessicaMeixner-NOAA Mar 16, 2021
4cf48e7
Updating matrix_ncep to use hpc-stack modules
JessicaMeixner-NOAA Mar 18, 2021
17e3295
update path for orion hpc-stack and change np, nr, nth
JessicaMeixner-NOAA Mar 18, 2021
868ca8c
add module purge and revert 8->24
JessicaMeixner-NOAA Mar 19, 2021
bfc1f03
Merge remote-tracking branch 'EMC/develop' into NetCDFcomplink
JessicaMeixner-NOAA Mar 19, 2021
3e4f764
update NetCDF in ww3_tp2.14 oasis test makefile
JessicaMeixner-NOAA Mar 24, 2021
8b1d62e
Merge remote-tracking branch 'EMC/develop' into NetCDFcomplink
JessicaMeixner-NOAA Mar 24, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion model/bin/comp.tmpl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,8 @@
case $WWATCH3_NETCDF in
NC3) opt="$opt -I$NETCDF_INCDIR" ;;
NC4) if [ "$mpi_mod" = 'no' ]; then comp="`$NETCDF_CONFIG --fc`"; fi
opt="$opt -I`$NETCDF_CONFIG --includedir`" ;;
#opt="$opt -I`$NETCDF_CONFIG --includedir`" ;;
opt="$opt `$NETCDF_CONFIG --cflags`" ;;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

--cflags is for c-compiler
for fortran-compiler, it must be --fflags to be used

esac
fi

Expand Down
2 changes: 1 addition & 1 deletion model/bin/link.tmpl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -116,7 +116,7 @@
case $WWATCH3_NETCDF in
NC3) libs="$libs -L$NETCDF_LIBDIR -lnetcdf" ;;
NC4) if [ "$mpi_mod" = 'no' ]; then comp="`$NETCDF_CONFIG --fc`"; fi
libs="$libs `$NETCDF_CONFIG --flibs`" ;;
libs="$libs `$NETCDF_CONFIG --flibs` `$NETCDF_CONFIG --libs`" ;;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the netcdf-fortran library is correctly compiled, all the needed dependencies are defined in --flibs.
the option --libs does not exist for all netcdf libraries (like version 4.5.2) so it crashes.

esac
fi

Expand Down
128 changes: 70 additions & 58 deletions regtests/bin/matrix_ncep
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,13 +4,14 @@
# #
# Remarks: #
# - This version is set up for automatic w3_setenv script and for the #
# NOAA RDHPC 'zeus' system. When using this for your own setup and #
# NOAA RDHPC systems. When using this for your own setup and #
# computer, please copy rather than modify. #
# #
# Hendrik L. Tolman #
# August 2013 #
# December 2013 #
# April 2018 #
# March 2020 #
# #
# Copyright 2013 National Weather Service (NWS), #
# National Oceanic and Atmospheric Administration. All rights #
Expand Down Expand Up @@ -39,70 +40,83 @@

# Set batchq queue, choose modules and other custom variables to fit system and
# to define headers etc (default to original version if empty)
ishera=`hostname | grep hfe`
isorion=`hostname | grep Orion`
if [ $ishera ]
then
# If no other h, assuming Hera
cmplr='hera.intel'
batchq="slurm"
modcomp='intel/18.0.5.274'
modmpi='impi/2018.0.4'
modnetcdf='netcdf/4.6.1'
metispath='/scratch2/COASTAL/coastal/save/Ali.Abdolali/parmetis-4.0.3'
elif [ $isorion ]
then
cmplr='orion.intel'
batchq="slurm"
modcomp='intel/2018.4'
modmpi='impi/2018.4'
modnetcdf='netcdf/4.7.2'
metispath='/work/noaa/marine/ali.abdolali/Source/parmetis-4.0.3'
else
batchq=
fi
ishera=`hostname | grep hfe`
isorion=`hostname | grep Orion`
if [ $ishera ]
then
# If no other h, assuming Hera
cmplr='hera.intel'
batchq='slurm'
hpcstackpath='/scratch2/NCEPDEV/nwprod/hpc-stack/libs/hpc-stack/modulefiles/stack'
hpcstackversion='hpc/1.1.0'
modcomp='hpc-intel/18.0.5.274'
modmpi='hpc-impi/2018.0.4'
modnetcdf='netcdf/4.7.4'
metispath='/scratch2/COASTAL/coastal/save/Ali.Abdolali/hpc-stack/parmetis-4.0.3'
elif [ $isorion ]
then
cmplr='orion.intel'
batchq='slurm'
hpcstackpath='/apps/contrib/NCEP/libs/hpc-stack/modulefiles/stack'
hpcstackversion='hpc/1.1.0'
modcomp='hpc-intel/2019.5'
modmpi='hpc-impi/2019.6'
modnetcdf='netcdf/4.7.4'
metispath='/work/noaa/marine/ali.abdolali/Source/hpc-stack/parmetis-4.0.3'
else
batchq=
fi

# 1. Set up

export np='24' #number of mpi tasks
export nr='4' #number of mpi tasks for hybrid
export nth='6' #number of threads

# 1.a Computer/ user dependent set up

echo '#!/bin/sh --login' > matrix.head
echo ' ' >> matrix.head
if [ $batchq = "slurm" ] && [ $isorion ]
then
echo '#SBATCH -n 24' >> matrix.head
echo '#SBATCH -q batch' >> matrix.head
echo '#SBATCH -t 08:00:00' >> matrix.head
echo '#SBATCH -A marine-cpu' >> matrix.head
echo '#SBATCH -J ww3_regtest' >> matrix.head
echo '#SBATCH -o matrix.out' >> matrix.head
echo '#SBATCH -p orion' >> matrix.head
elif [ $batchq = "slurm" ]
then
echo '#SBATCH -n 24' >> matrix.head
echo '#SBATCH -q batch' >> matrix.head
echo '#SBATCH -t 08:00:00' >> matrix.head
echo '#SBATCH -A marine-cpu' >> matrix.head
echo '#SBATCH -J ww3_regtest' >> matrix.head
echo '#SBATCH -o matrix.out' >> matrix.head
else
echo '#PBS -l procs=24' >> matrix.head
echo '#PBS -q batch' >> matrix.head
echo '#PBS -l walltime=08:00:00' >> matrix.head
echo '#PBS -A marine-cpu' >> matrix.head
echo '#PBS -N ww3_regtest' >> matrix.head
echo '#PBS -j oe' >> matrix.head
echo '#PBS -o matrix.out' >> matrix.head
echo ' ' >> matrix.head
fi
if [ $batchq = "slurm" ] && [ $isorion ]
then
echo "#SBATCH -n ${np}" >> matrix.head
echo '#SBATCH -q batch' >> matrix.head
echo '#SBATCH -t 08:00:00' >> matrix.head
echo '#SBATCH -A marine-cpu' >> matrix.head
echo '#SBATCH -J ww3_regtest' >> matrix.head
echo '#SBATCH -o matrix.out' >> matrix.head
echo '#SBATCH -p orion' >> matrix.head
elif [ $batchq = "slurm" ]
then
echo "#SBATCH -n ${np}" >> matrix.head
echo '#SBATCH -q batch' >> matrix.head
echo '#SBATCH -t 08:00:00' >> matrix.head
echo '#SBATCH -A marine-cpu' >> matrix.head
echo '#SBATCH -J ww3_regtest' >> matrix.head
echo '#SBATCH -o matrix.out' >> matrix.head
else
echo '#PBS -l procs=24' >> matrix.head
echo '#PBS -q batch' >> matrix.head
echo '#PBS -l walltime=08:00:00' >> matrix.head
echo '#PBS -A marine-cpu' >> matrix.head
echo '#PBS -N ww3_regtest' >> matrix.head
echo '#PBS -j oe' >> matrix.head
echo '#PBS -o matrix.out' >> matrix.head
echo ' ' >> matrix.head
fi
echo " cd $(dirname $main_dir)/regtests" >> matrix.head
echo ' ' >> matrix.head

# Netcdf and Parmetis modules & variables

echo " module load $modcomp $modmpi $modnetcdf" >> matrix.head
echo " module purge" >> matrix.head
echo " module use $hpcstackpath" >> matrix.head
echo " module load $hpcstackversion" >> matrix.head
echo " module load $modcomp" >> matrix.head
echo " module load $modmpi" >> matrix.head
echo " module load $modnetcdf" >> matrix.head
echo " export WWATCH3_NETCDF=NC4" >> matrix.head
echo " export NETCDF_CONFIG=`which nc-config`" >> matrix.head
echo " export METIS_PATH=${metispath}" >> matrix.head
echo ' export NETCDF_CONFIG=$NETCDF_ROOT/bin/nc-config' >> matrix.head
echo " export METIS_PATH=${metispath}" >> matrix.head
echo " export WW3_PARCOMPN=4" >> matrix.head
echo ' '

Expand All @@ -112,9 +126,7 @@ fi
else
export mpi='mpirun'
fi
export np='24'
export nr='4'
export nth='6'

# Compile option
if [ "$cmplOption" = 'y' ]
then
Expand Down Expand Up @@ -169,7 +181,7 @@ fi
export multi08='y' # mww3_test_08 (wind and ice tests)

# export filter='PR3 ST2 UQ'
# The filter does a set of consecutinve greps on the
# The filter does a set of consecutive greps on the
# command lines generated by filter.base with the above
# selected options.

Expand Down
4 changes: 2 additions & 2 deletions regtests/ww3_tp2.14/input/oasis3-mct/util/make_dir/cmplr.tmpl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,8 +30,8 @@ COUPLE = $(WWATCH3_DIR)/../regtests/ww3_tp2.14/input/oasis3-mct
ARCHDIR = $(WWATCH3_DIR)/../regtests/ww3_tp2.14/work_oasis3-mct
#
# NetCDF library
NETCDF_INCLUDE = $(shell $(NETCDF_CONFIG) --includedir)
NETCDF_LIBRARY = $(shell $(NETCDF_CONFIG) --flibs)
NETCDF_INCLUDE = $(shell $(NETCDF_CONFIG) --cflags)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

--cflags is for c-compiler
for fortran-compiler, it must be --fflags to be used

NETCDF_LIBRARY = $(shell $(NETCDF_CONFIG) --flibs) $(shell $(NETCDF_CONFIG) --libs)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the netcdf-fortran library is correctly compiled, all the needed dependencies are defined in --flibs.
the option --libs does not exist for all netcdf libraries (like version 4.5.2) so it crashes.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mickaelaccensi we have an issue now to make the manual specifically say nc-config consistently: #392 and any other checks to make sure things are working for everyone are welcome. Sorry this got pushed through while you were out and is now causing issues.

#
# Compilers and options
MAKE = gmake
Expand Down