Skip to content

SCHISM BMI Documentation and Build

Jump to bottom
Phil Miller - NOAA edited this page Nov 17, 2023 · 9 revisions

Summary

This wiki page here illustrates the SCHISM model and it's BMI dependencies for compilation, step by step instructions for compiling the BMI shared libraries and building the BMI driver unit test for a given SCHISM model setup to test out BMI functionality and model workflow within a theoretical NextGen formulation. Users are advised that this documentation is preliminary and coastal model evaluation as a NextGen formulation within the model engine is underway and initial BMI development stages have been confined to utilizing a SCHISM model setup on only a single thread (serial implementation). Future initiatives will expand BMI capabilities to deal with an MPI communicator long-term to run the SCHISM model on larger meshes.

Dependencies to build SCHISM model and BMI

  1. cmake/3.20.1
  2. intel/2022.3.0
  3. impi/2022.3.0
  4. netcdf-hdf5parallel/4.7.4
  • export FC=mpiifort
  • export CXX=mpiicpc
  • export CC=mpiicc

Top directory structure

  • src/ = SCHISM model engine source code
  • cmake/ = Cmake build setups to configure SCHISM model setup settings
  • src/CMakeLists.txt = SCHISM model CMakeLists.txt file that directly controls the SCHISM model build and compilation based on user environment
  • SCHISM_BMI/ = Directory where BMI source code, BMI driver Cmake build, and BMI driver source code is located for current SCHISM BMI build.

BMI directory structure

  • BMI/ = Directory where BMI shared library source code is located
  • src/ = Directory where BMI driver source code is located
  • SCHISM_LIB_NWM_BMI = Directory where the user will place SCHISM model engine compiled libraries and modules required to link and build BMI shared libraries and driver
  • module = Directory of Cmake modules that are used to find and compile libraries and packages required for SCHISM and it's BMI shared libraires.
  • iso_c_fortran_bmi tar gz file = Directory where BMI Fortran source code is linked for compiling C+ pointer functions within BMI shared library.
  • CMakeLists.txt, namelist.input = Files for compiling SCHISM BMI shared libraries and driver using CMake and the required "BMI configuration" file for DFLOWFM respectively.
  • ParMetis, Metis tar gz files = Tar zipped files that are required to be unzipped and available SCHISM libraries to build and link with the BMI shared libraries during its compilation.

Build the SCHISM model initial modules and libraries to link to BMI based on supercomputer dependencies (example is for RDHPCS Hera cluster)

  1. cd schism_NWM_BMI/cmake

  2. Insert/replace the following three lines in the file: "SCHISM.local.build"

  set (OLDIO ON CACHE BOOLEAN "Old nc output (each rank dumps its own data)")
  set (USE_ATMOS ON CACHE BOOLEAN "Coupling with atmospheric model via ESMF")
  set (USE_NWM_BMI ON CACHE BOOLEAN "Use NWM BMI for source and some b.c.")
  1. Insert the following line within CMakeLists.txt in ../src directory to enable BMI compliance for SCHISM
  add_compile_options("-fPIC")
  1. Modify the following lines within CMakeLists.txt in ../src directory to enable BMI compliance for SCHISM
  define_opt(OLDIO "Old nc output option" ON)
  define_opt(USE_ATMOS "Atmospheric model output option" ON)
  define_opt(USE_NWM_BMI "Use NWM BMI for source and some b.c." ON)
  1. To build SCHISM system using cmake version 3.12 or higher:
  mkdir ../build
  cd ../build; rm -rf *
  cmake -C ../cmake/SCHISM.local.build
  make VERBOSE=1 pschism
  1. Copy over include and lib SCHISM libraries/modules directory to the SCHISM BMI directory where we will link these shared libraries respectively
  scp -r lib ../SCHISM_BMI/SCHISM_LIB_NWM_BMI
  scp -r include ../SCHISM_BMI/SCHISM_LIB_NWM_BMI

Compile SCHISM BMI shared libraries and BMI driver that will allow the user to test the SCHISM BMI functionality for a given simulation

  1. cd ../SCHISM_BMI/

***note that small tweaks to CMakeLists.txt file may be warranted to configure to your environmental libraries and modules

  1. tar -xvf ParMetis-4.0.3.tar.gz ; tar -xvf iso_c_fortran_bmi.tar.gz ; tar -xvf metis-5.1.0.tar.gz

  2. mkdir build ; cd build

  3. cmake ../

  4. cmake --build . --target schism_driver

  5. Once these steps are completed, you should see that a schism_driver executable has been created. This is linked to the SCHISM BMI shared libraries and should allow you to evaluate a SCHISM model using BMI functionality.

  6. Copy over the schism_driver within the "build" directory you've created to your SCHISM model setup. Then, copy over the file within ./SCHISM_BMI/namelist.input to your SCHISM model setup. The namelist.input is basically the BMI configuration file required to specify SCHISM model dependencies need to execute its BMI shared libraries and a linked BMI driver.

  7. Within your SCHISM model setup directory, you execute the SCHISM BMI model with the following command line that essentially links the Intel MPI capabilities to the BMI driver executable: mpirun -np 1 schism_driver namelist.input

Clone this wiki locally