Skip to content

Making Code Changes in the UFS Weather Model and Its Subcomponents

Jump to bottom
Gillian Petro edited this page Aug 9, 2024 · 4 revisions

ufs-weather-model Code Development

TABLE OF CONTENTS

Introduction

The development process for the ufs-weather-model has been structured to make the best use of the tools that Git/GitHub provides for code development. The ufs-weather-model repository maintains a main branch for development called develop. The HEAD of develop reflects the latest development changes. It points to regularly updated hashes for individual subcomponents.

Build structure and submodules

The ufs-weather-model uses a CMake build environment which parallels well with the hierarchical code structure used in the repository.

UFS Repository Structure

  • Model build files/directories:
    • CMakeLists.txt -- Top level build instructions for CMake.
    • cmake/ -- Files used by CMake in building ufs-weather-model.
  • System setup:
    • conf/ -- model specific flag and variables for multiple machine types
    • modulefiles/ -- machine specific libraries and tools required for build.
  • Testing:
    • tests/* -- Regression testing files.
  • GIT specific files:
    • .gitmodules -- Defines the submodules needed for proper build of ufs-weather-model.

UFS Submodules

Notes

  • The model and its subcomponents are housed in authoritative (official) repositories. The ufs-weather-model uses Git submodules to manage its subcomponents.
  • All the authoritative repositories are read-only for users.
  • Branches under the authoritative repositories include a main development branch (usually named develop, master, or main), production branches, and public release branches.
  • Branches under the authoritative repositories are protected.

First steps

All development on the ufs-weather-model repository MUST have a corresponding GitHub Issue created. This allows the code managers and the community to discuss the proposed development's importance and timeline as it fits best into our development cycle. It is possible to fix multiple issues with a single Pull Request (PR), which is why every PR must be linked with at least one issue

Setting up for development

Before you start

Why use a fork?

  • A fork is a copy of an original repository on your GitHub account. You manage your own fork.
  • Forks let you make changes to a project without affecting the original repository.
  • You can fetch updates from the original repository.
  • You can submit changes to the original repository with pull requests (PRs).
  • For more details, please see Working with forks.

Fork and Clone

  • Create a fork via GitHub:
  • Cloning a repository creates a copy on to your local machine. To clone your fork: 
    git clone https://github.com/<your_github_username>/ufs-weather-model
cd ufs-weather-model

Checkout and update

  • Check out the develop branch: 
    git checkout develop
    • NOTE: You can check which branch you are using at any time by typing git branch -a in your clone of the repository.
  • Retrieve submodule files: 
    git submodule update --init --recursive
    • NOTE: --init will initialize any submodule you have not already initialized.
    • NOTE: --recursive will make sure to get submodules inside the submodules listed in your .gitmodules file.

Creating branch and committing

  • Create a feature branch in which to make changes: 
    git checkout -b feature/newfeaturename
    • NOTE: We typically use feature and bugfix (i.e., feature/addfunction or bugfix/missingtags)
    • You can confirm you are using the new branch name by typing git branch in your repository clone.
  • Make your change or edit or fix.
  • After each change or edit or fix: 
    git add <filename(s)>
git commit -m "Type an explanation of the change or edit or fix here"
    • NOTE: Committing after each change or edit or fix gives the code maintainers a very detailed understanding of what you're doing after you submit your PR. Commits are local to your machine until pushed to a repository.

Building with CMake

Why CMake?

  • Cross-platform support
  • Easy to port to newer systems
  • Less maintenance (number of makefiles), etc.
  • Integrated testing via ctest
  • Parallel builds
  • Alternative builds to GNUmake (e.g. Ninja)
  • Easy to deploy (and find) packages
  • Provides compile time characteristics (e.g., compile definitions) transitively

UFS CMake Structure

  • CMake is a hierarchical, structured build system
  • Top-level CMakeLists.txt:
    • Contains all the instructions for the project and where to find each subcomponent
    • Typically treated as a default setup; edit build options externally
  • cmake/ directory
    • Contains all compiler-specific flags and definitions for the project
  • Each subcomponent has its own CMakeLists.txt specific to its own build and a cmake/ directory containing its own compiler-specific flags and definitions

Building UFS with CMake

  • Model Prerequisites: 
    MACHINE_ID=<machine>
source ./module-setup.sh
module use $PWD/modulefiles
module load ufs_<machine>.<compiler> (i.e. ufs_orion.intel, ufs_hera.intel, ufs_hera.gnu)
  • Setting flags:
    • Export before the call to build.sh:
    export CMAKE_FLAGS="-DAPP=S2SW"
export CCPP_SUITES="FV3_GFS_v17_coupled_p8"
./build.sh
    • OR on the same line as build.sh without export call:
    CMAKE_FLAGS="-DAPP=S2SW=" CCPP_SUITES="FV3_GFS_v17_coupled_p8" ./build.sh

*If you edit source code files, just run the same command again. CMake does not need to be cleaned (as much).

Adding a New Subcomponent

  • Gather list of files (<component>_files.cmake)
    • Keep file chunks separated if they need different compile flags/definitions
  • Create CmakeLists.txt
    • Add compiler-specific flags (cmake/)
  • Test build
  • Edit top-level CMakeLists.txt

Developing in subcomponents

When doing development work in subcomponents, you need to create a fork of the subcomponent in your account, clone it, and commit and push changes in the same way you would for the WM repo. The steps that follow pertain to what needs to be done inside your forked ufs-weather-model repository to bring in the changes you have made to subcomponents.

Bringing in your changes

  • If you have not yet cloned your fork locally using git clone:
    • Go to your fork in GitHub and edit your .gitmodules file.
    • If you made changes in fv3atm that you want to bring in to the ufs-weather-model, you would edit: 
      [submodule "FV3"]
path = FV3
url = https://github.com/NOAA-EMC/fv3atm
branch = develop
    • such that you comment out the url and branch lines and add the appropriate information from your personal fork of fv3atm to look like this: 
      [submodule "FV3"]
path = FV3
#url = https://github.com/NOAA-EMC/fv3atm
#branch = develop
url = https://github.com/<your_github_username>/fv3atm
branch = <your branch name>
    • then clone, checkout, and submodule update as previously instructed.
  • If you have already used clone and submodule update locally:
    • You should update your .gitmodules file locally with the same changes indicated above:
    • cd into your FV3 directory: cd FV3
    • update your remotes 
      git remote rename origin upstream
git remote add origin https://github.com/<your_github_username>/fv3atm
    • checkout your branch 
      git fetch origin
git checkout origin/<your_branch_name>
      • NOTE: You can verify your branch by typing in git branch.

Regression Testing

  • Regression testing is your way to prove to us that the changes you are proposing through a PR do not change the build or operation of the model in a negative way. When development work is done, users need to sync their feature branch with the latest develop/master branch in the official repository and run the regression tests: 
    cd tests
./rt.sh -e -a <account>
  • For an in-depth tutorial on Regression Testing please see the Regression Testing section of the wiki.

Adding a new test in rt.conf

rt.conf contains test cases with operational or pre-operational model configurations. All the tests in rt.conf will be tested in every code update. Below are the requirements to add a feature test in rt.conf:

  • The feature is based on the latest version of the operational model and is targeting future operational model implementation.
  • The test case has been tested successfully on the platforms developers have access to. Details on how to conduct the test and testing results should be provided in the issue or PR.
  • The test case has been tested successfully on at least one of the operational platforms and at least one NOAA R&D machine. Developers can contact UFS code managers if they don't have access to the operational or NOAA R&D platforms.

Pushing code and PRs

Once all of your commits have been completed and you have passed all the regression tests (if applicable) you will need to push those changes to your fork on GitHub inside the branch you created

Setting up remotes

  1. Check to see if you already have an origin and upstream remote location set up 
    git remote -v
    • You should see something like the following if it is already setup correctly:
    upstream        https://github.com/ufs-community/ufs-weather-model (fetch)  
upstream        https://github.com/ufs-community/ufs-weather-model (push)  
origin  https://github.com/your_github_account/ufs-weather-model (fetch)  
origin  https://github.com/your_github_account/ufs-weather-model (push)
  2. If you are missing upstream then you need to setup your remotes to add the authoritative repository: 
    git remote add upstream https://github.com/ufs-community/ufs-weather-model
    Upstream remote may be used to sync your feature branch with the latest develop/master branch in the official repository
  3. Lastly, you should check again to make sure your remotes look like above.

Pushing Changes

  • Push all your commits to your fork. 
    git push origin feature/newfeaturename
    • NOTE: Make sure you choose the remote of origin as that is where your personal fork of the authoritative repository is.
    • NOTE: Make sure to match the branch name you created earlier (i.e. feature/newfeaturename).

Syncing latest HEAD

  1. Add a remote pointing to the authoritative repository, if you haven't done so already: 
    git remote add upstream https://github.com/ufs-community/ufs-weather-model
  2. Fetch the authoritative repository using: 
    git fetch upstream
  3. Merge the upstream/develop branch into your branch: 
    git merge upstream/develop
  4. Push back to your fork: 
    git push origin BRANCH_NAME
* where BRANCH_NAME is the name of the branch that you are actively working
  5. Moving forward, you can use: 
    git remote update

to update the upstream branches and resync.

Creating GitHub Issues for UFS

  • All the development work needs to have an issue associated with it.
  • Two categories:
    • Please fill up all the information in the issue template
    • Bug report
      • Please include a detailed description and explain why it is a bug
      • List all the steps to reproduce the case
      • If a solution is provided, please show some results (plots, etc.) and create a branch to show the code changes
    • Feature request:
      • Please provide a clear and concise description of the feature
      • Please indicate the benefits of the new feature and make suggestions on who may work on it
  • NO emails!
  • Please provide further information if requested. Issues will be closed if no response is received for three months.

Pull Requests

A PR tells the code managers that your code changes from your feature branch are ready to be merged into the develop branch in the official repository. Users need to make a PR for each of the repositories that you changed in .gitmodules. You should start from the lowest subcomponent repository and then make a PR to each repository up to the ufs-weather-model application repository.

Before making a UFS PR

  • Create an issue which will be resolved by the PR. This is mandatory.
  • All interactions with code managers regarding the PR must occur on GitHub. This is mandatory.
  • The user's feature branch must be up-to-date with the the corresponding branch in the authoritative repository with all of the conflicts resolved. This is mandatory.
  • Regression tests need to pass on all supported platforms.
    • Code manager(s) will run the regression tests on the required platforms that users don’t have access to.
    • If a user needs a test run on a system they do not have access to, then please note this in the PR.
  • It is strongly encouraged that developers create a new regression test for any feature added and test with utest for reproducibility in addition to running the regression tests.

Creating a PR

  • Go to the your fork of the repository in GitHub, choose your branch, then click on “new pull request”.
  • On the next page, choose base repository and base branch. By default, PRs are based on the parent repository's default branch. Click “create pull request”.
  • Provide a Title. If the PR is for production or release branch, put the production or release branch at the beginning of the title. E.g. “Release/public-v2: update build script.”
  • Fill out the template
  • Click on the “Create Pull Request” or “Create Draft Pull Request” button.

PR Labeling

Labeling helps the code managers manage the commit queue.

  • Baseline impact : Labels of "no change" or "change"
  • Category: “bug”, “feature”, “documentation”
  • Status: Label of "draft", "ready for review", "ready for commit"
    • Draft: draft PR serves as notice that the PR will be ready within ~10 days. It allows the code managers to start planning the commit sequence for all upcoming PRs. It is the responsibility of the PR author to continue to merge all commits made to develop while in Draft. It is suggested to add ‘skip-ci” in commit message to skip CI test for draft PR
    • Ready for review: The PR is open and is considered complete by the author. It is current with the develop branch for all components which are not specifically relevant to the PR. Review may lead to additional changes; therefore, baselines tests have not yet been run.
    • Ready for commit: The PR is has been reviewed and changes, if needed, have been made and committed. Baselines are ready to be run.

Code review

  • Reviews allow collaborators to comment on the changes proposed in pull requests, approve the changes, or request further changes before the pull request is merged.
  • Anyone with read access can review and comment on the changes it proposes.
  • Developers are expected to address the comments made by the reviewers.
  • Reviewers are expected to review the code promptly so that the commit will not be delayed.

Code Commit

  • After the code changes are reviewed and approved by at least one code reviewer, code can be merged.
  • It is recommended that code managers also review and approve the code changes before the code is merged to the develop branch.
  • A commit queue is available at:
  • Developers will be contacted by the code managers when it is their turn to commit the code.

Specifics for NOAA staff

Setting up a Github account

Questions/suggestions

Please create an Issue if you have any questions/suggestions.

Documentation

ReadTheDocs

Home

Getting Started

  1. Download Prerequisite Libraries
  2. Download the Code
  3. Download Data
  4. Run the Test Case

UFS WM User's Guide

Developer Information

UFS

Applications (include workflow)

Models

Utilities

  • spack-stack - builds prerequisite software stack for the UFS WM, including NCEPLIBS and NCEPLIBS-external
Clone this wiki locally