Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Run Bronx in Canopy #156

Open
3 of 6 tasks
cwhitlock-NOAA opened this issue Aug 12, 2024 · 2 comments
Open
3 of 6 tasks

Run Bronx in Canopy #156

cwhitlock-NOAA opened this issue Aug 12, 2024 · 2 comments
Assignees

Comments

@cwhitlock-NOAA
Copy link
Collaborator

cwhitlock-NOAA commented Aug 12, 2024

  • Is your feature request related to a problem? Please describe.
    For production work, scientists would like to be able to run bronx XMLs (and yamls, now that we can make them) in the new canopy workflow. If we combine together the new yaml structure/command grouping and the-tool-currently-known-as-fre-pp-wrapper, we can get most of the way there.

Describe the solution you'd like

This divides into 6 (maybe 5) tasks:

  • Modify the pp starter for freVersion != Bronx
  • Remove custom platform csh from the xmls
  • Keep the am5 yaml include and the attached directory structure
  • Main yaml has 3 sections
  • configure_yaml now loads 2 yamls
  • offline conversion of xml to yamls in a separate tool

  1. Modify the pp starter such that if the freVersion is not bronx, then do some different stuff in pp.starter:
    Swap out "-x am5.xml" with "-c $(includeDir)/am5.yaml"

Do not retrieve the history and ptmp directories using frelist, and do not stage the history files to PTMP.

  1. Custom platform csh is not needed in the XMLs any longer.
    User specify that they want to use canopy in the section as is allowed today:
canopy
  1. FRE Bronx will copy over all the files in $(includeDir), so we'll keep the am5.yaml and the yaml_include files underneath it. e.g.
  • am5.xml
  • xml_include/am5.yaml
  • xml_include/yaml_include/pp.c96.yaml
  • xml_include/yaml_include/pp.c192.yaml
  1. main yaml has 3 sections going forward:
  • properties: user-defined variables that get expanded in directories and experiments sections
  • directories: output from pp
  • experiments: a set of keyed pairs that contain 2 things: a name (experiment name) and an experiment yaml
  1. configure_yaml needs to load 2 yamls - the main and an experiment yaml that is referred to in the main yaml

  2. We need an offline tool for converting XML -> YAML but there's not much urgency on that - for now, we can hand-edit and check to make sure XML and YAML properties match.

Additional context
This is modified from an email chain sent out on 2022-07-08

@cwhitlock-NOAA
Copy link
Collaborator Author

@ceblanton
Copy link
Collaborator

save for 2025.01

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants