Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Wrapper logic: fre pp validate #175

Open
2 tasks
cwhitlock-NOAA opened this issue Sep 10, 2024 · 0 comments
Open
2 tasks

Wrapper logic: fre pp validate #175

cwhitlock-NOAA opened this issue Sep 10, 2024 · 0 comments
Assignees

Comments

@cwhitlock-NOAA
Copy link
Collaborator

Preamble:
The thing-currently-named-wrapper (wrapper.py) works for testing fre-cli, when we have all history files already available at the time of running. However, when we get to running production jobs, the models will be sending over bundles of history files and post-processing them in parallel. This breaks some of the logic currently in the wrapper flow - in particular, the assumptions that there's not already a pre-existing experiment belonging to the same user that the current set of history files is being added to, and the assumption that there's not already an experiment with that name running.

The logic we need is encapsulated in a flowchart at the end of this issue; this breaks it down by tool.

The tool:
Fre pp validate does not have any logic for preexisting experiments. We need to add a new logic branch that checks to see that the config (pp.yaml) matches between the prior and current experiments and updates the experiment's pp.yaml if they do not. This is the behavior that we want to have when a user discovers partway through an experiment that the config was wrong but only for parts of the experiment that have not yet run; it should come with loud warning messages about reproducing past results.

fre pp validate:
[ ] Is the pp.yaml for your prior experiment up-to-date with the current configuration?

  • corresponding test: not much more than a diff command
    [ ] If not, update pp.yaml for the prior experiment (and save off the old config for reference)
  • corresponding test: pretty simple, probably involves copying around a file with the "wrong" config

image

@cwhitlock-NOAA cwhitlock-NOAA self-assigned this Sep 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant