Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

interventions: setting off-flow prereqs #745

Open
oliver-sanders opened this issue Jun 19, 2024 · 0 comments
Open

interventions: setting off-flow prereqs #745

oliver-sanders opened this issue Jun 19, 2024 · 0 comments
Labels
content Addition or modification of documentation
Milestone

Comments

@oliver-sanders
Copy link
Member

We have documented how to re-run a chain of tasks using --flow=new.

https://cylc.github.io/cylc-doc/stable/html/user-guide/interventions/index.html#re-run-multiple-tasks

This example is great, but it doesn't show you what to do if your sub-graph ain't a chain.


Context...

For example, take this workflow:

[scheduler]
    allow implicit tasks = True

[scheduling]
    [[graph]]
        R1 = """
            a => b => c => d
            x => b
        """

[runtime]
    [[d]]
        script = false

If we run this workflow, it will get to task "d", which will fail causing the workflow to stall, leaving this message in the log:

ERROR - Incomplete tasks:
* 1/d did not complete required outputs: ['succeeded']

Let's say we need to make a change to the task "a" and re-run everything after it to recover. We would trigger "a" in a new flow like so:

$ cylc trigger //1/a --flow=new

This starts a new flow, however, the workflow quickly stalls again because the task "1/x" hasn't run in the new flow, resulting in this message in the workflow log:

WARNING - Partially satisfied prerequisites:
* 1/b is waiting on ['1/x:succeeded']

To solve this, we tell Cylc that "1/x" has already succeeded, effectively telling Cylc to pull in this task from the previous flow:

$ cylc set-outputs //1/x

After this, the workflow runs on as expected:

INFO - [1/x waiting(runahead) job:00 flows:none] Forced spawning on succeeded
INFO - [1/b waiting job:01 flows:2] => waiting(queued)

It's the "set-outputs" (now just "set") part of this process which isn't presently explained.

Suggest updating the example, or adding a new one, which shows a graph like the one above and explains that the "off-flow prerequisite" either needs to be re-run (trigger) or ignored (set) and shows how to do it.

@oliver-sanders oliver-sanders added the content Addition or modification of documentation label Jun 19, 2024
@oliver-sanders oliver-sanders added this to the 8.3.x milestone Jun 19, 2024
@oliver-sanders oliver-sanders changed the title interventions: setting prerequisites for new flow interventions: setting off-flow prereqs Jun 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
content Addition or modification of documentation
Projects
None yet
Development

No branches or pull requests

1 participant