Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

skip mode: consider using special "skip" task messages #6380

Open
oliver-sanders opened this issue Sep 24, 2024 · 1 comment
Open

skip mode: consider using special "skip" task messages #6380

oliver-sanders opened this issue Sep 24, 2024 · 1 comment
Labels
Milestone

Comments

@oliver-sanders
Copy link
Member

Extension to #6039 and https://github.com/cylc/cylc-admin/blob/master/docs/proposal-skip-mode.md#proposal-skip-run-mode-for-tasks

Note: This applies to all "non-live" run modes (i.e. dummy, skip and simulation), for the purpose of this issue I've focussed on skip.

Task outputs in the DB

When tasks are run, we store any messages and outputs they generated into the task_outputs table in the database.

E.G:

[scheduling]
    [[graph]]
        R1 = foo

[runtime]
    [[foo]]
        script = cylc message -- my-message
        [[[outputs]]]
            my-trigger = my-message
        [[[skip]]]
            outputs = my-trigger
$ cylc vip -N .
$ sqlite3 ~/cylc-run/tmp.PQ7h6sdpgP/runN/.service/db 'SELECT * FROM task_outputs;'
1|foo|[1]|{"submitted": "submitted", "started": "started", "succeeded": "succeeded", "my-trigger": "my-message"}

The custom trigger "my-trigger" gets stored along with the "my-message" task message that generated it.

Skip mode messages

Skip mode is no different, we still have to store the task outputs, however, the task messages don't make much sense here as they were internally generated.

E.G. The message "my-message" shouldn't really be in the DB if the task was run in skip mode because it doesn't make sense.

Skip mode prerequisites

In the proposal we agreed on this:

When tasks are run in skip mode, the prerequisites which correspond to the outputs they generate should be marked as satisfied by skip mode rather than satisfied naturally for provenance reasons.

For the purpose of cylc remove logic, satisfied by skip mode should be treated the same as satisfied naturally.

Proposal / Question

Extend this prerequisite logic to cover task messages.

I.E. The task message for any output satisfied by skip mode should be recorded as "satisfied by skip mode".

$ cylc vip -N .
$ sqlite3 ~/cylc-run/tmp.PQ7h6sdpgP/runN/.service/db 'SELECT * FROM task_outputs;'
1|foo|[1]|{"submitted": "satisfied by skip mode", "started": "satisfied by skip mode", "satisfied by skip mode": "succeeded", "my-trigger": "satisfied by skip mode"}

This would also extend to the other non-live modes (dummy & simulation).

This change would be visible in the GUI:

Screenshot from 2024-09-24 16-08-39

@oliver-sanders oliver-sanders added small question Flag this as a question for the next Cylc project meeting. labels Sep 24, 2024
@oliver-sanders oliver-sanders added this to the 8.x milestone Sep 24, 2024
@oliver-sanders
Copy link
Member Author

Meeting 2024-11-04

Agreed, removing the question label.

@oliver-sanders oliver-sanders removed the question Flag this as a question for the next Cylc project meeting. label Nov 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant