-
Notifications
You must be signed in to change notification settings - Fork 1.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
repro: downstream: reproducing in wrong order #3602
Comments
Can be reproduced with def test_repro_downstream(dvc):
assert (
main(
[
"run",
"-o",
"A",
"-o",
"B",
"-f",
"step1.dvc",
"echo 'A'>A; echo 'B'>B",
]
)
== 0
)
assert (
main(["run", "-d", "A", "-o", "C", "-f", "step2.dvc", "echo 'C'>C"])
== 0
)
assert (
main(
[
"run",
"-d",
"C",
"-d",
"B",
"-o",
"D",
"-f",
"step3.dvc",
"echo 'D'>D",
]
)
== 0
)
evaluation = dvc.reproduce("step1.dvc", downstream=True, force=True)
assert len(evaluation) == 3
assert evaluation[0].relpath == "step1.dvc"
assert evaluation[1].relpath == "step2.dvc"
assert evaluation[2].relpath == "step3.dvc" |
Looks like the issue is that for networkx I think what we should be doing is traversing the reversed graph w/reverse post-ordered search, to ensure that dependencies are run in the correct order. |
Also reproducible using our existing Line 1584 in fefbcc0
For the graph
downstream A is run as |
* tests: ensure repro --downstream preserves dependency order * repo: use reverse post-order DFS in repro --downstream - Fixes #3602 * make test_repro and test_repro_multistage consistent * rebase master, update multistage test
DVC version : 0.92.0 (PyPI) + MacOS
It seems
dvc repro --downstream
(could also be the-f
option) sometimes executes DAGs in the wrong order.Steps to reproduction:
Last bit does :
Even though the DAG looks like this :
In this case step2 (
touch 3
) should be executed before step3 (touch 4
)The text was updated successfully, but these errors were encountered: