-
Notifications
You must be signed in to change notification settings - Fork 14.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[AIRFLOW-6043] Update dag reference to all tasks in sub_dag #6638
Conversation
Graph UI is broken when tasks are filtered by root if root is in the middle of chain of tasks. The initial of the problem is in `sub_dag` method that keeps references to the original DAG. `deepcopy` call is optimized and the optimization is not 100% correct.
Codecov Report
@@ Coverage Diff @@
## master #6638 +/- ##
==========================================
- Coverage 83.71% 83.46% -0.25%
==========================================
Files 672 672
Lines 37568 37566 -2
==========================================
- Hits 31450 31355 -95
- Misses 6118 6211 +93
Continue to review full report at Codecov.
|
t1 >> t2 | ||
t2 >> t3 | ||
|
||
sub_dag = dag.sub_dag('t2', include_upstream=True, include_downstream=False) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Does this make sense? This sub_dag
is an odd beast that is similarly named but subtly different to an actual subdag via the SubDag operator.
When is this code path exercised?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This code is not related to SubDag operator at all and is about completely different thing. Naming is very confusing, true.
This sub_dag
method is called from
https://github.com/apache/airflow/blob/master/airflow/www/views.py#L1349 when Graph UI is filtered by root. It takes some mental effort to understand what happens in def graph
to render corresponding view but the logic there is fine. The problem is in the incorrect deep_copy optimizations in sub_dag
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
And when is "filter by root" passed in from the UI?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is very useful for big DAGs.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I NEVER KNEW THAT FEATURE EXISTED!!!
…ion of dag (#6638) Graph UI is broken when tasks are filtered by root if root is in the middle of chain of tasks. The initial of the problem is in `sub_dag` method that keeps references to the original DAG. `deepcopy` call is optimized and the optimization is not 100% correct. (cherry picked from commit 6fbe683)
…ion of dag (#6638) Graph UI is broken when tasks are filtered by root if root is in the middle of chain of tasks. The initial of the problem is in `sub_dag` method that keeps references to the original DAG. `deepcopy` call is optimized and the optimization is not 100% correct. (cherry picked from commit 6fbe683)
…ion of dag (#6638) Graph UI is broken when tasks are filtered by root if root is in the middle of chain of tasks. The initial of the problem is in `sub_dag` method that keeps references to the original DAG. `deepcopy` call is optimized and the optimization is not 100% correct. (cherry picked from commit 6fbe683)
…ion of dag (apache#6638) Graph UI is broken when tasks are filtered by root if root is in the middle of chain of tasks. The initial of the problem is in `sub_dag` method that keeps references to the original DAG. `deepcopy` call is optimized and the optimization is not 100% correct.
Graph UI is broken when tasks are filtered by root if
root is in the middle of chain of tasks.
The origin of the problem is in
sub_dag
method that keeps referencesto the original DAG.
deepcopy
call is optimized and the optimization isnot 100% correct.
Make sure you have checked all steps below.
Jira
Description
Original issue:
Tests
Commits
Documentation