-
Notifications
You must be signed in to change notification settings - Fork 14.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Airflow 2.7.3: Mapped task group no longer works as expected #35541
Comments
Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval. |
Hello @ephraimbuddy, @uranusjr, and @eladkal , I'm pining you as you have contributed on this PR and hopefully that you have an idea about my issue 😄 thank you. |
I saw this behaviour change and mentioned it here but I guess, I acted fast in merging(though waited for some weeks). The issue is that the task has to wait for all the upstream tasks to be done before proceeding. Now, when proceeding, this check will be true as one is skipped and the ti would be marked as skipped. |
I also mentioned in that commit that the fix may not work right either though. I feel we should revert the PR entirely and investigate a correct fix altogether. |
Can we set a unit test to catch the problematic case thus when we try new approach we would be able to see if it passes or not? |
I believe now we have test so we can re-do attempt to fix it? |
Not sure :). Have not looked at it (yet). |
Apache Airflow version
2.7.3
What happened
Hello folks,
I have a use case that doesn't work anymore after the release of the new version 2.7.3.
I have done some investigation and found that is related to this PR as I have reverted this locally and run some tests it worked as expected.
Please find below how to reproduce the issue.
Cc @ephraimbuddy
Thanks.
What you think should happen instead
No response
How to reproduce
Below is the DAG code:
Running this DAG with the 2.7.3 version, will give you the result below:
As you can see, the status of the last task is skipped and the map index 0 and 1 was not launched.
You will obtain the same issue with AirflowFailException instead of AirflowSkipException and the status of the last task will be failed.
In the other hand, if you run the same DAG with the 2.7.2 version (or without the changes in the provided PR above), you will get the result below which is the correct one:
Operating System
linux
Versions of Apache Airflow Providers
No response
Deployment
Docker-Compose
Deployment details
Airflow Breeze
Anything else
No response
Are you willing to submit PR?
Code of Conduct
The text was updated successfully, but these errors were encountered: