-
Notifications
You must be signed in to change notification settings - Fork 14.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Scheduler crash when clear a previous run of a normal task that is now a mapped task #27478
Comments
I'm not able to reproduce this. Can you explain the reproduction steps? And it looks you have two issues here, 1) crashes when clearing a mapped task that was previously a normal task. 2) Issue with logs. I suggest you separate the issues so we can attend to them separately. In my case, the normal task was marked as removed which is expected. I don't think you can see the log again once marked as removed unless you have the log url |
This shouldn’t happen since TaskFail has CASCADE set on TaskInstance. Can you check your database to see if ON DELETE is set appropriately? |
look like it's related to this -> #27834 |
The linked PR allow the scheduler to rescure itself out of the faulty state, but how it ended up in that state is still not clear. |
Since we can't reproduce it and no further information provided I'm going to close this. |
problem is still present in 2.5.1 @eladkal did you try to reproduce ? recording of problem -> crash.airflow.webm |
Apache Airflow version
2.4.2
What happened
I have clear a task A that was a normal task but that is now a mapped task
What you think should happen instead
Airflow should evaluate the existing and previous runs as mapped task of 1 task
cause I can't see the logs anymore of a task that is now a mapped task
How to reproduce
dag with a normal task A
run dag
task A success
edit dag to make task A a mapped task ( without changing name of task )
clear task
scheduler crash
Operating System
ubuntu 22.04
Versions of Apache Airflow Providers
No response
Deployment
Docker-Compose
Deployment details
No response
Anything else
No response
Are you willing to submit PR?
Code of Conduct
The text was updated successfully, but these errors were encountered: