Workaround occasional deadlocks with MSSQL #19856
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
We already have a mechanism to retry operations that could result
in temporary deadlocks - this have been helpful with handling MySQL
deadlocks - however similar problems occur occasionally in MSSQL and
there we get a DBAPIError rather than OperationalError:
`sqlalchemy.exc.DBAPIError: (pyodbc.Error) ('40001', '[40001]
[Microsoft][ODBC Driver 17 for SQL Server][SQL Server]Transaction
(Process ID 55) was deadlocked on lock resources with another process
and has been chosen as the deadlock victim. Rerun the transaction.
(1205) (SQLExecDirectW)');
This PR adds DBAPIError to the list of errors that are handled
by `run_with_db_retries` to mitigate such occasional deadlocks.
potiuk
requested review from
ashb,
ephraimbuddy,
jedcunningham,
mik-laj,
uranusjr and
kaxil
uranusjr
approved these changes
Nov 27, 2021
github-actions
bot
added
the
full tests needed
|
The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and take the risk. If they don't merge it quickly - please rebase it to the latest main at your convenience, or amend the last commit of the PR, and push it with --force-with-lease. |
dillonjohnson
pushed a commit
to dillonjohnson/airflow
that referenced
this pull request
Dec 1, 2021
We already have a mechanism to retry operations that could result
in temporary deadlocks - this have been helpful with handling MySQL
deadlocks - however similar problems occur occasionally in MSSQL and
there we get a DBAPIError rather than OperationalError:
`sqlalchemy.exc.DBAPIError: (pyodbc.Error) ('40001', '[40001]
[Microsoft][ODBC Driver 17 for SQL Server][SQL Server]Transaction
(Process ID 55) was deadlocked on lock resources with another process
and has been chosen as the deadlock victim. Rerun the transaction.
(1205) (SQLExecDirectW)');
This PR adds DBAPIError to the list of errors that are handled
by `run_with_db_retries` to mitigate such occasional deadlocks.
jedcunningham
pushed a commit
that referenced
this pull request
Dec 7, 2021
We already have a mechanism to retry operations that could result
in temporary deadlocks - this have been helpful with handling MySQL
deadlocks - however similar problems occur occasionally in MSSQL and
there we get a DBAPIError rather than OperationalError:
`sqlalchemy.exc.DBAPIError: (pyodbc.Error) ('40001', '[40001]
[Microsoft][ODBC Driver 17 for SQL Server][SQL Server]Transaction
(Process ID 55) was deadlocked on lock resources with another process
and has been chosen as the deadlock victim. Rerun the transaction.
(1205) (SQLExecDirectW)');
This PR adds DBAPIError to the list of errors that are handled
by `run_with_db_retries` to mitigate such occasional deadlocks.
(cherry picked from commit e87856d)
38 tasks
2 tasks
jmaicher
added a commit
to jmaicher/airflow
that referenced
this pull request
Jul 19, 2024
In apache#19856, we added `DBAPIError` besides `OperationalError` to the retry exception types, but did not change the `retry_db_transaction` decorator to rollback transaction after failures and before a retry. In certain cases (see apache#40882), this is needed as otherwise all retries will fail when the current session/transaction was "poisened" by the initial error.
potiuk
pushed a commit
that referenced
this pull request
Jul 19, 2024
In #19856, we added `DBAPIError` besides `OperationalError` to the retry exception types, but did not change the `retry_db_transaction` decorator to rollback transaction after failures and before a retry. In certain cases (see #40882), this is needed as otherwise all retries will fail when the current session/transaction was "poisened" by the initial error.
romsharon98
pushed a commit
to romsharon98/airflow
that referenced
this pull request
Jul 26, 2024
In apache#19856, we added `DBAPIError` besides `OperationalError` to the retry exception types, but did not change the `retry_db_transaction` decorator to rollback transaction after failures and before a retry. In certain cases (see apache#40882), this is needed as otherwise all retries will fail when the current session/transaction was "poisened" by the initial error.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
We already have a mechanism to retry operations that could result
in temporary deadlocks - this have been helpful with handling MySQL
deadlocks - however similar problems occur occasionally in MSSQL and
there we get a DBAPIError rather than OperationalError:
`sqlalchemy.exc.DBAPIError: (pyodbc.Error) ('40001', '[40001]
[Microsoft][ODBC Driver 17 for SQL Server][SQL Server]Transaction
(Process ID 55) was deadlocked on lock resources with another process
and has been chosen as the deadlock victim. Rerun the transaction.
(1205) (SQLExecDirectW)');
This PR adds DBAPIError to the list of errors that are handled
by
run_with_db_retriesto mitigate such occasional deadlocks.^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code change, Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in UPDATING.md.