-
Notifications
You must be signed in to change notification settings - Fork 14.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Prepare ad hoc provider release for Docker, Cassandra, Papermill #28999
Conversation
Quotiing from the PR: @ephraimbuddy @pierrejeambrun -> in case you will be pushing a new version of v2_5_test, likely that one should be cherry-picked. It is only relevant for the reason of constraints - because it turns out that new dnspython is not compatible with the way how cassandra driver + eventlet patch the dnspython during import (and fail) so it would be good when next constraints are regenerated for 2.5, the dnspython is not upgraded to 2.3.0. |
There are already several people in the eventlet/eventlet#781 thanking me (or rather our canary builds) for saving them hours of debugging - even though they did not have eventlet or dnspython as dependency. So what I am trying to do here is to save a number of airflow users who will install constraints with eventlet failing to import hours of debugging. This error is very, very specifc and nasty. The problem is that eventlet is trying to patch dnspython when importing, and fails. It's not cassandra related, it's eventlet + dnspython combination that creates the problem - cassandra is only one of many ways how the problem can be triggered. |
OK let me add it. I'll also review #28996 if merged we can release docker provider 3.5.0RC1 rather than having 3.4.1RC2 |
Ah... I was commenting on a wrong one :) I was a bit sleepy. I thought it was RC2 for Airflow :) |
So scrap my comments |
Note. I do not think we need to release Cassandra (but if you want - I am ok with it too). The fact that eventlet team quickly fixed the problem, made the problem far less..... of a problem really. The thing is that people who had older version of eventlet and dnspython will be fine. only when they install them from the scratch, they would install the latest versions - and latest versions of both are "properly" collaborating with each other., Also our constraints never point to "bad" combinations, - it's either "old good" or "old new" combination (because we included the latest fixes in 2.5.1rc2, constraints will be "good". But yeah. There are very subtle edge-cases that the release of Cassandra provider might help to avoid (when you are upgrading dnspython but not upgrading eventlet). So I have no problem with relasing Cassandra now. |
Prepare 3.4.1rc2 for docker providerPrepare 3.5.0rc1 for docker provider and minor release for cassandra, papermill
^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named
{pr_number}.significant.rst
or{issue_number}.significant.rst
, in newsfragments.