Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prepare ad hoc provider release for Docker, Cassandra, Papermill #28999

Merged
merged 2 commits into from
Jan 23, 2023

Conversation

eladkal
Copy link
Contributor

@eladkal eladkal commented Jan 17, 2023

Prepare 3.4.1rc2 for docker provider

Prepare 3.5.0rc1 for docker provider and minor release for cassandra, papermill


^ Add meaningful description above

Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.

@potiuk
Copy link
Member

potiuk commented Jan 17, 2023

As discussed in PR - you also shoudl add this one too: #28962

@eladkal
Copy link
Contributor Author

eladkal commented Jan 17, 2023

As discussed in PR - you also shoudl add this one too: #28962

How is Cassandra provider relates to docker RC2?

@potiuk
Copy link
Member

potiuk commented Jan 17, 2023

As discussed in PR - you also shoudl add this one too: #28962

How is Cassandra provider relates to docker RC2?

Quotiing from the PR:

@ephraimbuddy @pierrejeambrun -> in case you will be pushing a new version of v2_5_test, likely that one should be cherry-picked. It is only relevant for the reason of constraints - because it turns out that new dnspython is not compatible with the way how cassandra driver + eventlet patch the dnspython during import (and fail) so it would be good when next constraints are regenerated for 2.5, the dnspython is not upgraded to 2.3.0.

@potiuk
Copy link
Member

potiuk commented Jan 17, 2023

There are already several people in the eventlet/eventlet#781 thanking me (or rather our canary builds) for saving them hours of debugging - even though they did not have eventlet or dnspython as dependency.

So what I am trying to do here is to save a number of airflow users who will install constraints with eventlet failing to import hours of debugging.

This error is very, very specifc and nasty. The problem is that eventlet is trying to patch dnspython when importing, and fails. It's not cassandra related, it's eventlet + dnspython combination that creates the problem - cassandra is only one of many ways how the problem can be triggered.

@eladkal
Copy link
Contributor Author

eladkal commented Jan 17, 2023

OK let me add it.

I'll also review #28996 if merged we can release docker provider 3.5.0RC1 rather than having 3.4.1RC2

@potiuk
Copy link
Member

potiuk commented Jan 18, 2023

Ah... I was commenting on a wrong one :) I was a bit sleepy. I thought it was RC2 for Airflow :)

@potiuk
Copy link
Member

potiuk commented Jan 18, 2023

So scrap my comments

@eladkal eladkal changed the title Prepare RC2 for docker provider release Prepare ad hoc provider release for Docker and Cassandra Jan 19, 2023
@eladkal eladkal requested a review from potiuk January 19, 2023 08:25
@potiuk
Copy link
Member

potiuk commented Jan 19, 2023

Note. I do not think we need to release Cassandra (but if you want - I am ok with it too). The fact that eventlet team quickly fixed the problem, made the problem far less..... of a problem really. The thing is that people who had older version of eventlet and dnspython will be fine. only when they install them from the scratch, they would install the latest versions - and latest versions of both are "properly" collaborating with each other., Also our constraints never point to "bad" combinations, - it's either "old good" or "old new" combination (because we included the latest fixes in 2.5.1rc2, constraints will be "good".

But yeah. There are very subtle edge-cases that the release of Cassandra provider might help to avoid (when you are upgrading dnspython but not upgrading eventlet). So I have no problem with relasing Cassandra now.

@eladkal eladkal changed the title Prepare ad hoc provider release for Docker and Cassandra Prepare ad hoc provider release for Docke, Cassandra, Papermill Jan 23, 2023
@eladkal eladkal changed the title Prepare ad hoc provider release for Docke, Cassandra, Papermill Prepare ad hoc provider release for Docker, Cassandra, Papermill Jan 23, 2023
@eladkal eladkal merged commit dd6cef7 into apache:main Jan 23, 2023
@eladkal eladkal deleted the docker branch January 23, 2023 12:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants