-
Notifications
You must be signed in to change notification settings - Fork 14.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Feature: Move celery.default_queue to operators.default_queue to allow re-use between executors #14699
Conversation
Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contribution Guide (https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst)
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks reasonable to me
The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and take the risk. If they don't merge it quickly - please rebase it to the latest master at your convenience, or amend the last commit of the PR, and push it with --force-with-lease. |
@leonsmith could you please take a look at the CI issues? |
Sure @turbaszek might need a little assistance as they look pretty unrelated at a surface glance? |
The test suit looked like it mostly passed last night (after a rebase onto master), 2 of the actions look like they got killed as they exited with a 137 code. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
(non blocking change request)
8243062
to
bef1190
Compare
@leonsmith Can you do a rebase? I would like to merge this change. |
…between executors
…ig location from celery to operators
Remove `default_queue` from airflow/config_templates/default_test.cfg as its set to the same value from airflow.cfg Co-authored-by: Ash Berlin-Taylor <[email protected]>
@mik-laj rebased as requested 👍 |
The
default_queue
config option resides in thecelery
section.We already have an operators config section which would be a better place to specify a default.
This also allows other executors to re-use the default queue functionality without having to set celery specific config.
closes: #14696
^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code change, Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in UPDATING.md.