forked from apache/airflow
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Introduce mechanism to support multiple executor configuration (apach…
…e#37635) * Introduce mechanism to support multiple executor configuration This commit delivers an isolated component of AIP-61: https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-61+Hybrid+Execution It updates the executor configuration logic to allow configuring multiple executors (and aliases per executor) in a list form (comma delimited), as well as associated methods for core Airflow code to load the default executor (first executor in the list) or any other configured executor. Testing is included with the changes but not documentation, because the feature is not yet complete and is currently disabled. User facing documentation will be delivered when the entire AIP-61 feature is released. * Move caches to module and use reloads to empty them in tests Executor data is now more aggressively cached and this was failing some tests that are trying to override the executor conf. Moved the caches to the module level and added some reload(executor_loader) calls to empty the caches for certain tests that require it. * Fix TestBaseSensor::test_prepare_for_execution This unit test was broken to begin with and was working mostly by chance. Updated the patch to be the correct method to mock and test is passing again * More reloads * Comment grammar fix * New code needed a reload of executor loader module * Fix unit test is-->are
- Loading branch information
Showing
17 changed files
with
563 additions
and
67 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.