Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce mechanism to support multiple executor configuration #37635

Merged
merged 8 commits into from
Mar 9, 2024

Commits on Feb 22, 2024

  1. Introduce mechanism to support multiple executor configuration

    This commit delivers an isolated component of AIP-61:
    https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-61+Hybrid+Execution
    
    It updates the executor configuration logic to allow configuring multiple
    executors (and aliases per executor) in a list form (comma delimited), as
    well as associated methods for core Airflow code to load the default executor
    (first executor in the list) or any other configured executor.
    
    Testing is included with the changes but not documentation, because the
    feature is not yet complete and is currently disabled. User facing
    documentation will be delivered when the entire AIP-61 feature is
    released.
    o-nikolas committed Feb 22, 2024
    Configuration menu
    Copy the full SHA
    de6dca1 View commit details
    Browse the repository at this point in the history

Commits on Feb 23, 2024

  1. Move caches to module and use reloads to empty them in tests

    Executor data is now more aggressively cached and this was failing some tests
    that are trying to override the executor conf. Moved the caches to the module
    level and added some reload(executor_loader) calls to empty the caches for
    certain tests that require it.
    o-nikolas committed Feb 23, 2024
    Configuration menu
    Copy the full SHA
    94419ff View commit details
    Browse the repository at this point in the history
  2. Fix TestBaseSensor::test_prepare_for_execution

    This unit test was broken to begin with and was working mostly by
    chance. Updated the patch to be the correct method to mock and test
    is passing again
    o-nikolas committed Feb 23, 2024
    Configuration menu
    Copy the full SHA
    68b920b View commit details
    Browse the repository at this point in the history
  3. More reloads

    o-nikolas committed Feb 23, 2024
    Configuration menu
    Copy the full SHA
    306dd61 View commit details
    Browse the repository at this point in the history

Commits on Mar 4, 2024

  1. Comment grammar fix

    o-nikolas committed Mar 4, 2024
    Configuration menu
    Copy the full SHA
    ec4c86a View commit details
    Browse the repository at this point in the history

Commits on Mar 5, 2024

  1. Configuration menu
    Copy the full SHA
    fb2cec2 View commit details
    Browse the repository at this point in the history

Commits on Mar 6, 2024

  1. Configuration menu
    Copy the full SHA
    3d26b78 View commit details
    Browse the repository at this point in the history

Commits on Mar 7, 2024

  1. Fix unit test is-->are

    o-nikolas committed Mar 7, 2024
    Configuration menu
    Copy the full SHA
    c5e6551 View commit details
    Browse the repository at this point in the history