Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support of capacity provider strategy for ECSOperator #15848

Conversation

pavelhlushchanka
Copy link
Contributor

@pavelhlushchanka pavelhlushchanka commented May 14, 2021

This PR adds support of support of capacity provider strategy for ECSOperator. The logic is based on ECS.Client.run_task documentation.

Closes #8381

Note:
There is a logic related to the platform_version that I find tricky:

            if self.launch_type == 'FARGATE':
                run_opts['platformVersion'] = self.platform_version

As for me, the operator should pass to the run_task every parameter that user provided. And that should be responsibility of the user to think about which parameters and under which conditions are needed. But to be consistent with the logic, I also provide platform_version if capacity_provider_strategy is specified. capacity_provider_strategy can be used only with Fargate, so the platform version should be passed as well. But as we can see, picking the parameters introduces an unnecessary complexity.

@boring-cyborg boring-cyborg bot added area:providers provider:amazon-aws AWS/Amazon - related issues labels May 14, 2021
@github-actions
Copy link

The Workflow run is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks,^Build docs$,^Spell check docs$,^Provider packages,^Checks: Helm tests$,^Test OpenAPI*.

@eladkal eladkal requested a review from feluelle May 17, 2021 05:45
@pavelhlushchanka
Copy link
Contributor Author

@feluelle could you please have a look if anything else needs to be done? Thanks.

@potiuk
Copy link
Member

potiuk commented Jun 12, 2021

Can you please rebase it to latest main. I think there was an unrelated error when run last time

@pavelhlushchanka pavelhlushchanka force-pushed the capacity_provider_strategy_for_ecs_operator branch from fe82930 to a4099ac Compare June 12, 2021 21:47
@pavelhlushchanka
Copy link
Contributor Author

@potiuk done

@potiuk
Copy link
Member

potiuk commented Jun 12, 2021

Unrelated pylint error (to be fixed in #16405

@potiuk potiuk merged commit 30708b5 into apache:main Jun 12, 2021
@github-actions
Copy link

The PR is likely OK to be merged with just subset of tests for default Python and Database versions without running the full matrix of tests, because it does not modify the core of Airflow. If the committers decide that the full tests matrix is needed, they will add the label 'full tests needed'. Then you should rebase to the latest main or amend the last commit of the PR, and push it with --force-with-lease.

@github-actions github-actions bot added the okay to merge It's ok to merge this PR as it does not require more tests label Jun 12, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:providers okay to merge It's ok to merge this PR as it does not require more tests provider:amazon-aws AWS/Amazon - related issues
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support 'capacityProviderStrategy' option for ECSOperator
2 participants