Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[AIRFLOW-4843] Allow orchestration via Docker Swarm aka SwarmOperator #5483

Closed
wants to merge 1 commit into from

Conversation

akki
Copy link
Contributor

@akki akki commented Jun 26, 2019

Make sure you have checked all steps below.

Jira

  • My PR addresses the following AIRFLOW-4843 issues and references them in the PR title.
    • https://issues.apache.org/jira/browse/AIRFLOW-4843
    • In case you are fixing a typo in the documentation you can prepend your commit with [AIRFLOW-XXX], code changes always need a Jira issue.
    • In case you are proposing a fundamental code change, you need to create an Airflow Improvement Proposal (AIP).
    • In case you are adding a dependency, check if the license complies with the ASF 3rd Party License Policy.

Description

Tests

  • My PR adds the following unit tests:
    SwarmOperatorTestCase

Commits

  • My commits all reference Jira issues in their subject lines, and I have squashed multiple commits if they address the same issue. In addition, my commits follow the guidelines from "How to write a good git commit message":
    1. Subject is separated from body by a blank line
    2. Subject is limited to 50 characters (not including Jira issue reference)
    3. Subject does not end with a period
    4. Subject uses the imperative mood ("add", not "adding")
    5. Body wraps at 72 characters
    6. Body explains "what" and "why", not "how"

Documentation

  • In case of new functionality, my PR adds documentation that describes how to use it.
    • All the public functions and the classes in the PR contain docstrings that explain what it does
    • If you implement backwards incompatible changes, please leave a note in the Updating.md so we can assign it to a appropriate release

Code Quality

  • Passes flake8

Add support for running Docker containers via Docker Swarm
which allows the task to run on any machine (node) which
is a part of your Swarm cluster

More details: https://issues.apache.org/jira/browse/AIRFLOW-4843
@akki
Copy link
Contributor Author

akki commented Jun 26, 2019

The check that has failed is Check license header and the error doesn't seem to be related to this PR: Our attempt to download rat locally to /tmp/lib/apache-rat-0.12.jar failed..

I see the same error with another recent PR so I am guessing there's some problem with the Travis build? Is there any such known issue?

@ashb
Copy link
Member

ashb commented Jun 26, 2019

Please target the master branch for PRs.

@akki
Copy link
Contributor Author

akki commented Jun 26, 2019

@ashb I was planning to work on the master branch earlier but soon realised that there are a lot of breaking changes between that and the current version that gets installed with pip i.e. 1.10.3.

Would you still suggest me to port this code as per master? How would these changes get integrated into 1.10.3 in that case because that is what we are currently using in my company (and have other scripts which break if run against master)?

@akki
Copy link
Contributor Author

akki commented Jul 1, 2019

Created another PR against master, just in case it is possible to add this feature to v1.10 as well - #5489.

@akki
Copy link
Contributor Author

akki commented Jul 30, 2019

This branch has diverged a lot from the actual branch, so closing this (at least for now).
Current PR - #5489.

@akki akki closed this Jul 30, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants