Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[AIRFLOW-4482] Add execution_date to trigger DAG run API response #5260

Merged
merged 1 commit into from
May 9, 2019
Merged

[AIRFLOW-4482] Add execution_date to trigger DAG run API response #5260

merged 1 commit into from
May 9, 2019

Conversation

shuva10v
Copy link
Contributor

@shuva10v shuva10v commented May 8, 2019

Make sure you have checked all steps below.

Jira

Description

  • Here are some details about my PR, including screenshots of any UI changes:

We are using experimental REST API for automating Airflow from Jenkins and our workflow looks like this:

  • Jenkins job triggers DAG run (using POST /api/experimental/dags/<dag_id>/dag_runs)
  • Airflow API returns response like this: 

{"message":"Created <DagRun dag_id @ 2019-05-08 11:42:43+00:00: run_id, externally triggered: True>"}

  • Jenkins job parses the response for execution_date (2019-05-08T11:42:43 for response above)
  • Jenkins job further uses execution_date to check DAG run state using POST /api/experimental/dags/<dag_id>/dag_runs/<execution_date>
  • Also Jenkins job generates DAG run UI link

Here is the code - how it works.

Actually it is not a good idea to parse message string for execution_date. So it is proposed to add execution_date in DAG run trigger API response as a field. It makes it possible to get execution_date directly from the response without dirty and unpredictable parsing. After this improvement API response will be like this:

{'execution_date': '2019-05-08T07:03:09+00:00', 'message': 'Created <DagRun dag_id @ 2019-05-08 07:03:09+00:00: manual__2019-05-08T07:03:09+00:00, externally triggered: True>'} 

Tests

  • My PR adds the following unit tests OR does not need testing for this extremely good reason:

Asserts added for test_endpoints.test_trigger_dag

Commits

  • My commits all reference Jira issues in their subject lines, and I have squashed multiple commits if they address the same issue. In addition, my commits follow the guidelines from "How to write a good git commit message":
    1. Subject is separated from body by a blank line
    2. Subject is limited to 50 characters (not including Jira issue reference)
    3. Subject does not end with a period
    4. Subject uses the imperative mood ("add", not "adding")
    5. Body wraps at 72 characters
    6. Body explains "what" and "why", not "how"

Documentation

  • In case of new functionality, my PR adds documentation that describes how to use it.
    • All the public functions and the classes in the PR contain docstrings that explain what it does
    • If you implement backwards incompatible changes, please leave a note in the Updating.md so we can assign it to a appropriate release

Code Quality

  • Passes flake8

@XD-DENG
Copy link
Member

XD-DENG commented May 9, 2019

Thanks @shuva10v A useful feature to me.

I have restarted the failing test which may be due to transient error.

@codecov-io
Copy link

Codecov Report

Merging #5260 into master will increase coverage by 0.26%.
The diff coverage is 100%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #5260      +/-   ##
==========================================
+ Coverage   78.39%   78.66%   +0.26%     
==========================================
  Files         470      470              
  Lines       30013    30013              
==========================================
+ Hits        23528    23609      +81     
+ Misses       6485     6404      -81
Impacted Files Coverage Δ
airflow/www/api/experimental/endpoints.py 88.77% <100%> (ø) ⬆️
airflow/hooks/dbapi_hook.py 88.69% <0%> (+0.86%) ⬆️
airflow/models/connection.py 65.73% <0%> (+1.12%) ⬆️
airflow/hooks/hive_hooks.py 75% <0%> (+1.86%) ⬆️
airflow/utils/sqlalchemy.py 80.95% <0%> (+4.76%) ⬆️
airflow/operators/mysql_operator.py 100% <0%> (+100%) ⬆️
airflow/operators/mysql_to_hive.py 100% <0%> (+100%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ec05661...a9fff94. Read the comment docs.

@XD-DENG XD-DENG merged commit 66b3054 into apache:master May 9, 2019
JCoder01 pushed a commit to JCoder01/airflow that referenced this pull request May 14, 2019
andriisoldatenko pushed a commit to andriisoldatenko/airflow that referenced this pull request Jul 26, 2019
wmorris75 pushed a commit to modmed/incubator-airflow that referenced this pull request Jul 29, 2019
ashb pushed a commit to ashb/airflow that referenced this pull request Dec 18, 2019
ashb pushed a commit that referenced this pull request Dec 18, 2019
ashb pushed a commit that referenced this pull request Dec 19, 2019
kaxil pushed a commit that referenced this pull request Dec 19, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants