Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[AIRFLOW-6527] Make send_task_to_executor timeout configurable #7143

Merged
merged 1 commit into from
Feb 2, 2020

Conversation

yuqian90
Copy link
Contributor

@yuqian90 yuqian90 commented Jan 12, 2020

This change makes the timeout for sending tasks to Celery worker configurable. If people encounter the same Celery timeout error described in AIRFLOW-6527, they may increase [celery]operation_timeout to make the timeout less likely to happen.


Issue link: AIRFLOW-6527

  • Description above provides context of the change
  • Commit message/PR title starts with [AIRFLOW-NNNN]. AIRFLOW-NNNN = JIRA ID*
  • Unit tests coverage for changes (not needed for documentation changes)
  • Commits follow "How to write a good git commit message"
  • Relevant documentation is updated including usage instructions.
  • I will engage committers as explained in Contribution Workflow Example.

* For document-only changes commit message can start with [AIRFLOW-XXXX].


In case of fundamental code change, Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in UPDATING.md.
Read the Pull Request Guidelines for more information.

@boring-cyborg boring-cyborg bot added the area:Scheduler including HA (high availability) scheduler label Jan 12, 2020
@codecov-io
Copy link

codecov-io commented Jan 13, 2020

Codecov Report

Merging #7143 into master will decrease coverage by 0.27%.
The diff coverage is 100%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #7143      +/-   ##
==========================================
- Coverage   85.33%   85.05%   -0.28%     
==========================================
  Files         791      791              
  Lines       40137    40138       +1     
==========================================
- Hits        34251    34141     -110     
- Misses       5886     5997     +111
Impacted Files Coverage Δ
airflow/executors/celery_executor.py 88.51% <100%> (+0.07%) ⬆️
airflow/kubernetes/volume_mount.py 44.44% <0%> (-55.56%) ⬇️
airflow/kubernetes/volume.py 52.94% <0%> (-47.06%) ⬇️
airflow/kubernetes/pod_launcher.py 45.25% <0%> (-46.72%) ⬇️
airflow/kubernetes/refresh_config.py 50.98% <0%> (-23.53%) ⬇️
...viders/cncf/kubernetes/operators/kubernetes_pod.py 76.47% <0%> (-21.18%) ⬇️
airflow/jobs/backfill_job.py 91.88% <0%> (+0.28%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4e7bb5f...21e20aa. Read the comment docs.

@yuqian90 yuqian90 force-pushed the send_task_timeout branch 2 times, most recently from ccae256 to 69a9a9a Compare January 25, 2020 13:13
@yuqian90 yuqian90 force-pushed the send_task_timeout branch 3 times, most recently from 88fb2c1 to d86d795 Compare January 26, 2020 09:20
@kaxil kaxil merged commit f757a54 into apache:master Feb 2, 2020
kaxil pushed a commit that referenced this pull request Feb 3, 2020
kaxil pushed a commit that referenced this pull request Feb 3, 2020
@yuqian90 yuqian90 deleted the send_task_timeout branch February 13, 2020 08:19
galuszkak pushed a commit to FlyrInc/apache-airflow that referenced this pull request Mar 5, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:Scheduler including HA (high availability) scheduler
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants