Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[AIRFLOW-6177] Log DAG processors timeout event at error level, not info #6731

Merged
merged 1 commit into from
Dec 6, 2019
Merged

[AIRFLOW-6177] Log DAG processors timeout event at error level, not info #6731

merged 1 commit into from
Dec 6, 2019

Conversation

robinedwards
Copy link
Contributor

@robinedwards robinedwards commented Dec 4, 2019 •
edited
Loading

This case prevents a DAG from being scheduled by the scheduler so should be an error to make troubleshooting easier.

https://issues.apache.org/jira/browse/AIRFLOW-6177

@codecov-io
Copy link

codecov-io commented Dec 4, 2019 •
edited
Loading

Codecov Report

Merging #6731 into master will decrease coverage by <.01%.
The diff coverage is 100%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #6731      +/-   ##
==========================================
- Coverage   84.54%   84.54%   -0.01%     
==========================================
  Files         669      669              
  Lines       37738    37738              
==========================================
- Hits        31907    31904       -3     
- Misses       5831     5834       +3
Impacted Files Coverage Δ
airflow/utils/dag_processing.py 87.79% <100%> (+7.75%) ⬆️
airflow/kubernetes/volume_mount.py 44.44% <0%> (-55.56%) ⬇️
airflow/kubernetes/volume.py 52.94% <0%> (-47.06%) ⬇️
airflow/kubernetes/pod_launcher.py 45.25% <0%> (-46.72%) ⬇️
airflow/kubernetes/refresh_config.py 50.98% <0%> (-23.53%) ⬇️
...rflow/contrib/operators/kubernetes_pod_operator.py 78.2% <0%> (-20.52%) ⬇️
airflow/jobs/backfill_job.py 91.18% <0%> (-0.31%) ⬇️
airflow/models/dag.py 90.95% <0%> (+0.13%) ⬆️
airflow/www/views.py 75.93% <0%> (+0.82%) ⬆️
airflow/utils/helpers.py 77.97% <0%> (+1.19%) ⬆️
... and 8 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 167fa33...5755d33. Read the comment docs.

@XD-DENG
Copy link
Member

XD-DENG commented Dec 5, 2019

Hi @robinedwards

  • This is a code change, so JIRA ticket is required.
  • You can always change logging level in your configuration, so I don't find this change necessary.

@ashb
Copy link
Member

ashb commented Dec 5, 2019

@XD-DENG It is already logged at the default level (since it's Info), this is more just to make it more obvious that something is going wrong -- if a user sees this in their logs it is a sign of problems (dag file is too complex, not enough resources for scheduler etc) so error level makes sense.

@ashb ashb changed the title [AIRFLOW-XXX] Increase log level for clarity [AIRFLOW-XXX] Log DAG processors timeout event at error level, not info Dec 5, 2019
@XD-DENG
Copy link
Member

XD-DENG commented Dec 5, 2019

Get it. Thanks @ash for the clarification.

Let’s have a JIRA for this before we merge then.

@robinedwards
Copy link
Contributor Author

Super I will file one now :-)

@robinedwards robinedwards changed the title [AIRFLOW-XXX] Log DAG processors timeout event at error level, not info [AIRFLOW-6177] Log DAG processors timeout event at error level, not info Dec 5, 2019
@robinedwards
[AIRFLOW-6177] Increase log level for clarity
5755d33 
This case prevents a DAG from being scheduled so should be an error.
@XD-DENG XD-DENG merged commit fb929eb into apache:master Dec 6, 2019
@XD-DENG
Copy link
Member

XD-DENG commented Dec 6, 2019

Merged. Thanks @robinedwards

potiuk pushed a commit that referenced this pull request Dec 10, 2019
@robinedwards @potiuk
[AIRFLOW-6177] Log DAG processors timeout event at error level, not i…
166f347 
…nfo (#6731)

This case prevents a DAG from being scheduled so should be an error.

(cherry picked from commit fb929eb)
potiuk pushed a commit that referenced this pull request Dec 10, 2019
@robinedwards @potiuk
[AIRFLOW-6177] Log DAG processors timeout event at error level, not i…
faa62a2 
…nfo (#6731)

This case prevents a DAG from being scheduled so should be an error.

(cherry picked from commit fb929eb)
potiuk pushed a commit that referenced this pull request Dec 10, 2019
@robinedwards @potiuk
[AIRFLOW-6177] Log DAG processors timeout event at error level, not i…
21dfa8c 
…nfo (#6731)

This case prevents a DAG from being scheduled so should be an error.

(cherry picked from commit fb929eb)
potiuk pushed a commit that referenced this pull request Dec 10, 2019
@robinedwards @potiuk
[AIRFLOW-6177] Log DAG processors timeout event at error level, not i…
845e9fe 
…nfo (#6731)

This case prevents a DAG from being scheduled so should be an error.

(cherry picked from commit fb929eb)
potiuk pushed a commit that referenced this pull request Dec 11, 2019
@robinedwards @potiuk
[AIRFLOW-6177] Log DAG processors timeout event at error level, not i…
fd7bb38 
…nfo (#6731)

This case prevents a DAG from being scheduled so should be an error.

(cherry picked from commit fb929eb)
potiuk pushed a commit that referenced this pull request Dec 11, 2019
@robinedwards @potiuk
[AIRFLOW-6177] Log DAG processors timeout event at error level, not i…
1f44135 
…nfo (#6731)

This case prevents a DAG from being scheduled so should be an error.

(cherry picked from commit fb929eb)
potiuk pushed a commit that referenced this pull request Dec 12, 2019
@robinedwards @potiuk
[AIRFLOW-6177] Log DAG processors timeout event at error level, not i…
3bbe03a 
…nfo (#6731)

This case prevents a DAG from being scheduled so should be an error.

(cherry picked from commit fb929eb)
kaxil pushed a commit that referenced this pull request Dec 12, 2019
@robinedwards @kaxil
[AIRFLOW-6177] Log DAG processors timeout event at error level, not i…
801ebd7 
…nfo (#6731)

This case prevents a DAG from being scheduled so should be an error.

(cherry picked from commit fb929eb)
kaxil pushed a commit that referenced this pull request Dec 12, 2019
@robinedwards @kaxil
[AIRFLOW-6177] Log DAG processors timeout event at error level, not i…
13e5798 
…nfo (#6731)

This case prevents a DAG from being scheduled so should be an error.

(cherry picked from commit fb929eb)
kaxil pushed a commit that referenced this pull request Dec 17, 2019
@robinedwards @kaxil
[AIRFLOW-6177] Log DAG processors timeout event at error level, not i…
7e03dbb 
…nfo (#6731)

This case prevents a DAG from being scheduled so should be an error.

(cherry picked from commit fb929eb)
galuszkak pushed a commit to FlyrInc/apache-airflow that referenced this pull request Mar 5, 2020
@robinedwards @galuszkak
[AIRFLOW-6177] Log DAG processors timeout event at error level, not i…
b0e47f8 
…nfo (apache#6731)

This case prevents a DAG from being scheduled so should be an error.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ashb ashb ashb approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@robinedwards @codecov-io @XD-DENG @ashb