Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[AIRFLOW-6702] Dumping kind logs to file.io. #7319

Merged

Conversation

potiuk
Copy link
Member

@potiuk potiuk commented Feb 1, 2020

Kubernetes Kind logs are now always dumped to file.io
service.

Kind logs were not dumped on failure because run_command takes
only string now and not list


Issue link: AIRFLOW-6702

Make sure to mark the boxes below before creating PR: [x]

  • Description above provides context of the change
  • Commit message/PR title starts with [AIRFLOW-NNNN]. AIRFLOW-NNNN = JIRA ID*
  • Unit tests coverage for changes (not needed for documentation changes)
  • Commits follow "How to write a good git commit message"
  • Relevant documentation is updated including usage instructions.
  • I will engage committers as explained in Contribution Workflow Example.

* For document-only changes commit message can start with [AIRFLOW-XXXX].


In case of fundamental code change, Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in UPDATING.md.
Read the Pull Request Guidelines for more information.

@potiuk potiuk force-pushed the AIRFLOW-6702-dump-kind-logs-fails branch 3 times, most recently from 49bcf59 to 3a901e5 Compare February 1, 2020 18:13
@codecov-io
Copy link

codecov-io commented Feb 1, 2020

Codecov Report

Merging #7319 into master will increase coverage by 0.13%.
The diff coverage is n/a.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #7319      +/-   ##
==========================================
+ Coverage   85.87%   86.01%   +0.13%     
==========================================
  Files         863      866       +3     
  Lines       40520    40556      +36     
==========================================
+ Hits        34798    34885      +87     
+ Misses       5722     5671      -51
Impacted Files Coverage Δ
airflow/kubernetes/volume_mount.py 44.44% <0%> (-55.56%) ⬇️
airflow/kubernetes/volume.py 52.94% <0%> (-47.06%) ⬇️
airflow/kubernetes/pod_launcher.py 45.25% <0%> (-46.72%) ⬇️
airflow/kubernetes/refresh_config.py 50.98% <0%> (-23.53%) ⬇️
...viders/cncf/kubernetes/operators/kubernetes_pod.py 76.47% <0%> (-21.18%) ⬇️
airflow/providers/amazon/aws/operators/datasync.py 33.54% <0%> (-0.68%) ⬇️
airflow/models/taskinstance.py 94.28% <0%> (-0.43%) ⬇️
airflow/www/validators.py 100% <0%> (ø) ⬆️
airflow/www/forms.py 100% <0%> (ø) ⬆️
.../providers/amazon/aws/operators/cloud_formation.py 100% <0%> (ø)
... and 16 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 27cc835...395dba3. Read the comment docs.

Copy link
Member

@ashb ashb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Green woohoo! Looks like a bit of tidying up to do before we merge this though?

@potiuk potiuk force-pushed the AIRFLOW-6702-dump-kind-logs-fails branch from 26f39a4 to 30b8ea4 Compare February 2, 2020 07:22
@potiuk potiuk changed the title [AIRFLOW-6702] Dumping kind logs on failure works now [AIRFLOW-6702] Dumping kind logs to file.io. Feb 2, 2020
Kubernetes Kind logs are now always dumped to file.io
service.
@potiuk potiuk force-pushed the AIRFLOW-6702-dump-kind-logs-fails branch from 30b8ea4 to 64fa4b0 Compare February 2, 2020 07:27
@potiuk
Copy link
Member Author

potiuk commented Feb 2, 2020

@ashb, @dimberman @kaxil @mik-laj @nuclearpinguin -> this one introduces extracting logs after tests for Kubernetes Kind. It dumps all the logs to file.io service - which will hold any files you send for free for 14 days. I am .tar.gz ipping all the logs from kind and print URL to download the .tar.gz in the Travis CI logs.

I decided to run it for all builds, not only the failed ones - this will make it easy to compare "ok" and "failed" logs and find problems more easily.

airflow/configuration.py Outdated Show resolved Hide resolved
airflow/configuration.py Outdated Show resolved Hide resolved
@potiuk potiuk merged commit d8763a2 into apache:master Feb 2, 2020
potiuk added a commit that referenced this pull request Feb 2, 2020
* [AIRFLOW-6702] Dumping kind logs works now

Kubernetes Kind logs are now always dumped to file.io
service.

(cherry picked from commit d8763a2)
kaxil pushed a commit that referenced this pull request Feb 3, 2020
* [AIRFLOW-6702] Dumping kind logs works now

Kubernetes Kind logs are now always dumped to file.io
service.

(cherry picked from commit d8763a2)
galuszkak pushed a commit to FlyrInc/apache-airflow that referenced this pull request Mar 5, 2020
* [AIRFLOW-6702] Dumping kind logs works now

Kubernetes Kind logs are now always dumped to file.io
service.
kaxil pushed a commit to astronomer/airflow that referenced this pull request Mar 30, 2020
* [AIRFLOW-6702] Dumping kind logs works now

Kubernetes Kind logs are now always dumped to file.io
service.

(cherry picked from commit d8763a2)
(cherry picked from commit 517c978)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants