Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[AIRFLOW-6471] Add pytest-instafail plugin #7064

Conversation

potiuk
Copy link
Member

@potiuk potiuk commented Jan 5, 2020

We have a problem currently that if a test fails in CI we do not see the
failures immediately - only when it finishes, but when tests hang, sometimes
the details about failed tests are not shown immediately. We tried to increase
verbosity but it's not very helpful.

The pytest-instafail plugin solves the problem without increasing verbosity.


Link to JIRA issue: https://issues.apache.org/jira/browse/AIRFLOW-6471

  • Description above provides context of the change
  • Commit message starts with [AIRFLOW-NNNN], where AIRFLOW-NNNN = JIRA ID*
  • Unit tests coverage for changes (not needed for documentation changes)
  • Commits follow "How to write a good git commit message"
  • Relevant documentation is updated including usage instructions.
  • I will engage committers as explained in Contribution Workflow Example.

(*) For document-only changes, no JIRA issue is needed. Commit message starts [AIRFLOW-XXXX].


In case of fundamental code change, Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in UPDATING.md.
Read the Pull Request Guidelines for more information.

We have a problem currently that if a test fails in CI we do not see the
failures immediately - only when it finishes, but when tests hang, sometimes
the details about failed tests are not shown immediately. We tried to increase
verbosity but it's not very helpful.

The pytest-instafail plugin solves the problem without increasing verbosity.
@potiuk
Copy link
Member Author

potiuk commented Jan 5, 2020

This is how it will look like - no more disappearing test failures after the build times out.

Screenshot 2020-01-05 at 16 59 45

I think it removes the need of decreasing the timeout (which can lead to some unnecessary errors when a test runs significantly longer than usual) and increasing pytest verbosity (when a test fail we will immediately see all necessary details to see what was going on).

@potiuk potiuk merged commit 869ca96 into apache:master Jan 5, 2020
@codecov-io
Copy link

codecov-io commented Jan 5, 2020

Codecov Report

Merging #7064 into master will decrease coverage by 0.06%.
The diff coverage is n/a.

Impacted file tree graph

@@            Coverage Diff            @@
##           master   #7064      +/-   ##
=========================================
- Coverage   84.76%   84.7%   -0.07%     
=========================================
  Files         679     679              
  Lines       38647   38647              
=========================================
- Hits        32761   32736      -25     
- Misses       5886    5911      +25
Impacted Files Coverage Δ
airflow/kubernetes/volume_mount.py 44.44% <0%> (-55.56%) ⬇️
airflow/kubernetes/volume.py 52.94% <0%> (-47.06%) ⬇️
airflow/kubernetes/pod_launcher.py 45.25% <0%> (-46.72%) ⬇️
airflow/kubernetes/refresh_config.py 50.98% <0%> (-23.53%) ⬇️
...rflow/contrib/operators/kubernetes_pod_operator.py 78.75% <0%> (-20%) ⬇️
airflow/jobs/backfill_job.py 91.59% <0%> (-0.29%) ⬇️
airflow/hooks/dbapi_hook.py 91.73% <0%> (+0.82%) ⬆️
airflow/models/connection.py 68.78% <0%> (+0.97%) ⬆️
airflow/hooks/hive_hooks.py 77.55% <0%> (+1.53%) ⬆️
airflow/utils/sqlalchemy.py 96.66% <0%> (+6.66%) ⬆️
... and 2 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update df2c288...d35d0b6. Read the comment docs.

@tooptoop4
Copy link
Contributor

@potiuk
Copy link
Member Author

potiuk commented Jan 5, 2020

It succeeded now I think!

potiuk added a commit that referenced this pull request Jan 21, 2020
We have a problem currently that if a test fails in CI we do not see the
failures immediately - only when it finishes, but when tests hang, sometimes
the details about failed tests are not shown immediately. We tried to increase
verbosity but it's not very helpful.

The pytest-instafail plugin solves the problem without increasing verbosity.

(cherry picked from commit 869ca96)
kaxil pushed a commit that referenced this pull request Jan 22, 2020
We have a problem currently that if a test fails in CI we do not see the
failures immediately - only when it finishes, but when tests hang, sometimes
the details about failed tests are not shown immediately. We tried to increase
verbosity but it's not very helpful.

The pytest-instafail plugin solves the problem without increasing verbosity.

(cherry picked from commit 869ca96)
kaxil pushed a commit that referenced this pull request Jan 23, 2020
We have a problem currently that if a test fails in CI we do not see the
failures immediately - only when it finishes, but when tests hang, sometimes
the details about failed tests are not shown immediately. We tried to increase
verbosity but it's not very helpful.

The pytest-instafail plugin solves the problem without increasing verbosity.

(cherry picked from commit 869ca96)
potiuk added a commit that referenced this pull request Jan 26, 2020
We have a problem currently that if a test fails in CI we do not see the
failures immediately - only when it finishes, but when tests hang, sometimes
the details about failed tests are not shown immediately. We tried to increase
verbosity but it's not very helpful.

The pytest-instafail plugin solves the problem without increasing verbosity.

(cherry picked from commit 869ca96)
kaxil pushed a commit that referenced this pull request Jan 26, 2020
We have a problem currently that if a test fails in CI we do not see the
failures immediately - only when it finishes, but when tests hang, sometimes
the details about failed tests are not shown immediately. We tried to increase
verbosity but it's not very helpful.

The pytest-instafail plugin solves the problem without increasing verbosity.

(cherry picked from commit 869ca96)
galuszkak pushed a commit to FlyrInc/apache-airflow that referenced this pull request Mar 5, 2020
We have a problem currently that if a test fails in CI we do not see the
failures immediately - only when it finishes, but when tests hang, sometimes
the details about failed tests are not shown immediately. We tried to increase
verbosity but it's not very helpful.

The pytest-instafail plugin solves the problem without increasing verbosity.
kaxil pushed a commit to astronomer/airflow that referenced this pull request Mar 30, 2020
We have a problem currently that if a test fails in CI we do not see the
failures immediately - only when it finishes, but when tests hang, sometimes
the details about failed tests are not shown immediately. We tried to increase
verbosity but it's not very helpful.

The pytest-instafail plugin solves the problem without increasing verbosity.

(cherry picked from commit 869ca96)
(cherry picked from commit bab5551)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants