Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Retry flaky UI tests in CI #1262

Merged
merged 3 commits into from
Aug 10, 2024
Merged

Conversation

mahendrapaipuri
Copy link
Contributor

@mahendrapaipuri mahendrapaipuri commented Jul 17, 2024

See comment

Closes #1230

Copy link

Thank you for making this pull request.

Did you know? You can try it on Binder: Binder:lab or Binder:notebook.

Also, the version of Jupytext developed in this PR can be installed with pip:

HATCH_BUILD_HOOKS_ENABLE=true pip install git+https://github.com/mahendrapaipuri/jupytext.git@plywrgt_config

(this requires nodejs, see more at Developing Jupytext)

Copy link

codecov bot commented Jul 17, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 95.91%. Comparing base (6089897) to head (a34d772).

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1262      +/-   ##
==========================================
- Coverage   95.93%   95.91%   -0.03%     
==========================================
  Files          29       29              
  Lines        4476     4476              
==========================================
- Hits         4294     4293       -1     
- Misses        182      183       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@mwouts
Copy link
Owner

mwouts commented Jul 31, 2024

Hi @mahendrapaipuri , I am not sure this test is "flaky" - I had relaunched the publish CI multiple times and it failed on each attempt (https://github.com/mwouts/jupytext/actions/runs/9958694800/job/28138283717) - curious to learn what makes you think that retry would improve this?

@mahendrapaipuri
Copy link
Contributor Author

Well, if you look into the CI runs of this PR, playwright actually marked that test as flaky. I guess the first run of that test is always a failure and it passes in the subsequent retries.

Anyways, that test does not add any value and hence, I removed it. I added a workflow to update snapshots directly from a PR by commenting "update snapshots". Let me know what you think!

@mwouts
Copy link
Owner

mwouts commented Aug 2, 2024

Thank you @mahendrapaipuri , this looks great! I will have to fix #1266 first but I am looking forward to taking your PR! Thanks

Signed-off-by: Mahendra Paipuri <[email protected]>
* Bump actions versions

Signed-off-by: Mahendra Paipuri <[email protected]>
@mwouts mwouts merged commit 070ba88 into mwouts:main Aug 10, 2024
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add CI workflow to update playwright snapshots automatically
2 participants