Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

#9087 Skip tests for pre-release e2e workflow on msedge/msedge-beta #9126

Merged
merged 7 commits into from
Sep 9, 2024

Conversation

mnholtz
Copy link
Collaborator

@mnholtz mnholtz commented Sep 9, 2024

What does this PR do?

Copy link

github-actions bot commented Sep 9, 2024

Playwright test results

passed  125 passed
flaky  3 flaky
skipped  4 skipped

Details

report  Open report ↗︎
stats  132 tests across 44 suites
duration  12 minutes, 2 seconds
commit  a45fc8b
info  For more information on how to debug and view this report, see our readme

Flaky tests

chrome › tests/pageEditor/addStarterBrick.spec.ts › Add starter brick to mod
msedge › tests/pageEditor/addStarterBrick.spec.ts › Add starter brick to mod
chrome › tests/regressions/welcomeStarterBricks.spec.ts › #8740: can view the starter mods on the pixiebrix.com/welcome page

Skipped tests

chrome › tests/regressions/doNotCloseSidebarOnPageEditorSave.spec.ts › #8104: Do not automatically close the sidebar when saving in the Page Editor
msedge › tests/regressions/doNotCloseSidebarOnPageEditorSave.spec.ts › #8104: Do not automatically close the sidebar when saving in the Page Editor
chrome › tests/runtime/googleSheetsIntegration.spec.ts › can activate a google spreadsheet mod with config options
msedge › tests/runtime/googleSheetsIntegration.spec.ts › can activate a google spreadsheet mod with config options

Copy link

codecov bot commented Sep 9, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 74.72%. Comparing base (8318d74) to head (a45fc8b).
Report is 275 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #9126      +/-   ##
==========================================
+ Coverage   74.24%   74.72%   +0.47%     
==========================================
  Files        1332     1345      +13     
  Lines       40817    41664     +847     
  Branches     7634     7766     +132     
==========================================
+ Hits        30306    31133     +827     
- Misses      10511    10531      +20     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@@ -258,6 +263,13 @@ test("Add starter brick to mod", async ({
});

await test.step("Add Trigger starter brick to mod", async () => {
// FIXME: https://github.com/pixiebrix/pixiebrix-extension/issues/9125
test.skip(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: we can use test.fixme instead which is functionally the same, but indicates we plan to fix it. Also the fixme comment is redundant - you can include the link in the additional description in the skip/fixme call.

https://playwright.dev/docs/api/class-test#test-fixme

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, thank you!

test.skip(
process.env.GITHUB_WORKFLOW === PRE_RELEASE_BROWSER_WORKFLOW_NAME &&
isMsEdge(chromiumChannel),
"Skipping test for MS Edge in pre-release workflow",
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it only failing in the pre-release workflow, but not in PR CI?

Copy link
Collaborator Author

@mnholtz mnholtz Sep 9, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, based off of inventory of the CI runs in the last week or so. I'm not sure what is going on yet.

Copy link

github-actions bot commented Sep 9, 2024

No loom links were found in the first post. Please add one there if you'd like to it to appear on Slack.

Do not edit this comment manually.

@mnholtz mnholtz enabled auto-merge (squash) September 9, 2024 21:01
@mnholtz mnholtz merged commit edb2563 into main Sep 9, 2024
20 checks passed
@mnholtz mnholtz deleted the bugfix/9087_skip_tests_on_pre_release_workflow branch September 9, 2024 21:18
@grahamlangford grahamlangford added this to the 2.1.2 milestone Sep 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants