Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DRAFT] Playwright wf #8465

Closed
wants to merge 2 commits into from
Closed

[DRAFT] Playwright wf #8465

wants to merge 2 commits into from

Conversation

fungairino
Copy link
Collaborator

What does this PR do?

Reviewer Tips

  • What order should the reviewer review the files in?
  • Are there any implementation approaches you want feedback on?

Discussion

  • Were there multiple approaches you were deciding between?

Demo

  • Paste a screenshot or demo video here

Future Work

  • Work for the issue/ticket that will be in a follow-up PR

Team Coordination

Leave all that are relevant and check off as completed

  • This PR requires security review
  • This PR introduces a new library: double check it's MIT/Apache2/permissively licensed
  • This PR requires a node/npm version update: let the team know on #engineering
  • This PR requires a documentation change (link to old docs)
  • This PR requires a tutorial update (link to old tutorials)
  • This PR requires a feature flag
  • This PR requires a environment variable change

Checklist

  • Add jest or playwright tests and/or storybook stories
  • Designate a primary reviewer

Copy link

codecov bot commented May 15, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 73.40%. Comparing base (6d95047) to head (1358f69).
Report is 66 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #8465      +/-   ##
==========================================
- Coverage   73.47%   73.40%   -0.08%     
==========================================
  Files        1334     1352      +18     
  Lines       41259    41555     +296     
  Branches     7686     7799     +113     
==========================================
+ Hits        30316    30504     +188     
- Misses      10943    11051     +108     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link

Playwright test results - MV2

failed  4 failed
passed  18 passed
flaky  1 flaky
skipped  31 skipped

Details

report  Open report ↗︎
stats  54 tests across 19 suites
duration  15 minutes, 13 seconds
commit  1358f69

Failed tests

edgeSetup › auth.setup.ts › authenticate
chrome › tests/runtime/localIntegrations.spec.ts › Local Integrations Page › can create a new integration
chrome › tests/runtime/localIntegrations.spec.ts › Local Integrations Page › #8067: blank numeric text integration configuration field validated on save
chrome › tests/smoke/workshopPage.spec.ts › extension console workshop smoke test › can navigate to workshop page without a username

Flaky tests

chrome › tests/extensionConsoleActivation.spec.ts › can activate a mod with a database

Skipped tests

edge › tests/bricks/sidebarEffects.spec.ts › sidebar effect bricks › toggle sidebar brick
chrome › tests/extensionConsoleActivation.spec.ts › can activate a mod with built-in integration
edge › tests/extensionConsoleActivation.spec.ts › can activate a mod with no config options
edge › tests/extensionConsoleActivation.spec.ts › can activate a mod with built-in integration
edge › tests/extensionConsoleActivation.spec.ts › can activate a mod with a database
edge › tests/extensionConsoleActivation.spec.ts › activating a mod when the quickbar shortcut is not configured
edge › tests/pageEditor/saveMod.spec.ts › can save a standalone trigger mod
edge › tests/pageEditor/saveMod.spec.ts › shows error notification when updating a public mod without incrementing the version
edge › tests/regressions/doNotCloseSidebarOnPageEditorSave.spec.ts › #8104: Do not automatically close the sidebar when saving in the Page Editor
edge › tests/regressions/formFlicker.spec.ts › forms flickering due to components unexpectedly unmounting/remounting › #8320: Search field loses focus while typing on snippet mod
chrome › tests/regressions/sidebarLinks.spec.ts › #8206: clicking links from the sidebar doesn't crash browser
edge › tests/regressions/sidebarLinks.spec.ts › #8206: clicking links from the sidebar doesn't crash browser
edge › tests/runtime/insertAtCursor.spec.ts › Insert at Cursor › 8157: can insert at cursor from side bar
edge › tests/runtime/insertAtCursor.spec.ts › Insert at Cursor › 8154: can insert at cursor after opening sidebar from selection menu
edge › tests/runtime/localIntegrations.spec.ts › Local Integrations Page › can create a new integration
edge › tests/runtime/localIntegrations.spec.ts › Local Integrations Page › #8067: blank numeric text integration configuration field validated on save
edge › tests/runtime/setInputValue.spec.ts › can set input value
chrome › tests/runtime/sidebarController.spec.ts › sidebar controller › shows focus dialog in top-level frame
edge › tests/runtime/sidebarController.spec.ts › sidebar controller › can open sidebar immediately from iframe without focus dialog
edge › tests/runtime/sidebarController.spec.ts › sidebar controller › shows focus dialog in top-level frame
chrome › tests/runtime/sidebarNavigation.spec.ts › sidebar mod panels are persistent during navigation
chrome › tests/runtime/sidebarNavigation.spec.ts › sidebar form and temporary panels are unavailable after navigation
edge › tests/runtime/sidebarNavigation.spec.ts › sidebar mod panels are persistent during navigation
edge › tests/runtime/sidebarNavigation.spec.ts › sidebar form and temporary panels are unavailable after navigation
edge › tests/runtime/sidebarPanelTheme.spec.ts › custom sidebar theme css file is applied to all levels of sidebar document
edge › tests/runtime/srcdocFrames.spec.ts › 8143: mods can run in srcdoc iframes
edge › tests/smoke/modsPage.spec.ts › extension console mods page smoke test › can view available mods
edge › tests/smoke/pageEditor.spec.ts › page editor smoke test › can open the page editor and connect to an open tab
edge › tests/smoke/sidebar.spec.ts › sidebar page smoke test › can open the sidebar from selection menu action and view the related mod's sidebar panel
edge › tests/smoke/workshopPage.spec.ts › extension console workshop smoke test › can navigate to workshop page without a username
edge › tests/telemetry/errors.spec.ts › can report application error to telemetry service

Copy link

Playwright test results - MV3

failed  4 failed
passed  24 passed
skipped  26 skipped

Details

report  Open report ↗︎
stats  54 tests across 19 suites
duration  16 minutes, 19 seconds
commit  1358f69

Failed tests

edgeSetup › auth.setup.ts › authenticate
chrome › tests/runtime/localIntegrations.spec.ts › Local Integrations Page › can create a new integration
chrome › tests/runtime/localIntegrations.spec.ts › Local Integrations Page › #8067: blank numeric text integration configuration field validated on save
chrome › tests/smoke/workshopPage.spec.ts › extension console workshop smoke test › can navigate to workshop page without a username

Skipped tests

edge › tests/bricks/sidebarEffects.spec.ts › sidebar effect bricks › toggle sidebar brick
edge › tests/extensionConsoleActivation.spec.ts › can activate a mod with no config options
edge › tests/extensionConsoleActivation.spec.ts › can activate a mod with built-in integration
edge › tests/extensionConsoleActivation.spec.ts › can activate a mod with a database
edge › tests/extensionConsoleActivation.spec.ts › activating a mod when the quickbar shortcut is not configured
edge › tests/pageEditor/saveMod.spec.ts › can save a standalone trigger mod
edge › tests/pageEditor/saveMod.spec.ts › shows error notification when updating a public mod without incrementing the version
edge › tests/regressions/doNotCloseSidebarOnPageEditorSave.spec.ts › #8104: Do not automatically close the sidebar when saving in the Page Editor
edge › tests/regressions/formFlicker.spec.ts › forms flickering due to components unexpectedly unmounting/remounting › #8320: Search field loses focus while typing on snippet mod
edge › tests/regressions/sidebarLinks.spec.ts › #8206: clicking links from the sidebar doesn't crash browser
edge › tests/runtime/insertAtCursor.spec.ts › Insert at Cursor › 8157: can insert at cursor from side bar
edge › tests/runtime/insertAtCursor.spec.ts › Insert at Cursor › 8154: can insert at cursor after opening sidebar from selection menu
edge › tests/runtime/localIntegrations.spec.ts › Local Integrations Page › can create a new integration
edge › tests/runtime/localIntegrations.spec.ts › Local Integrations Page › #8067: blank numeric text integration configuration field validated on save
edge › tests/runtime/setInputValue.spec.ts › can set input value
edge › tests/runtime/sidebarController.spec.ts › sidebar controller › can open sidebar immediately from iframe without focus dialog
edge › tests/runtime/sidebarController.spec.ts › sidebar controller › shows focus dialog in top-level frame
edge › tests/runtime/sidebarNavigation.spec.ts › sidebar mod panels are persistent during navigation
edge › tests/runtime/sidebarNavigation.spec.ts › sidebar form and temporary panels are unavailable after navigation
edge › tests/runtime/sidebarPanelTheme.spec.ts › custom sidebar theme css file is applied to all levels of sidebar document
edge › tests/runtime/srcdocFrames.spec.ts › 8143: mods can run in srcdoc iframes
edge › tests/smoke/modsPage.spec.ts › extension console mods page smoke test › can view available mods
edge › tests/smoke/pageEditor.spec.ts › page editor smoke test › can open the page editor and connect to an open tab
edge › tests/smoke/sidebar.spec.ts › sidebar page smoke test › can open the sidebar from selection menu action and view the related mod's sidebar panel
edge › tests/smoke/workshopPage.spec.ts › extension console workshop smoke test › can navigate to workshop page without a username
edge › tests/telemetry/errors.spec.ts › can report application error to telemetry service

@fungairino fungairino closed this May 15, 2024
@twschiller twschiller deleted the playwright-wf branch November 14, 2024 03:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant