Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

#9087 Skip tests for pre-release e2e workflow on msedge/msedge-beta #9126

Merged
merged 7 commits into from
Sep 9, 2024
Merged

#9087 Skip tests for pre-release e2e workflow on msedge/msedge-beta #9126

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
f0feb42
skip sidebarLink test
mnholtz Sep 9, 2024
7a199fe
create utils for checking chromium channel
mnholtz Sep 9, 2024
f2ff7a6
skip remaining tets
mnholtz Sep 9, 2024
ab23209
introduce new constant for pre-release browser workflow name
mnholtz Sep 9, 2024
a1f0d97
replace test.skip with test.fixme
mnholtz Sep 9, 2024
cfa2747
remove unused chromium util
mnholtz Sep 9, 2024
a45fc8b
replace logic with isMsEdge where appropriate
mnholtz Sep 9, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 13 additions & 1 deletion end-to-end-tests/tests/pageEditor/addStarterBrick.spec.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,11 @@ import { test, expect } from "../../fixtures/testBase";

// @ts-expect-error -- https://youtrack.jetbrains.com/issue/AQUA-711/Provide-a-run-configuration-for-Playwright-tests-in-specs-with-fixture-imports-only
import { type Page, test as base } from "@playwright/test";
import { getSidebarPage } from "../../utils";
import {
getSidebarPage,
isMsEdge,
PRE_RELEASE_BROWSER_WORKFLOW_NAME,
} from "../../utils";
import { SupportedChannels } from "playwright.config";

test("Add new starter brick", async ({
Expand Down Expand Up @@ -140,6 +144,7 @@ test("Add starter brick to mod", async ({
newPageEditorPage,
extensionId,
verifyModDefinitionSnapshot,
chromiumChannel,
}) => {
await page.goto("/");
const pageEditorPage = await newPageEditorPage(page.url());
Expand Down Expand Up @@ -258,6 +263,13 @@ test("Add starter brick to mod", async ({
});

await test.step("Add Trigger starter brick to mod", async () => {
// FIXME: https://github.com/pixiebrix/pixiebrix-extension/issues/9125
test.skip(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: we can use test.fixme instead which is functionally the same, but indicates we plan to fix it. Also the fixme comment is redundant - you can include the link in the additional description in the skip/fixme call.

https://playwright.dev/docs/api/class-test#test-fixme

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, thank you!

process.env.GITHUB_WORKFLOW === PRE_RELEASE_BROWSER_WORKFLOW_NAME &&
isMsEdge(chromiumChannel),
"Skipping test for MS Edge in pre-release workflow",
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it only failing in the pre-release workflow, but not in PR CI?

Copy link
Collaborator Author

@mnholtz mnholtz Sep 9, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, based off of inventory of the CI runs in the last week or so. I'm not sure what is going on yet.

);

const modActionMenu = await openModActionMenu();
await modActionMenu.addStarterBrick("Trigger");

Expand Down
15 changes: 14 additions & 1 deletion end-to-end-tests/tests/pageEditor/copyMod.spec.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,12 @@ import { uuidv4 } from "@/types/helpers";
import { type Serializable } from "playwright-core/types/structs";
import path from "node:path";
import { FloatingActionButton } from "end-to-end-tests/pageObjects/floatingActionButton";
import { getSidebarPage, runModViaQuickBar } from "end-to-end-tests/utils";
import {
getSidebarPage,
runModViaQuickBar,
isMsEdge,
PRE_RELEASE_BROWSER_WORKFLOW_NAME,
} from "end-to-end-tests/utils";
import { VALID_UUID_REGEX } from "@/types/stringTypes";

test("copying a mod that uses the PixieBrix API is copied correctly", async ({
Expand Down Expand Up @@ -81,7 +86,15 @@ test("run a copied mod with a built-in integration", async ({
context,
newPageEditorPage,
verifyModDefinitionSnapshot,
chromiumChannel,
}) => {
// FIXME: https://github.com/pixiebrix/pixiebrix-extension/issues/9125
test.skip(
process.env.GITHUB_WORKFLOW === PRE_RELEASE_BROWSER_WORKFLOW_NAME &&
isMsEdge(chromiumChannel),
"Skipping test for MS Edge in pre-release workflow",
);

let giphyRequestPostData: Serializable;
// The giphy search request is proxied through the PixieBrix server, which is kicked off in the background/service
// worker. Playwright experimentally supports mocking service worker requests, see
Expand Down
15 changes: 14 additions & 1 deletion end-to-end-tests/tests/pageEditor/liveEditing.spec.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,15 +19,28 @@ import { expect, test } from "../../fixtures/testBase";
// @ts-expect-error -- https://youtrack.jetbrains.com/issue/AQUA-711/Provide-a-run-configuration-for-Playwright-tests-in-specs-with-fixture-imports-only
import { test as base } from "@playwright/test";
import { ActivateModPage } from "../../pageObjects/extensionConsole/modsPage";
import { getSidebarPage, isSidebarOpen } from "../../utils";
import {
getSidebarPage,
isMsEdge,
isSidebarOpen,
PRE_RELEASE_BROWSER_WORKFLOW_NAME,
} from "../../utils";

test("live editing behavior", async ({
page,
extensionId,
modDefinitionsMap,
newPageEditorPage,
verifyModDefinitionSnapshot,
chromiumChannel,
}) => {
// FIXME: https://github.com/pixiebrix/pixiebrix-extension/issues/9125
test.skip(
process.env.GITHUB_WORKFLOW === PRE_RELEASE_BROWSER_WORKFLOW_NAME &&
isMsEdge(chromiumChannel),
"Skipping test for MS Edge in pre-release workflow",
);

await test.step("Activate test mod and navigate to testing site", async () => {
const modId = "@e2e-testing/page-editor-live-editing-test";
const activationPage = new ActivateModPage(page, extensionId, modId);
Expand Down
43 changes: 18 additions & 25 deletions end-to-end-tests/tests/regressions/sidebarLinks.spec.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,13 @@ import {
// @ts-expect-error -- https://youtrack.jetbrains.com/issue/AQUA-711/Provide-a-run-configuration-for-Playwright-tests-in-specs-with-fixture-imports-only
test as base,
} from "@playwright/test";
import { ensureVisibility, getBrowserOs, getSidebarPage } from "../../utils";
import {
ensureVisibility,
getBrowserOs,
getSidebarPage,
isMsEdge,
PRE_RELEASE_BROWSER_WORKFLOW_NAME,
} from "../../utils";
import { getBaseExtensionConsoleUrl } from "../../pageObjects/constants";
import { SupportedChannels } from "../../../playwright.config";

Expand Down Expand Up @@ -80,6 +86,13 @@ test("#8206: clicking links from the sidebar doesn't crash browser", async ({
chromiumChannel,
baseURL,
}) => {
// FIXME: https://github.com/pixiebrix/pixiebrix-extension/issues/9125
test.skip(
process.env.GITHUB_WORKFLOW === PRE_RELEASE_BROWSER_WORKFLOW_NAME &&
isMsEdge(chromiumChannel),
"Skipping test for MS Edge in pre-release workflow",
);

const browserOSName = await getBrowserOs(page);
const modId = "@pixies/test/sidebar-links";
const modActivationPage = new ActivateModPage(page, extensionId, modId);
Expand Down Expand Up @@ -123,12 +136,7 @@ test("#8206: clicking links from the sidebar doesn't crash browser", async ({

await test.step("Clicking markdown text link", async () => {
/* eslint-disable playwright/no-conditional-in-test -- msedge and linux bug that causes the sidebar to close on clicking a link */
if (
browserOSName === "Linux" ||
[SupportedChannels.MSEDGE, SupportedChannels.MSEDGE_BETA].includes(
chromiumChannel,
)
) {
if (browserOSName === "Linux" || isMsEdge(chromiumChannel)) {
sideBarPage = await reopenSidebar(page, extensionId);
}
/* eslint-enable playwright/no-conditional-in-test */
Expand All @@ -143,12 +151,7 @@ test("#8206: clicking links from the sidebar doesn't crash browser", async ({

await test.step("Clicking react bootstrap link", async () => {
/* eslint-disable playwright/no-conditional-in-test -- msedge/linux bug */
if (
browserOSName === "Linux" ||
[SupportedChannels.MSEDGE, SupportedChannels.MSEDGE_BETA].includes(
chromiumChannel,
)
) {
if (browserOSName === "Linux" || isMsEdge(chromiumChannel)) {
sideBarPage = await reopenSidebar(page, extensionId);
}
/* eslint-enable playwright/no-conditional-in-test */
Expand All @@ -163,12 +166,7 @@ test("#8206: clicking links from the sidebar doesn't crash browser", async ({

await test.step("Clicking html renderer link", async () => {
/* eslint-disable playwright/no-conditional-in-test -- msedge/linux bug */
if (
browserOSName === "Linux" ||
[SupportedChannels.MSEDGE, SupportedChannels.MSEDGE_BETA].includes(
chromiumChannel,
)
) {
if (browserOSName === "Linux" || isMsEdge(chromiumChannel)) {
sideBarPage = await reopenSidebar(page, extensionId);
}
/* eslint-enable playwright/no-conditional-in-test */
Expand All @@ -183,12 +181,7 @@ test("#8206: clicking links from the sidebar doesn't crash browser", async ({

await test.step("Clicking embedded form link", async () => {
/* eslint-disable playwright/no-conditional-in-test -- msedge/linux bug */
if (
browserOSName === "Linux" ||
[SupportedChannels.MSEDGE, SupportedChannels.MSEDGE_BETA].includes(
chromiumChannel,
)
) {
if (browserOSName === "Linux" || isMsEdge(chromiumChannel)) {
sideBarPage = await reopenSidebar(page, extensionId);
}
/* eslint-enable playwright/no-conditional-in-test */
Expand Down
15 changes: 14 additions & 1 deletion end-to-end-tests/tests/regressions/welcomeStarterBricks.spec.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,12 +19,25 @@ import { test, expect } from "../../fixtures/testBase";
// @ts-expect-error -- https://youtrack.jetbrains.com/issue/AQUA-711/Provide-a-run-configuration-for-Playwright-tests-in-specs-with-fixture-imports-only
import { test as base } from "@playwright/test";
import { ActivateModPage } from "../../pageObjects/extensionConsole/modsPage";
import { getSidebarPage, runModViaQuickBar } from "../../utils";
import {
getSidebarPage,
runModViaQuickBar,
isMsEdge,
PRE_RELEASE_BROWSER_WORKFLOW_NAME,
} from "../../utils";

test("#8740: can view the starter mods on the pixiebrix.com/welcome page", async ({
page,
extensionId,
chromiumChannel,
}) => {
// FIXME: https://github.com/pixiebrix/pixiebrix-extension/issues/9125
test.skip(
process.env.GITHUB_WORKFLOW === PRE_RELEASE_BROWSER_WORKFLOW_NAME &&
isMsEdge(chromiumChannel),
"Skipping test for MS Edge in pre-release workflow",
);

const modId = "@e2e-testing/open-sidebar-via-quickbar";
const modActivationPage = new ActivateModPage(page, extensionId, modId);
await modActivationPage.goto();
Expand Down
19 changes: 19 additions & 0 deletions end-to-end-tests/utils.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,9 @@ import {
type BrowserContext,
} from "@playwright/test";
import { TOTP } from "otpauth";
import { type SupportedChannel, SupportedChannels } from "../playwright.config";

export const PRE_RELEASE_BROWSER_WORKFLOW_NAME = "Pre-release Browsers";

type AxeResults = Awaited<ReturnType<typeof AxeBuilder.prototype.analyze>>;

Expand Down Expand Up @@ -233,3 +236,19 @@ export function generateOTP(secret: string) {

return totp.generate();
}

export function isMsEdge(chromiumChannel: SupportedChannel): boolean {
return [SupportedChannels.MSEDGE, SupportedChannels.MSEDGE_BETA].includes(
chromiumChannel,
);
}

export function isChrome(chromiumChannel: SupportedChannel): boolean {
return [SupportedChannels.CHROME, SupportedChannels.CHROME_BETA].includes(
chromiumChannel,
);
}

export function isChromium(chromiumChannel: SupportedChannel): boolean {
return chromiumChannel === SupportedChannels.CHROMIUM;
}
Loading