test: run e2e tests in parallel #674

sobolk · 2023-11-16T17:49:26Z

Issue #, if available:

Description of changes:

This PR enables parallelization in our e2e tests.

Runtime before:

Runtime after:

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

changeset-bot · 2023-11-16T17:49:29Z

🦋 Changeset detected

Latest commit: 1213a13

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package

Name	Type
@aws-amplify/integration-tests	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

packages/integration-tests/src/test-e2e/deployment.test.ts

…ent-tests

packages/integration-tests/src/test-e2e/deployment.test.ts

…ent-tests

Amplifiyer · 2023-11-16T22:03:52Z

packages/integration-tests/src/test-e2e/minimal_with_typescript_idioms.ts

+
+  createProject = async (e2eProjectDir: string): Promise<TestProjectBase> => {
+    const { projectName, projectRoot, projectAmplifyDir } =
+      await createEmptyAmplifyProject('typescript-idiom', e2eProjectDir);


you can use the this.name here?

Amplifiyer · 2023-11-16T22:09:12Z

packages/integration-tests/src/test-e2e/data_storage_auth_with_triggers.ts

can we move these files in a test-project-setup directory or something? Keep the test files only here

Amplifiyer · 2023-11-16T22:12:40Z

packages/integration-tests/src/test-e2e/test_project.ts

This file no longer returns test_projects or contains, perhaps it should be renamed test_project_creator_creator 🤣 ?

…ent-tests

sobolk · 2023-11-17T03:27:04Z

.github/actions/install_with_cache/action.yml

@@ -10,7 +10,7 @@ runs:
        path: |
          node_modules
          packages/**/node_modules
-        key: ${{ runner.os }}-${{ hashFiles('package-lock.json') }}
+        key: ${{ runner.os }}-${{ hashFiles('package-lock.json') }}-v2


This is needed to invalidate cache that's incompatible with larger worker and is leading to, will be reverted in subsequent PRs:

Prepare all required actions Getting action download info Download action repository 'actions/cache@885[2](https://github.com/aws-amplify/amplify-backend/actions/runs/6899407583/job/18770983822#step:4:2)2ab9f39a2ea568f7027eddc7d8d8bc9d59c8' (SHA:88522ab9f39a2ea568f7027eddc7d8d8bc9d59c8) Run ./.github/actions/restore_build_cache Prepare all required actions Getting action download info Run ./.github/actions/restore_install_cache Run actions/cache@88522ab9f[3](https://github.com/aws-amplify/amplify-backend/actions/runs/6899407583/job/18770983822#step:4:3)9a2ea568f7027eddc7d8d8bc9d59c8 Received 13[4](https://github.com/aws-amplify/amplify-backend/actions/runs/6899407583/job/18770983822#step:4:4)217728 of 24[5](https://github.com/aws-amplify/amplify-backend/actions/runs/6899407583/job/18770983822#step:4:6)825942 (54.[6](https://github.com/aws-amplify/amplify-backend/actions/runs/6899407583/job/18770983822#step:4:7)%), 126.2 MBs/sec Received 245825942 of 245825942 (100.0%), 133.0 MBs/sec Cache Size: ~234 MB (245825942 B) "C:\Program Files\Git\usr\bin\tar.exe" -xf C:/a/_temp/00c0e1[7](https://github.com/aws-amplify/amplify-backend/actions/runs/6899407583/job/18770983822#step:4:8)2-e349-4f14-92f9-10005a622fb[8](https://github.com/aws-amplify/amplify-backend/actions/runs/6899407583/job/18770983822#step:4:10)/cache.tzst -P -C C:/a/amplify-backend/amplify-backend --force-local --use-compress-program "zstd -d" Cache restored successfully Cache restored from key: Windows-418f36fb2696ff545b34d4f90f527534293b1ff5110a4729ea71303b9b53485d Run actions/cache@88522ab9f39a2ea568f7027eddc7d8d8bc9d59c8 Received 1342[17](https://github.com/aws-amplify/amplify-backend/actions/runs/6899407583/job/18770983822#step:4:20)728 of 167938[22](https://github.com/aws-amplify/amplify-backend/actions/runs/6899407583/job/18770983822#step:4:25)8 (79.9%), 114.7 MBs/sec Received 167938228 of 167938228 (100.0%), 116.9 MBs/sec Cache Size: ~160 MB (167938228 B) "C:\Program Files\Git\usr\bin\tar.exe" -xf C:/a/_temp/bb0ebe96-47ed-43e9-890f-ead3ad[23](https://github.com/aws-amplify/amplify-backend/actions/runs/6899407583/job/18770983822#step:4:26)ea8b/cache.tzst -P -C C:/a/amplify-backend/amplify-backend --force-local --use-compress-program "zstd -d" /usr/bin/tar: node_modules/@aws-amplify/auth-construct-alpha/lib: Cannot mkdir: No medium found /usr/bin/tar: node_modules/@aws-amplify/auth-construct-alpha/lib/defaults.d.ts.map: Cannot open: No medium found /usr/bin/tar: node_modules/@aws-amplify/auth-construct-alpha/lib/string_maps.d.ts.map: Cannot open: No medium found /usr/bin/tar: node_modules/@aws-amplify/auth-construct-alpha/lib/construct.js: Cannot open: No medium found /usr/bin/tar: node_modules/@aws-amplify/auth-construct-alpha/lib/string_maps.js: Cannot open: No medium found /usr/bin/tar: node_modules/@aws-amplify/auth-construct-alpha/lib/trigger_events.d.ts: Cannot open: No medium found /usr/bin/tar: node_modules/@aws-amplify/auth-construct-alpha/lib/index.js: Cannot open: No medium found /usr/bin/tar: node_modules/@aws-amplify/auth-construct-alpha/lib/trigger_events.js: Cannot open: No medium found

sobolk · 2023-11-17T03:28:56Z

.github/workflows/health_checks.yml

+        # Windows install must happen on the same worker size as subsequent jobs.
+        # Larger workers use different drive (C: instead of D:) to check out project and NPM installation
+        # creates file system links that include drive letter.
+        os: [ubuntu-latest, macos-latest, amplify-backend_windows-latest_8-core]


See https://github.com/aws-amplify/amplify-backend/pull/674/files#r1396662876

alharris-at

Approving, since I don't think the dup changeset has any negative impact.

alharris-at · 2023-11-17T15:53:01Z

.changeset/tall-turkeys-double.md

+---
+---


Looks like a duplicate changeset file?

yeah, I had to add non-empty to force local publish.
This is harmless and disappears during releases.

sdstolworthy · 2023-11-17T16:01:57Z

packages/integration-tests/src/test-e2e/deployment.test.ts

+          await testProject.tearDown(branchBackendIdentifier);
+        });
+
+        void it(`[${testProjectCreator.name}] deploys fully`, async () => {


nit: having project name here and on line 39 is repetitive

same for sandbox, but it doesn't seem to produce bad output, so I skipped this.

0618 · 2023-11-28T21:49:08Z

packages/integration-tests/src/test-e2e/create_amplify.test.ts

+
+void describe(
+  'create-amplify script',
+  { concurrency: testConcurrencyLevel },


Here's where the parallel magic happens. ref: https://nodejs.org/api/test.html#testname-options-fn

test: run e2e tests in parallel

fabf280

sobolk added the run-e2e Label that will include e2e tests in PR checks workflow label Nov 16, 2023

github-advanced-security bot found potential problems Nov 16, 2023

View reviewed changes

packages/integration-tests/src/test-e2e/deployment.test.ts Fixed Show fixed Hide fixed

packages/integration-tests/src/test-e2e/deployment.test.ts Fixed Show fixed Hide fixed

sobolk added 2 commits November 16, 2023 10:23

try that

201f7a8

Merge remote-tracking branch 'upstream/main' into parallelize-deploym…

3cacfd9

…ent-tests

github-advanced-security bot found potential problems Nov 16, 2023

View reviewed changes

packages/integration-tests/src/test-e2e/deployment.test.ts Fixed Show fixed Hide fixed

sobolk added 8 commits November 16, 2023 11:31

try that

9d3309d

try that

dace615

try that

3df4a38

try that

12e614c

Merge remote-tracking branch 'upstream/main' into parallelize-deploym…

d5729ca

…ent-tests

try that

7f930f4

try that

1a5a6c3

try that

c995bee

sobolk marked this pull request as ready for review November 16, 2023 21:37

sobolk requested review from edwardfoyle and Amplifiyer as code owners November 16, 2023 21:37

Amplifiyer reviewed Nov 16, 2023

View reviewed changes

pr feedback

ad66f7a

Amplifiyer previously approved these changes Nov 16, 2023

View reviewed changes

lets see

2fc810e

sobolk dismissed Amplifiyer’s stale review via 2fc810e November 16, 2023 23:37

sobolk added 5 commits November 16, 2023 18:48

add that

6f09266

Merge remote-tracking branch 'upstream/main' into parallelize-deploym…

da38c09

…ent-tests

try that?

6786258

try that?

08ad741

comment that

e9a4351

sobolk commented Nov 17, 2023

View reviewed changes

sobolk added 2 commits November 16, 2023 19:45

that!

b2e67d4

bump timeout

1213a13

alharris-at approved these changes Nov 17, 2023

View reviewed changes

sdstolworthy reviewed Nov 17, 2023

View reviewed changes

sdstolworthy approved these changes Nov 17, 2023

View reviewed changes

Amplifiyer approved these changes Nov 17, 2023

View reviewed changes

sobolk merged commit fe2d96f into main Nov 17, 2023
22 checks passed

sobolk deleted the parallelize-deployment-tests branch November 17, 2023 16:13

0618 reviewed Nov 28, 2023

View reviewed changes

This was referenced Dec 4, 2023

fix: e2e tests #763

Closed

fix: e2e tests #771

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

test: run e2e tests in parallel #674

test: run e2e tests in parallel #674

sobolk commented Nov 16, 2023 •

edited

Loading

changeset-bot bot commented Nov 16, 2023 •

edited

Loading

Amplifiyer Nov 16, 2023

Amplifiyer Nov 16, 2023

Amplifiyer Nov 16, 2023

sobolk Nov 17, 2023 •

edited

Loading

sobolk Nov 17, 2023

alharris-at left a comment

alharris-at Nov 17, 2023

sobolk Nov 17, 2023

sdstolworthy Nov 17, 2023

sobolk Nov 17, 2023

0618 Nov 28, 2023

		---
		---

test: run e2e tests in parallel #674

test: run e2e tests in parallel #674

Conversation

sobolk commented Nov 16, 2023 • edited Loading

changeset-bot bot commented Nov 16, 2023 • edited Loading

🦋 Changeset detected

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sobolk Nov 17, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

alharris-at left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sobolk commented Nov 16, 2023 •

edited

Loading

changeset-bot bot commented Nov 16, 2023 •

edited

Loading

sobolk Nov 17, 2023 •

edited

Loading