[wasm] Split `runtime-wasm` pipeline into `runtime-wasm-libtests`, and `runtime-wasm-non-libtests` #74859

radical · 2022-08-31T07:57:21Z

Adds two new pipelines:

runtime-wasm-libtests - runs all the wasm library tests
runtime-wasm-non-libtests - runs all the wasm non-library tests like WBT, debugger etc.

And runtime-wasm runs the combination of the two above.

For all 3 pipelines, jobs are run only if they weren't run on the default (runtime) pipeline

Which pipeline should I run?

if runtime didn't run the tests that would be useful for testing the changes, then:
- If the changes only relate to libraries, the runtime, then use runtime-wasm-libtests
- If the changes only affect Wasm.Build.Tests, or debugger tests, or runtime tests, then use runtime-wasm-non-libtests
- Use runtime-wasm when you want to run all the wasm tests

Updated CI matrix for wasm

PR:

only-pc means only on relevant path changes

Run by default

runtime runs jobs only when relevant paths change. And for AOT, only smoke tests are run.

.	runtime
libtests	linux+windows: all, only-pc
libtests eat	linux+windows: smoke, only-pc
libtests aot	linux+windows: smoke, only-pc
high resource aot	none
Wasm.Build.Tests	linux+windows: only-pc
Debugger tests	linux+windows: only-pc
Runtime tests	linux+windows: only-pc

Run manually with `/azp run ..`

runtime-wasm* pipelines are triggered manually, and they only run the jobs that would not run on any default pipelines based on path changes.
The AOT jobs run only smoke tests on runtime, and on runtime-wasm* pipelines all the AOT tests are run.

.	runtime-wasm	runtime-wasm-libtests	runtime-wasm-non-libtests
libtests	linux+windows: all	linux+windows: all	none
libtests eat	linux: all	linux: all	none
libtests aot	linux+windows: all	linux+windows: all	none
high resource aot	linux+windows: all	linux+windows: all	none
Wasm.Build.Tests	linux+windows	none	linux+windows
Debugger tests	linux+windows	none	linux+windows
Runtime tests	linux	none	linux
Perftrace	linux: all tests	linux: all tests
Multi-thread	linux: all tests	linux: all tests

runtime-extra-platforms does not run any wasm jobs on PRs
high resource aot runs a few specific library tests with AOT, that require more memory to AOT.

Rolling build (twice a day):

runtime runs all the wasm jobs, but AOT still only runs smoke tests.
runtime-extra-platforms also runs by default. And it runs only the cases not covered by runtime.

.	runtime	runtime-extra-platforms (always run)
libtests	linux+windows: all	none
libtests eat	linux: all	none
libtests aot	linux+windows: smoke	linux+windows: all
high resource aot	none	linux+windows: all

Wasm.Build.Tests	linux+windows	none
Debugger tests	linux+windows	none
Runtime tests	linux	none
Perftrace	linux: build only	none
Multi-thread	linux: build only	none

high resource aot runs a few specific library tests with AOT, that require more memory to AOT.

ghost · 2022-08-31T07:57:28Z

Tagging subscribers to 'arch-wasm': @lewing
See info in area-owners.md if you want to be subscribed.

Issue Details

null

Author:	radical
Assignees:	-
Labels:	`arch-wasm`, `area-Infrastructure-mono`
Milestone:	-

radical · 2022-08-31T08:46:35Z

/azp run runtime-wasm

azure-pipelines · 2022-08-31T08:46:44Z

Azure Pipelines successfully started running 1 pipeline(s).

radical · 2022-08-31T08:53:14Z

/azp run runtime-wasm-libtests

azure-pipelines · 2022-08-31T08:53:30Z

Azure Pipelines successfully started running 1 pipeline(s).

radical · 2022-08-31T08:54:49Z

/azp run runtime-wasm-libtests,runtime-wasm

azure-pipelines · 2022-08-31T08:55:01Z

Azure Pipelines successfully started running 2 pipeline(s).

radical · 2022-08-31T09:11:21Z

/azp run runtime-wasm-libtests,runtime-wasm,runtime-wasm-non-libtests

azure-pipelines · 2022-08-31T09:11:41Z

Azure Pipelines successfully started running 2 pipeline(s), but failed to run 1 pipeline(s).

radical · 2022-08-31T09:13:03Z

/azp run runtime-wasm-libtests,runtime-wasm,runtime-wasm-non-libtests

azure-pipelines · 2022-08-31T09:13:18Z

Azure Pipelines successfully started running 3 pipeline(s).

radical · 2022-08-31T09:43:37Z

/azp run runtime-wasm-libtests,runtime-wasm,runtime-wasm-non-libtests

azure-pipelines · 2022-08-31T09:43:51Z

Azure Pipelines successfully started running 3 pipeline(s).

radical · 2022-08-31T09:44:43Z

/azp run runtime-extra-platforms

azure-pipelines · 2022-08-31T09:44:55Z

Azure Pipelines successfully started running 1 pipeline(s).

radical · 2022-08-31T10:09:57Z

/azp run runtime-wasm-libtests,runtime-wasm,runtime-wasm-non-libtests,runtime-extra-platforms

azure-pipelines · 2022-08-31T10:10:20Z

Azure Pipelines successfully started running 4 pipeline(s).

radical · 2022-08-31T10:30:41Z

/azp run runtime-wasm-libtests,runtime-wasm,runtime-wasm-non-libtests,runtime-extra-platforms

azure-pipelines · 2022-08-31T10:31:01Z

Azure Pipelines successfully started running 4 pipeline(s).

radical · 2022-08-31T10:43:07Z

/azp run runtime-wasm-libtests,runtime-wasm,runtime-wasm-non-libtests,runtime-extra-platforms

azure-pipelines · 2022-08-31T10:43:29Z

Azure Pipelines successfully started running 4 pipeline(s).

…libtests

akoeplinger

LGTM. I don't love the runtime-wasm-non-libtests name but I couldn't come up with a better name either :)

radical · 2022-08-31T16:23:45Z

LGTM. I don't love the runtime-wasm-non-libtests name but I couldn't come up with a better name either :)

And we might end up doing more splits, depending on usage :)

…r by mistake

radical added arch-wasm WebAssembly architecture area-Infrastructure-mono labels Aug 31, 2022

radical added NO-MERGE The PR is not ready for merge yet (see discussion for detailed reasons) NO-REVIEW Experimental/testing PR, do NOT review it labels Aug 31, 2022

ghost assigned radical Aug 31, 2022

radical force-pushed the wasm-libtests-ci branch 3 times, most recently from a4bf18d to 9669e83 Compare August 31, 2022 09:42

radical force-pushed the wasm-libtests-ci branch from 9669e83 to 4c4bc3d Compare August 31, 2022 09:50

radical force-pushed the wasm-libtests-ci branch from 6e701d7 to 7ad8e68 Compare August 31, 2022 10:25

radical changed the title ~~[wasm] Split runtime-wasm pipeline~~ [wasm] Split runtime-wasm pipeline into runtime-wasm-libtests, and runtime-wasm-non-libtests Aug 31, 2022

radical force-pushed the wasm-libtests-ci branch from e93f943 to 7ad8e68 Compare August 31, 2022 10:53

radical requested review from lewing, steveisok and akoeplinger August 31, 2022 10:54

radical marked this pull request as ready for review August 31, 2022 10:55

radical requested a review from pavelsavara as a code owner August 31, 2022 10:55

radical removed NO-MERGE The PR is not ready for merge yet (see discussion for detailed reasons) NO-REVIEW Experimental/testing PR, do NOT review it labels Aug 31, 2022

radical added 3 commits August 31, 2022 07:15

CI: move isDefaultPipeline to variables.yml

5a9819f

CI: add support for running wasm jobs as only libtests, and only non-…

0688e04

…libtests

Update README with CI matrix for wasm

7cf965a

radical force-pushed the wasm-libtests-ci branch from 7ad8e68 to 7cf965a Compare August 31, 2022 11:15

radical mentioned this pull request Aug 31, 2022

[wasm] CI improvements to improve test coverage, and reduce load #74538

Open

23 tasks

akoeplinger approved these changes Aug 31, 2022

View reviewed changes

radical added 2 commits August 31, 2022 12:47

[wasm] Fix timeout for runtime tests to 180, which was changed earlie…

4f51e01

…r by mistake

Merge remote-tracking branch 'origin/main' into wasm-libtests-ci-2

b220c9a

radical merged commit c30cfbb into dotnet:main Aug 31, 2022

radical deleted the wasm-libtests-ci branch August 31, 2022 19:39

radekdoulik mentioned this pull request Sep 1, 2022

[wasm] Speedup Vector128.Shuffle with SIMD #74872

Merged

steveisok mentioned this pull request Sep 2, 2022

Add more granular pipelines for mobile configurations #75002

Closed

5 tasks

ghost locked as resolved and limited conversation to collaborators Oct 1, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[wasm] Split `runtime-wasm` pipeline into `runtime-wasm-libtests`, and `runtime-wasm-non-libtests` #74859

[wasm] Split `runtime-wasm` pipeline into `runtime-wasm-libtests`, and `runtime-wasm-non-libtests` #74859

radical commented Aug 31, 2022 •

edited

Loading

ghost commented Aug 31, 2022

radical commented Aug 31, 2022

azure-pipelines bot commented Aug 31, 2022

radical commented Aug 31, 2022

azure-pipelines bot commented Aug 31, 2022

radical commented Aug 31, 2022

azure-pipelines bot commented Aug 31, 2022

radical commented Aug 31, 2022

azure-pipelines bot commented Aug 31, 2022

radical commented Aug 31, 2022

azure-pipelines bot commented Aug 31, 2022

radical commented Aug 31, 2022

azure-pipelines bot commented Aug 31, 2022

radical commented Aug 31, 2022

azure-pipelines bot commented Aug 31, 2022

radical commented Aug 31, 2022

azure-pipelines bot commented Aug 31, 2022

radical commented Aug 31, 2022

azure-pipelines bot commented Aug 31, 2022

radical commented Aug 31, 2022

azure-pipelines bot commented Aug 31, 2022

akoeplinger left a comment

radical commented Aug 31, 2022

[wasm] Split runtime-wasm pipeline into runtime-wasm-libtests, and runtime-wasm-non-libtests #74859

[wasm] Split runtime-wasm pipeline into runtime-wasm-libtests, and runtime-wasm-non-libtests #74859

Conversation

radical commented Aug 31, 2022 • edited Loading

Which pipeline should I run?

Updated CI matrix for wasm

PR:

Run by default

Run manually with /azp run ..

Rolling build (twice a day):

ghost commented Aug 31, 2022

radical commented Aug 31, 2022

azure-pipelines bot commented Aug 31, 2022

radical commented Aug 31, 2022

azure-pipelines bot commented Aug 31, 2022

radical commented Aug 31, 2022

azure-pipelines bot commented Aug 31, 2022

radical commented Aug 31, 2022

azure-pipelines bot commented Aug 31, 2022

radical commented Aug 31, 2022

azure-pipelines bot commented Aug 31, 2022

radical commented Aug 31, 2022

azure-pipelines bot commented Aug 31, 2022

radical commented Aug 31, 2022

azure-pipelines bot commented Aug 31, 2022

radical commented Aug 31, 2022

azure-pipelines bot commented Aug 31, 2022

radical commented Aug 31, 2022

azure-pipelines bot commented Aug 31, 2022

radical commented Aug 31, 2022

azure-pipelines bot commented Aug 31, 2022

akoeplinger left a comment

Choose a reason for hiding this comment

radical commented Aug 31, 2022

[wasm] Split `runtime-wasm` pipeline into `runtime-wasm-libtests`, and `runtime-wasm-non-libtests` #74859

[wasm] Split `runtime-wasm` pipeline into `runtime-wasm-libtests`, and `runtime-wasm-non-libtests` #74859

radical commented Aug 31, 2022 •

edited

Loading

Run manually with `/azp run ..`