Enable use of RuntimeVariant to exclude runtime tests #51522
Conversation
|
Tagging subscribers to this area: @directhex Issue DetailsCurrently, we can only exclude tests based on architecture, runtime flavor (mono/coreclr), and operating system. This change additionally allows the exclusions of tests based on runtime variants, such as interpreter, jit, or llvmaot. This is in draft while I work out the CI parts of it.
|
naricc
force-pushed
the
naricc/runtime-mode
branch
2 times, most recently
from
413b869
to
f342783
Compare
|
@naricc - while thinking about this, also consider the possibility of adding an extra refinement for differentiating Interpreter WASM v/s AOT WASM test runs. Quite likely the set of failures maybe different. |
| <!-- The following only fail in the mono interpreter, but we don't have a way to exclude based on scenario yet --> | ||
|
|
||
| <ExcludeList Include = "$(XunitTestBinBase)/JIT/jit64/opt/cse/HugeArray1/**"> | ||
| <!-- Known failures for mono runtime on *all* architectures/operating systems in interpreter runtime mode --> |
|
@SamMonoRT Yeah, I had considered that. For interpreter wasm we can already just look at the architecture and variant; same for aot. The mechanism is general enough that we should be able to use it for any of the jit/aot/aotllvm things we have, just by defining a new variant for each. It would be nice if I could create an enum for this, but there is no such facility in msbuild, or any way to statically check that the names agree with what it is in the build yml files. |
|
Can we surface this in |
|
I was trying https://gist.github.com/radical/dcbadb80909bd78fbabc4020a6018298#file-foo-diff-L74-L96, but that wouldn't be enough. |
naricc
force-pushed
the
naricc/runtime-mode
branch
from
d13f086
to
8b2db81
Compare
|
@radical Maybe. I am not sure what you had in mind exactly, adding a "GetRuntimeVariant" method to PlatformDetection? Currently this is only for the runtime tests, which have their own unique way of building and don't use PlatfromDetection. |
naricc
changed the title
Oh ok, I was mistaken then. I'll add something for the library tests. |
|
|
|
Looks like hte libraries failures are this issue: #52031 |
|
Also this one: https://github.com/dotnet/core-eng/issues/12969 |
|
Some how I didn't merge this. Re-opening. |
|
Don't see how your changes can cause Library/WASM lane failures @steveisok are these known issues and we can merge this PR ? |
ghost
locked as resolved and limited conversation to collaborators
Currently, we can only exclude tests based on architecture, runtime flavor (mono/coreclr), and operating system. This change additionally allows the exclusions of tests based on runtime variants, such as interpreter, jit, or llvmaot.
I haven't gotten a full green build out of this yet, but I believe it trying to run the right things now. So I am making it ready for review.
Fixes #49653