[Unix|Windows|AllArch] Unify exclusion list and enable runtest.py #19213

jashook · 2018-07-31T16:30:48Z

The change has a few different pieces. I have kept the commit history in an effort to potentially help review the change. The change includes, but is not limited to:

Changing build-test.sh to always build the xunit wrappers. Before it would drop a token and check the existence of the token.
Unify x64 linux/OSX/Windows excludes into one file, issues.targets. I have added different locations in the file which show where to put excludes.
Remove all target specific aspects of issues.targets, all tests are excluded now via wildcard, this allows expanding to .cmd and .sh based on the build platform
Unify path separators to forward slash(/) in issues.targets to support both platforms
Clean up issues.targets by removing long standing exclude tests, specifically tests that have been excluded due to missing features like rva_statics
Add DisableProjectBuild to tests which have been removed from issues.targets
Conditionally add DisableProjectBuild to tests which have been marked as unsupported on unix. This is mostly a port of the unsupportedOnUnix.txt list. Instead of excluding the tests, unix will simply not build them. ~~Note that this refcount dependency on running tests on the same platform as where they were built~~
All exclusions ported to issues.targets for linux targets.
Expand runtest.py, this includes simple issues that made it past the original CR. In addition it adds more optional features to help with inner loop dev work such as: creating a repro folder under bin/repro/.. which sets up the env and calls the failing test. In addition a launch.json will now be created under bin/repro/.. which can be used to easily debug using vscode. More logging, such as printing failures, longest running tests ect.
Initial excludes ported for arm64 windows

What this PR does not include:

Arm and arm64, both windows and unix, support for runtest.py. This is currently blocked on tools acquisition using init-tools.cmd/init-tools.sh, these scripts will currently pull down the x64 tools and we are unable to run msbuild on those platforms.

Arm64 linux, armhf unix excludes and enables running runtest.sh for these targets.
arm64 windows and arm32 windows excludes and enables running runtest.cmd on arm64 targets
init-tools.sh changes to pull armhf and aarch64 dotnetcli
init-tools.cmd changes to pull x86 packages for dotnetcli for arm64 windows
runtest.cmd for almost all scenarios will call runtest.py
runtest.sh for almsot all scenarios will call runtest.py
Removes all logic for running tests using runtest.sh

Notes

In order to support different return codes used by runtest.sh and the xunit console runner. I have conditionally included different return codes for the test bash wrappers. This means that for our ci jobs which build bash targets on windows, unix testing will continue to succeed using runtest.sh. However, this will break all tests run using runtest.sh using a test build that is built on unix. Basically runtest.sh will be deprecated with this change, and runtest.py will be the only supported use case.

With this change, only the msbuild xunit console runner will be supported to run tests. Runtest.sh will now call runtest.py, which is a wrapper for runtest.proj, which is a wrapper (keeps going I know...) for calling the xunit console runner.

jashook · 2018-07-31T16:32:33Z

/cc @dotnet/jit-contrib @RussKeldorph please feel free to include any others.

jashook · 2018-07-31T18:32:45Z

@dotnet-bot test OSX10.12 x64 Checked Innerloop Build and Test
@dotnet-bot test Windows_NT arm64 Cross Checked Innerloop Build and Test

jashook · 2018-07-31T19:45:39Z

Based on suggestions, I will change runtest.sh to use runtest.py if on x64. Will work on that change later tonight.

AaronRobinsonMSFT · 2018-08-01T06:17:15Z

Please also update Documentation/building/test-configuration.md regarding test exclusion. Thanks.

sandreenko

LGTM

sandreenko · 2018-07-31T22:57:38Z

tests/runtest.py

+            print "Using default location for test_native_bin_location."
+            test_native_bin_location = os.path.join(os.path.join(coreclr_repo_location, "bin", "obj", "%s.%s.%s" % (host_os, arch, build_type), "tests"))
+            print "Native bin location: %s" % test_native_bin_location
+            print

    if host_os != "Windows_NT":


you can merge it with the previous block.

sbomer · 2018-08-01T18:25:26Z

tests/runtest.py

+# come from issues.targets. If there is a jit stress or gc stress exclude, 
+# please add GCStressIncompatible or JitOptimizationSensitive to the test's 
+# ilproj or csproj.
+#


The GCStressIncompatible and JitOptimizationSensitive excludes should probably be called out in docs too since this isn't very discoverable.

sbomer · 2018-08-01T18:26:35Z

tests/runtest.py

+# The xunit runner currently relies on tests being built on the same host as the
+# target platform. This requires all tests run on linux x64 to be built by the
+# same platform and arch. If this is not done, the tests will run correctly;
+# however, excpect failures due to incorrect exclusions in the xunit 


excpect -> expect

sbomer · 2018-08-01T18:27:11Z

tests/runtest.py

+#
+# The xunit runner currently relies on tests being built on the same host as the
+# target platform. This requires all tests run on linux x64 to be built by the
+# same platform and arch. If this is not done, the tests will run correctly;


Won't this cause the official build test run to fail? If I recall, these would build wrappers on windows and run them on linux.

This is not necessarily true for helix, which currently gets around this issue by setting an env variable BuiltAgainstPackages.

sbomer · 2018-08-01T18:27:27Z

tests/runtest.py

+# however, excpect failures due to incorrect exclusions in the xunit 
+# wrappers setup at build time.
+#
+# Note that for linux targets the native componants to the tests are still built


componants -> components

sbomer · 2018-08-01T18:27:41Z

tests/runtest.py

+#
+# Note that for linux targets the native componants to the tests are still built
+# by the product build. This requires all native components to be either copied
+# into the Core_Root directory or the test's managed directory. The later is


later -> latter

sbomer · 2018-08-01T18:28:39Z

tests/runtest.py

+If you are running tests on a different target than the host that built, the
+native tests components must be copied from:
+bin/obj/<Host>.<Arch>.<BuildType/tests to the target. If the location is not
+standard please pass the -test_native_bin_location flag to the script.""")



Same comments as above apply to this description.

sbomer · 2018-08-01T18:29:46Z

tests/runtest.py

+REM with the appropriate environment set and the correct arch and build_type
+REM passed.
+REM
+REM ============================================================================


This looks promising! It'll be great if this makes it easy to repro failures. Could you add some documentation on how this would be used?

I will address this as a separate PR

sbomer · 2018-08-01T18:31:23Z

tests/runtest.py

+# with the appropriate environment set and the correct arch and build_type
+# passed.
+#
+# ============================================================================


Could you maybe avoid duplicating all of this text? Also, this one should say create_bash_wrapper

Did not want to spend too much time on this, and could not think of a super simple way to avoid this. Worth living with it for now.

sbomer · 2018-08-01T18:31:49Z

tests/runtest.py

@@ -159,7 +477,7 @@ def call_msbuild(coreclr_repo_location,

    Args:
        coreclr_repo_location(str)  : path to coreclr repo
-        msbuild_location(str)       : path to msbuild
+        dotnetcli_location(str)     : path to the dotnet cli in the tools dir


I like this change!

sbomer · 2018-08-01T18:38:53Z

tests/tests.targets

@@ -65,7 +65,7 @@
      <XunitCommandLine>$(CorerunExecutable) $(XunitConsoleRunner) @(TestAssemblies->'%(Identity)', ' ') $(XunitArgs)</XunitCommandLine>
    </PropertyGroup>

-    <Error Condition="$(XunitCommandLine.Length) > 8191" Text="Xunit command line is too long." />
+    <Error Condition="$(XunitCommandLine.Length) > 8191 and '$(RunningOnUnix)' != 'true'" Text="Xunit command line is too long." />


Thanks for fixing this!

jashook · 2018-08-02T15:55:21Z

Adding no merge for now. With more recent changes to use runtest.py by default in runtest.sh/cmd I will be doing much more in depth testing before merging. Thanks for the reviews!

AaronRobinsonMSFT · 2018-08-02T16:38:12Z

Adding no merge for now. With more recent changes to use runtest.py by default in runtest.sh/cmd I will be doing much more in depth testing before merging. Thanks for the reviews!

Please don't forget to update documentation.

jashook · 2018-08-14T18:52:52Z

Many more changes have been posted. With these changes pri1 tests and gc stress (the longest tests I could think of) now run cleanly on all targets using runtest.py.

Runtest.sh has been gutted, it is now a simple wrapper around runtest.py. Runtest.cmd will keep a large amount of existing logic, to be addressed in the future. I have added a little documentation, but there still needs to be more. I have listed the work to still be done below:

~~Verify clean ci with this change (currently expecting issues)~~
~~Add more documentation.~~
Merge change
Modify CI to call runtest.sh/cmd without deprecated arguments
Maybe cleanup issues.targets

jashook · 2018-08-15T20:29:46Z

@dotnet-bot test OSX10.12 x64 Checked Innerloop Build and Test

jashook · 2018-08-16T15:57:34Z

@dotnet-bot test OSX10.12 x64 Checked Innerloop Build and Test

jashook · 2018-08-16T19:05:11Z

Rebased

jashook · 2018-08-17T14:27:03Z

@dotnet-bot test this please

jashook · 2018-08-17T18:30:07Z

Rebased

jashook · 2018-08-17T19:51:21Z

Squashed and rebased.

jashook · 2018-08-23T17:57:52Z

@dotnet-bot test Ubuntu arm Cross Checked r2r Build and Test
@dotnet-bot test Ubuntu arm Cross Release normal Build and Test
@dotnet-bot test OSX10.12 x64 Checked Build and Test
@dotnet-bot test Ubuntu16.04 x64 Checked Build
@dotnet-bot test Ubuntu arm Cross Checked corefx_baseline Build and Test
@dotnet-bot test Windows_NT x64 Checked minopts
@dotnet-bot test Windows_NT x64_arm64_altjit Checked normal
@dotnet-bot test Windows_NT x64_arm64_altjit Checked r2r
@dotnet-bot test Ubuntu x64 Checked Build and Test
@dotnet-bot test Windows_NT x64 Checked r2r
@dotnet-bot test Windows_NT x86 Checked Build and Test
@dotnet-bot test Windows_NT x86 Checked gcstress0x3
@dotnet-bot test Windows_NT x86 Checked gcstress0xC
@dotnet-bot test Windows_NT x64 Checked gcstress0x3
@dotnet-bot test Windows_NT x64 Checked gcstress0xC
@dotnet-bot test Ubuntu x64 Checked r2r
@dotnet-bot test OSX10.12 x64 Checked r2r
@dotnet-bot test CentOS7.1 x64 Checked r2r
@dotnet-bot test Windows_NT x86 Checked zapdisable
@dotnet-bot test Windows_NT x86 Checked r2r_jitstressregs8
@dotnet-bot test Ubuntu x64 Checked gcstress0xc
@dotnet-bot test OSX10.12 x64 Checked Innerloop Build and Test

jashook · 2018-08-23T18:00:22Z

@BruceForstall ptal

jashook · 2018-08-23T19:30:05Z

@dotnet-bot test Ubuntu arm Cross Checked r2r Build and Test
@dotnet-bot test Ubuntu arm Cross Release normal Build and Test
@dotnet-bot test OSX10.12 x64 Checked Build and Test
@dotnet-bot test Ubuntu16.04 x64 Checked Build
@dotnet-bot test Ubuntu arm Cross Checked corefx_baseline Build and Test
@dotnet-bot test Windows_NT x64 Checked minopts
@dotnet-bot test Windows_NT x64_arm64_altjit Checked normal
@dotnet-bot test Windows_NT x64_arm64_altjit Checked r2r
@dotnet-bot test Ubuntu x64 Checked Build and Test
@dotnet-bot test Windows_NT x64 Checked r2r
@dotnet-bot test Windows_NT x86 Checked Build and Test
@dotnet-bot test Windows_NT x86 Checked gcstress0x3
@dotnet-bot test Windows_NT x86 Checked gcstress0xC
@dotnet-bot test Windows_NT x64 Checked gcstress0x3
@dotnet-bot test Windows_NT x64 Checked gcstress0xC
@dotnet-bot test Ubuntu x64 Checked r2r
@dotnet-bot test OSX10.12 x64 Checked r2r
@dotnet-bot test CentOS7.1 x64 Checked r2r
@dotnet-bot test Windows_NT x86 Checked zapdisable
@dotnet-bot test Windows_NT x86 Checked r2r_jitstressregs8
@dotnet-bot test Ubuntu x64 Checked gcstress0xc
@dotnet-bot test OSX10.12 x64 Checked Innerloop Build and Test

jashook · 2018-08-23T20:54:53Z

@dotnet-bot test Ubuntu arm Cross Checked r2r Build and Test
@dotnet-bot test Ubuntu arm Cross Release normal Build and Test
@dotnet-bot test OSX10.12 x64 Checked Build and Test
@dotnet-bot test Ubuntu16.04 x64 Checked Build
@dotnet-bot test Ubuntu arm Cross Checked corefx_baseline Build and Test
@dotnet-bot test Windows_NT x64 Checked minopts
@dotnet-bot test Windows_NT x64_arm64_altjit Checked normal
@dotnet-bot test Windows_NT x64_arm64_altjit Checked r2r
@dotnet-bot test Ubuntu x64 Checked Build and Test
@dotnet-bot test Windows_NT x64 Checked r2r
@dotnet-bot test Windows_NT x86 Checked Build and Test
@dotnet-bot test Windows_NT x86 Checked gcstress0x3
@dotnet-bot test Windows_NT x86 Checked gcstress0xC
@dotnet-bot test Windows_NT x64 Checked gcstress0x3
@dotnet-bot test Windows_NT x64 Checked gcstress0xC
@dotnet-bot test Ubuntu x64 Checked r2r
@dotnet-bot test OSX10.12 x64 Checked r2r
@dotnet-bot test CentOS7.1 x64 Checked r2r
@dotnet-bot test Windows_NT x86 Checked zapdisable
@dotnet-bot test Windows_NT x86 Checked r2r_jitstressregs8
@dotnet-bot test Ubuntu x64 Checked gcstress0xc
@dotnet-bot test OSX10.12 x64 Checked Innerloop Build and Test

jashook · 2018-08-24T04:35:13Z

@dotnet-bot test Ubuntu arm Cross Checked r2r Build and Test
@dotnet-bot test Ubuntu arm Cross Release normal Build and Test
@dotnet-bot test OSX10.12 x64 Checked Build and Test
@dotnet-bot test Ubuntu16.04 x64 Checked Build
@dotnet-bot test Ubuntu arm Cross Checked corefx_baseline Build and Test
@dotnet-bot test Windows_NT x64 Checked minopts
@dotnet-bot test Windows_NT x64_arm64_altjit Checked normal
@dotnet-bot test Windows_NT x64_arm64_altjit Checked r2r
@dotnet-bot test Ubuntu x64 Checked Build and Test
@dotnet-bot test Windows_NT x64 Checked r2r
@dotnet-bot test Windows_NT x86 Checked Build and Test
@dotnet-bot test Windows_NT x86 Checked gcstress0x3
@dotnet-bot test Windows_NT x86 Checked gcstress0xC
@dotnet-bot test Windows_NT x64 Checked gcstress0x3
@dotnet-bot test Windows_NT x64 Checked gcstress0xC
@dotnet-bot test Ubuntu x64 Checked r2r
@dotnet-bot test OSX10.12 x64 Checked r2r
@dotnet-bot test CentOS7.1 x64 Checked r2r
@dotnet-bot test Windows_NT x86 Checked zapdisable
@dotnet-bot test Windows_NT x86 Checked r2r_jitstressregs8
@dotnet-bot test Ubuntu x64 Checked gcstress0xc
@dotnet-bot test OSX10.12 x64 Checked Innerloop Build and Test

jashook · 2018-08-24T04:39:17Z

@dotnet-bot test Ubuntu arm Cross Checked r2r Build and Test
@dotnet-bot test Ubuntu arm Cross Release normal Build and Test
@dotnet-bot test OSX10.12 x64 Checked Build and Test
@dotnet-bot test Ubuntu16.04 x64 Checked Build
@dotnet-bot test Ubuntu arm Cross Checked corefx_baseline Build and Test
@dotnet-bot test Windows_NT x64 Checked minopts
@dotnet-bot test Windows_NT x64_arm64_altjit Checked normal
@dotnet-bot test Windows_NT x64_arm64_altjit Checked r2r
@dotnet-bot test Ubuntu x64 Checked Build and Test
@dotnet-bot test Windows_NT x64 Checked r2r
@dotnet-bot test Windows_NT x86 Checked Build and Test
@dotnet-bot test Windows_NT x86 Checked gcstress0x3
@dotnet-bot test Windows_NT x86 Checked gcstress0xC
@dotnet-bot test Windows_NT x64 Checked gcstress0x3
@dotnet-bot test Windows_NT x64 Checked gcstress0xC
@dotnet-bot test Ubuntu x64 Checked r2r
@dotnet-bot test OSX10.12 x64 Checked r2r
@dotnet-bot test CentOS7.1 x64 Checked r2r
@dotnet-bot test Windows_NT x86 Checked zapdisable
@dotnet-bot test Windows_NT x86 Checked r2r_jitstressregs8
@dotnet-bot test Ubuntu x64 Checked gcstress0xc
@dotnet-bot test OSX10.12 x64 Checked Innerloop Build and Test

jashook · 2018-08-24T14:46:48Z

@dotnet-bot test Ubuntu arm Cross Checked r2r Build and Test
@dotnet-bot test Ubuntu arm Cross Release normal Build and Test
@dotnet-bot test OSX10.12 x64 Checked Build and Test
@dotnet-bot test Ubuntu16.04 x64 Checked Build
@dotnet-bot test Ubuntu arm Cross Checked corefx_baseline Build and Test
@dotnet-bot test Windows_NT x64 Checked minopts
@dotnet-bot test Windows_NT x64_arm64_altjit Checked normal
@dotnet-bot test Windows_NT x64_arm64_altjit Checked r2r
@dotnet-bot test Ubuntu x64 Checked Build and Test
@dotnet-bot test Windows_NT x64 Checked r2r
@dotnet-bot test Windows_NT x86 Checked Build and Test
@dotnet-bot test Windows_NT x86 Checked gcstress0x3
@dotnet-bot test Windows_NT x86 Checked gcstress0xC
@dotnet-bot test Windows_NT x64 Checked gcstress0x3
@dotnet-bot test Windows_NT x64 Checked gcstress0xC
@dotnet-bot test Ubuntu x64 Checked r2r
@dotnet-bot test OSX10.12 x64 Checked r2r
@dotnet-bot test CentOS7.1 x64 Checked r2r
@dotnet-bot test Windows_NT x86 Checked zapdisable
@dotnet-bot test Windows_NT x86 Checked r2r_jitstressregs8
@dotnet-bot test Ubuntu x64 Checked gcstress0xc
@dotnet-bot test OSX10.12 x64 Checked Innerloop Build and Test

jashook · 2018-08-24T14:49:11Z

@dotnet-bot test Ubuntu arm Cross Checked r2r Build and Test
@dotnet-bot test Ubuntu arm Cross Release normal Build and Test
@dotnet-bot test OSX10.12 x64 Checked Build and Test
@dotnet-bot test Ubuntu16.04 x64 Checked Build
@dotnet-bot test Ubuntu arm Cross Checked corefx_baseline Build and Test
@dotnet-bot test Windows_NT x64 Checked minopts
@dotnet-bot test Windows_NT x64_arm64_altjit Checked normal
@dotnet-bot test Windows_NT x64_arm64_altjit Checked r2r
@dotnet-bot test Ubuntu x64 Checked Build and Test
@dotnet-bot test Windows_NT x64 Checked r2r
@dotnet-bot test Windows_NT x86 Checked Build and Test
@dotnet-bot test Windows_NT x86 Checked gcstress0x3
@dotnet-bot test Windows_NT x86 Checked gcstress0xC
@dotnet-bot test Windows_NT x64 Checked gcstress0x3
@dotnet-bot test Windows_NT x64 Checked gcstress0xC
@dotnet-bot test Ubuntu x64 Checked r2r
@dotnet-bot test OSX10.12 x64 Checked r2r
@dotnet-bot test CentOS7.1 x64 Checked r2r
@dotnet-bot test Windows_NT x86 Checked zapdisable
@dotnet-bot test Windows_NT x86 Checked r2r_jitstressregs8
@dotnet-bot test Ubuntu x64 Checked gcstress0xc
@dotnet-bot test OSX10.12 x64 Checked Innerloop Build and Test

Change build-test.sh to always build the xunit wrappers. Before it would drop a token and check the existence of the token. Unify x64 linux/OSX/Windows excludes into one file, issues.targets. I have added different locations in the file which show where to put excludes. Remove all target specific aspects of issues.targets, all tests are excluded now via wildcard, this allows expanding to .cmd and .sh based on the build platform Unify path separators to forward slash(/) in issues.targets to support both platforms Clean up issues.targets by removing long standing exclude tests, specifically tests that have been excluded due to missing features like rva_statics Add DisableProjectBuild to tests which have been removed from issues.targets Conditionally add DisableProjectBuild to tests which have been marked as unsupported on unix. This is mostly a port of the unsupportedOnUnix.txt list. Instead of excluding the tests, unix will simply not build them. Note that this refcount dependency on running tests on the same platform as where they were built All exclusions ported to issues.targets for linux targets. Expand runtest.py, this includes simple issues that made it past the original CR. In addition it adds more optional features to help with inner loop dev work such as: creating a repro folder under bin/repro/.. which sets up the env and calls the failing test. In addition a launch.json will now be created under bin/repro/.. which can be used to easily debug using vscode. More logging, such as printing failures, longest running tests ect. Initial excludes ported for arm64 windows Arm64 linux, armhf unix excludes and enables running runtest.sh for these targets. arm64 windows and arm32 windows excludes and enables running runtest.cmd on arm64 targets init-tools.sh changes to pull armhf and aarch64 dotnetcli init-tools.cmd changes to pull x86 packages for dotnetcli for arm64 windows runtest.cmd for almost all scenarios will call runtest.py runtest.sh for almsot all scenarios will call runtest.py Removes all logic for running tests using runtest.sh

jashook · 2018-08-24T20:44:29Z

At this point outerloops jobs have been confirmed to work as expected. Running the innerloop jobs once more with a comment change. Then I think it is good to merge. Will check over the outerloop jobs this weekend to see if anything has gone wrong.

jashook · 2018-08-24T21:40:28Z

@dotnet-bot test Windows_NT x64 Release CoreFX Tests

jashook · 2018-08-25T04:20:15Z

Yay merged!

…lix queue Port (partially) the following changes: * Build xunit wrappers the same way on windows and unix (#18695) * Update vc-runtime package to support ARM and ARM64 with current builds (#19265) * build-test - fix TestWrapper CS warnings (#19180) * Clean up build.cmd/build-test.cmd/runtest.cmd (#19291) * Use runtest.py to run tests for all platforms (#19213) * Updating runtest.py so that it works with Python 3 (#19768) * Fix build-test.sh wrapper build (#19779) * Move ARM64 Windows boxen to be Helix-provisioned (#20204) * Runtest.py on Windows Arm(64) (#20227) * Use runtest.cmd for arm(64) windows testing (#20301) * Do not restore or initialize buildtools for x86/arm64 (#20370) * Add hack for arm64/x86 to skip build tools restore (#20390) Update Xunit to 2.4.1 prerelease version Use the latest version (dbf0bf1) of tests/runtest.py

…lix queue Port (partially) the following changes: * Build xunit wrappers the same way on windows and unix (#18695) * Work around cmd command length limit in xunit Exec task (#19095) * Update vc-runtime package to support ARM and ARM64 with current builds (#19265) * build-test - fix TestWrapper CS warnings (#19180) * Clean up build.cmd/build-test.cmd/runtest.cmd (#19291) * Use runtest.py to run tests for all platforms (#19213) * Updating runtest.py so that it works with Python 3 (#19768) * Fix build-test.sh wrapper build (#19779) * Move ARM64 Windows boxen to be Helix-provisioned (#20204) * Runtest.py on Windows Arm(64) (#20227) * Use runtest.cmd for arm(64) windows testing (#20301) * Do not restore or initialize buildtools for x86/arm64 (#20370) * Add hack for arm64/x86 to skip build tools restore (#20390) Update Xunit to 2.4.1 prerelease version Use the latest version (dbf0bf1) of tests/runtest.py

jashook requested review from sbomer and RussKeldorph July 31, 2018 16:30

AaronRobinsonMSFT added the test enhancement Improvements of test source code label Aug 1, 2018

jashook added the area-Infrastructure-coreclr label Aug 1, 2018

sandreenko approved these changes Aug 1, 2018

View reviewed changes

sbomer approved these changes Aug 1, 2018

View reviewed changes

RussKeldorph approved these changes Aug 1, 2018

View reviewed changes

jashook added the * NO MERGE * The PR is not ready for merge yet (see discussion for detailed reasons) label Aug 2, 2018

jashook mentioned this pull request Aug 6, 2018

Add basic check that the correct number of tests is built #19290

Merged

jashook force-pushed the clean_runtest_py branch from d9aa922 to 8c89e2b Compare August 14, 2018 18:44

jashook changed the title ~~[Unix|Windows|x64] Unify exclusion list and enable runtest.py~~ [Unix|Windows|AllArch] Unify exclusion list and enable runtest.py Aug 14, 2018

jashook force-pushed the clean_runtest_py branch from cf111c6 to a71d711 Compare August 16, 2018 19:04

jashook mentioned this pull request Aug 17, 2018

Remove coreclr.*txt upload #19524

Merged

jashook closed this Aug 17, 2018

jashook reopened this Aug 17, 2018

jashook force-pushed the clean_runtest_py branch from c31bb13 to f8f97d6 Compare August 17, 2018 18:27

jashook force-pushed the clean_runtest_py branch from f8f97d6 to b39debe Compare August 17, 2018 19:50

jashook force-pushed the clean_runtest_py branch from fb55fdd to c8897b7 Compare August 23, 2018 19:29

jashook force-pushed the clean_runtest_py branch from c8897b7 to 9ec1188 Compare August 23, 2018 20:54

jashook force-pushed the clean_runtest_py branch from 9ec1188 to 25cdfe9 Compare August 24, 2018 04:34

jashook force-pushed the clean_runtest_py branch from 25cdfe9 to 0fac8f1 Compare August 24, 2018 04:39

jashook force-pushed the clean_runtest_py branch from 0fac8f1 to b317c71 Compare August 24, 2018 14:46

BruceForstall approved these changes Aug 24, 2018

View reviewed changes

jashook force-pushed the clean_runtest_py branch from b317c71 to 3fe2153 Compare August 24, 2018 20:41

jashook merged commit b34f2bf into dotnet:master Aug 25, 2018

sandreenko mentioned this pull request Aug 29, 2018

fix merge conflict in build-test.sh #19744

Merged

tannergooding mentioned this pull request Aug 30, 2018

Updating runtest.py so that it works with Python 3 #19768

Merged

echesakov pushed a commit that referenced this pull request Oct 23, 2018

[release/2.2] Use runtest.py to run tests for all platforms (#19213)

3552dad

echesakov mentioned this pull request Oct 25, 2018

[release/2.2] Remove the Smarty dependency and move Windows ARM to Helix queue #20613

Merged

jashook mentioned this pull request Jan 31, 2020

[Windows|Arm64] init-tools.cmd supports native dotnet cli dotnet/runtime#10946

Closed

kouvel mentioned this pull request May 7, 2020

Re-enable default interface methods tests on ARM64 dotnet/runtime#35889

Merged

[Unix|Windows|AllArch] Unify exclusion list and enable runtest.py #19213

[Unix|Windows|AllArch] Unify exclusion list and enable runtest.py #19213

Conversation

jashook commented Jul 31, 2018 • edited Loading

jashook commented Jul 31, 2018

jashook commented Jul 31, 2018

jashook commented Jul 31, 2018

AaronRobinsonMSFT commented Aug 1, 2018

sandreenko left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jashook commented Aug 2, 2018

AaronRobinsonMSFT commented Aug 2, 2018

jashook commented Aug 14, 2018 • edited Loading

jashook commented Aug 15, 2018

jashook commented Aug 16, 2018

jashook commented Aug 16, 2018

jashook commented Aug 17, 2018

jashook commented Aug 17, 2018

jashook commented Aug 17, 2018

jashook commented Aug 23, 2018

jashook commented Aug 23, 2018

jashook commented Aug 23, 2018

jashook commented Aug 23, 2018

jashook commented Aug 24, 2018

jashook commented Aug 24, 2018

jashook commented Aug 24, 2018

jashook commented Aug 24, 2018

jashook commented Aug 24, 2018

jashook commented Aug 24, 2018

jashook commented Aug 25, 2018

jashook commented Jul 31, 2018 •

edited

Loading

jashook commented Aug 14, 2018 •

edited

Loading