Issues found so far

[japaric]

• Ubuntu 16.04 qemu-ppc64le doesn't work. Invalid instruction even with the simplest C program.
• i586-unknown-linux-gnu: catch_unwind doesn't work. i686-musl + catch_unwind = fatal runtime error: failed to initiate panic, error 5 rust-lang/rust#35599
• Various targets: libc-test doesn't pass. Assorted libc-test failures rust-lang/libc#357

[japaric]

For several targets like arm, aarch64, mips, when running the unit tests of the std crate I'm seeing these tests fail under QEMU:

• net::tcp::tests::clone_accept_concurrent
• net::tcp::tests::multiple_connect_interleaved_greedy_schedule
• net::tcp::tests::multiple_connect_interleaved_lazy_schedule
• net::udp::tests::timeouts
• net::tcp::tests::timeouts
• sys::ext::net::test::timeouts

But if I run std unit tests on real ARM hardware, the whole test suite passes. This means that what I'm really observing are QEMU failures.

If I want to turn the test green, I'll have to skip these tests on these targets. But Rust test runner doesn't provide a way to ignore tests via command line arguments.

My options:

• Add a feature to the test runner (libtest) that lets me skip some test, perhaps --skip FILTER. I'm not sure what's the likelihood of that being accepted. cc @alexcrichton
• Switch from QEMU user emulation to system emulation. That would require a kernel and a sysroot per architecture, which would make this testing framework a LOT more complicated. Right now it's very simple. I like simple.
• Patch the source code to #[ignore] the troublesome tests. That's a non starter for std because it requires upstream changes.

I like the first option best.

[japaric]

For several targets like arm, aarch64, mips, when running the unit tests of the std crate I'm seeing these tests fail under QEMU:

FWIW, I additionally tested reducing the number of tests that libtest uses for each unit tests from 2 to 1 (as implemented in rust-lang/rust#36339). But that didn't reduce the number of failures. Probably because each unit tests, itself, spawns a bunch of threads.

[alexcrichton]

Add a feature to the test runner (libtest) that lets me skip some test

👍

Switch from QEMU user emulation to system emulation

Note that this flexibility would also allow running tests on BSD platforms as well, so may have other added benefits! From doing it in libc, though, it's super complicated.