Windows support in the test suite + appveyor

[oli-obk]

We seriously need to give compiletest a compile-pass test kind... These hacks are getting out of hand

[RalfJung]

Indeed they're getting out of hand. I am not very in favor of this PR. What is even the problem with mir-opt?

[oli-obk]

What is even the problem with mir-opt?

It tries to execute the final binary. Which doesn't work in the cross compile case. We hacked this away by adding a binary runner echo "" ||. That doesn't work on windows and there's no general way to do that (I couldn't find any way to get the same behaviour on windows)

[RalfJung]

We seriously need to give compiletest a compile-pass test kind.

compiletest just mirrors upstream rustc. Do you think they would accept a patch adding a mode they don't use? A less invasive solution may be to change the mir-opt pass to honor the run_pass flag like UI tests do. Maybe they'd rather accept that.

Or we just fork compiletest-rs...

[oli-obk]

Or we just fork compiletest-rs...

nonono, it breaks every now and then. We don't want yet another randomly breaking dependency.

If we scrap the aux test, we already have a test runner that works... the one i'm throwing on the rustc test suite.

[RalfJung]

What's it missing to handle dependencies?

[oli-obk]

What's it missing to handle dependencies?

1. find all files in an auxilary directory
2. compile them into a temporary dir
3. pass -L to all normal tests giving the link directory
4. pass --extern to all normal tests for each file in the auxilary directory

[oli-obk]

Oh oh... now the fullmir tests are failing... This will be a pain...

no mir for std::sys::imp::c::::AddVectoredExceptionHandler

[RalfJung]

I will prepare a quick PR against upstream Rust to see what they think about a CompilePass mode.

[oli-obk]

waiting for rust-lang/rust#44537

[RalfJung]

With #343 landed, it should now be possible to do this without new hacks, right?

[oli-obk]

Yes, I rebased over master to do those changes myself, and saw that you already did them XD Then I started getting the github mail notfications. I'll probably disable the fullmir tests on windows for now though

[RalfJung]

I'll probably disable the fullmir tests on windows for now though

What's the matter with those?

[oli-obk]

At minimum rust-lang/rust#44537

This isn't an issue on linux, becaus memchr is a libc function there, but on windows libstd has its own Rust impl.

[oli-obk]

error: a raw memory access tried to access part of a pointer value as raw bytes
  --> C:\Users\ca8159\.rustup\toolchains\nightly-x86_64-pc-windows-msvc\lib\rustlib\src\rust\src\libstd\sys_common\memchr.rs:68:21
   |
68 |         let align = (ptr as usize) & (usize_bytes- 1);
   |                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   |
note: inside call to std::sys_common::memchr::fallback::memchr
  --> C:\Users\ca8159\.rustup\toolchains\nightly-x86_64-pc-windows-msvc\lib\rustlib\src\rust\src\libstd\memchr.rs:35:5
   |
35 |     ::sys::memchr::memchr(needle, haystack)
   |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
note: inside call to std::memchr::memchr
  --> C:\Users\ca8159\.rustup\toolchains\nightly-x86_64-pc-windows-msvc\lib\rustlib\src\rust\src\libstd\ffi\c_str.rs:246:15
   |
246|         match memchr::memchr(0, &bytes) {
   |               ^^^^^^^^^^^^^^^^^^^^^^^^^
note: inside call to std::ffi::CString::_new
  --> C:\Users\ca8159\.rustup\toolchains\nightly-x86_64-pc-windows-msvc\lib\rustlib\src\rust\src\libstd\ffi\c_str.rs:242:9
   |
242|         Self::_new(t.into())
   |         ^^^^^^^^^^^^^^^^^^^^
note: inside call to std::ffi::CString::new::<&str>
  --> C:\Users\ca8159\.rustup\toolchains\nightly-x86_64-pc-windows-msvc\lib\rustlib\src\rust\src\libstd\sys\windows\compat.rs:31:18
   |
31 |     let symbol = CString::new(symbol).unwrap();
   |                  ^^^^^^^^^^^^^^^^^^^^
note: inside call to std::sys::imp::compat::lookup
  --> C:\Users\ca8159\.rustup\toolchains\nightly-x86_64-pc-windows-msvc\lib\rustlib\src\rust\src\libstd\sys\windows\compat.rs:43:17
   |
43 |     let value = lookup(module, symbol).unwrap_or(fallback);
   |                 ^^^^^^^^^^^^^^^^^^^^^^
note: inside call to std::sys::imp::compat::store_func
  --> C:\Users\ca8159\.rustup\toolchains\nightly-x86_64-pc-windows-msvc\lib\rustlib\src\rust\src\libstd\sys\windows\compat.rs:64:17
   |
64 | /                 ::sys::compat::store_func(&PTR,
65 | |                                           stringify!($module),
66 | |                                           stringify!($symbol),
67 | |                                           fallback as usize)
   | |____________________________________________________________^
note: inside call to std::sys::imp::c::SetThreadStackGuarantee::load
  --> C:\Users\ca8159\.rustup\toolchains\nightly-x86_64-pc-windows-msvc\lib\rustlib\src\rust\src\libstd\sys\windows\compat.rs:75:22
   |
75 |                 0 => load(),
   |                      ^^^^^^
note: inside call to std::sys::imp::c::SetThreadStackGuarantee
  --> C:\Users\ca8159\.rustup\toolchains\nightly-x86_64-pc-windows-msvc\lib\rustlib\src\rust\src\libstd\sys\windows\stack_overflow.rs:22:12
   |
22 |         if c::SetThreadStackGuarantee(&mut 0x5000) == 0 {
   |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
note: inside call to std::sys::imp::stack_overflow::Handler::new
  --> C:\Users\ca8159\.rustup\toolchains\nightly-x86_64-pc-windows-msvc\lib\rustlib\src\rust\src\libstd\sys\windows\stack_overflow.rs:49:14
   |
49 |     let _h = Handler::new();
   |              ^^^^^^^^^^^^^^
note: inside call to std::sys::imp::stack_overflow::init
  --> C:\Users\ca8159\.rustup\toolchains\nightly-x86_64-pc-windows-msvc\lib\rustlib\src\rust\src\libstd\rt.rs:45:9
   |
45 |         sys::stack_overflow::init();
   |         ^^^^^^^^^^^^^^^^^^^^^^^^^^^
note: inside call to std::rt::lang_start
  --> C:\Users\ca8159\.rustup\toolchains\nightly-x86_64-pc-windows-msvc\lib\rustlib\src\rust\src\libstd\rt.rs:32:1
   |
32 | / fn lang_start(main: fn(), argc: isize, argv: *const *const u8) -> isize {
33 | |     use panic;
34 | |     use sys;
35 | |     use sys_common;
...  |
72 | |     }
73 | | }
   | |_^

[RalfJung]

Will this be entirely fixed by your PR? I can imagine that it may still do byte-wise accesses which would still break when there are pointers in the buffer.

[oli-obk]

I can imagine that it may still do byte-wise accesses which would still break when there are pointers in the buffer.

Strings usually don't contain pointers ;) It'll fix it in the use cases we have in CI

[RalfJung]

What about using --locked --release like we do for Travis?

(Maybe --locked fails due to platform-specific dependencies, but at least --release should help speeding things up a little. If it doesn't slow down compliation too much.)

[oli-obk]

Yay, everything passes. We don't have appveyor CI automatically running though

[oli-obk]

Let's see if appveyor works

[oli-obk]

🎉 it does