Skeptic tests fail sporadically on travis

[Marwes]

When running skeptic tests on travis a few tests fail sporadically as in https://travis-ci.org/Marwes/gluon/jobs/148843400 (starting at line 629). I thought it had something to do with the use of travis_wait (which the build above used) but I have seen it after removing travis_wait as well.

I am afraid I don't have any more specific information to give than this :(.

Might be related to #4?

[brson]

First step to fixing this is probably to make the error output here more useful so we can see what's failing. I also suggested a fix for #4 on that bug.

[Marwes]

Yeah I thought to add some better error output as I work on #8 as well (might submit it as a separate PR however).

[Marwes]

Using the additional info from #13 I have gotten skeptic to error in this build https://travis-ci.org/Marwes/gluon/jobs/154866961. Everything looks fine as far as I can tell but posting it here in case someone else has the same issue and spots something.

thread 'tutorial_6' panicked at 'Command failed:

"rustc" "/tmp/rust-skeptic.SRgMRBCJv9uD/test.rs" "--verbose" "-o" "/tmp/rust-skeptic.SRgMRBCJv9uD/out.exe" "--crate-type=bin" "-L" "/home/travis/build/Marwes/gluon/c-api/target/debug" "-L" "/home/travis/build/Marwes/gluon/c-api/target/debug/deps" "--extern" "libstrsim=/home/travis/build/Marwes/gluon/c-api/target/debug/deps/libstrsim-cb9b0c8013c1be6f.rlib" "--extern" "libcombine_language=/home/travis/build/Marwes/gluon/c-api/target/debug/deps/libcombine_language-de0f9e17a7f3001f.rlib" "--extern" "libbitflags=/home/travis/build/Marwes/gluon/c-api/target/debug/deps/libbitflags-75746cc7f0e9d928.rlib" "--extern" "libfnv=/home/travis/build/Marwes/gluon/c-api/target/debug/deps/libfnv-73998ef0f702dc63.rlib" "--extern" "libthread_local=/home/travis/build/Marwes/gluon/c-api/target/debug/deps/libthread_local-e5ce0d44bcaf00e6.rlib" "--extern" "libenv_logger=/home/travis/build/Marwes/gluon/c-api/target/debug/deps/libenv_logger-2296687296dd1e8e.rlib" "--extern" "libunion_find=/home/travis/build/Marwes/gluon/c-api/target/debug/deps/libunion_find-45cd064b6098790b.rlib" "--extern" "libpulldown_cmark=/home/travis/build/Marwes/gluon/c-api/target/debug/deps/libpulldown_cmark-db3a99e46a6fde1d.rlib" "--extern" "libquick_error=/home/travis/build/Marwes/gluon/c-api/target/debug/deps/libquick_error-729eced693f2612d.rlib" "--extern" "libregex=/home/travis/build/Marwes/gluon/c-api/target/debug/deps/libregex-a1c323daba09617d.rlib" "--extern" "libterm_size=/home/travis/build/Marwes/gluon/c-api/target/debug/deps/libterm_size-19002b3950ee8264.rlib" "--extern" "libtyped_arena=/home/travis/build/Marwes/gluon/c-api/target/debug/deps/libtyped_arena-db724362e3145ac6.rlib" "--extern" "liblog=/home/travis/build/Marwes/gluon/c-api/target/debug/deps/liblog-bf16bb9a4912b11d.rlib" "--extern" "libregex_syntax=/home/travis/build/Marwes/gluon/c-api/target/debug/deps/libregex_syntax-b24da02611351433.rlib" "--extern" "libgluon_parser=/home/travis/build/Marwes/gluon/c-api/target/debug/deps/libgluon_parser-40b5ccd2caf17cdb.rlib" "--extern" "libgluon_check=/home/travis/build/Marwes/gluon/c-api/target/debug/deps/libgluon_check-519caf9a18391972.rlib" "--extern" "libvec_map=/home/travis/build/Marwes/gluon/c-api/target/debug/deps/libvec_map-d843695eff67d492.rlib" "--extern" "libpretty=/home/travis/build/Marwes/gluon/c-api/target/debug/deps/libpretty-eb115aef535b0f70.rlib" "--extern" "libaho_corasick=/home/travis/build/Marwes/gluon/c-api/target/debug/deps/libaho_corasick-e528bf4fdf3954ff.rlib" "--extern" "libtempdir=/home/travis/build/Marwes/gluon/c-api/target/debug/deps/libtempdir-c5a9975021308003.rlib" "--extern" "libclap=/home/travis/build/Marwes/gluon/c-api/target/debug/deps/libclap-e8dde56376072b43.rlib" "--extern" "libgluon=/home/travis/build/Marwes/gluon/c-api/target/debug/deps/libgluon-9e8d3942b032677b.rlib" "--extern" "libgluon_base=/home/travis/build/Marwes/gluon/c-api/target/debug/deps/libgluon_base-2e60e760b22bbd7e.rlib" "--extern" "libgetopts=/home/travis/build/Marwes/gluon/c-api/target/debug/deps/libgetopts-33691dbdf8852281.rlib" "--extern" "libansi_term=/home/travis/build/Marwes/gluon/c-api/target/debug/deps/libansi_term-7a5cfeb616239ed5.rlib" "--extern" "librustyline=/home/travis/build/Marwes/gluon/c-api/target/debug/deps/librustyline-35ce0820bc5700a5.rlib" "--extern" "libbuild=/home/travis/build/Marwes/gluon/c-api/target/debug/deps/libbuild-493a7b0628804707.rlib" "--extern" "libmemchr=/home/travis/build/Marwes/gluon/c-api/target/debug/deps/libmemchr-c555f740a543880f.rlib" "--extern" "libcombine=/home/travis/build/Marwes/gluon/c-api/target/debug/deps/libcombine-78ced705a372d3d5.rlib" "--extern" "libskeptic=/home/travis/build/Marwes/gluon/c-api/target/debug/deps/libskeptic-4ff5b3fa7d7d6186.rlib" "--extern" "libwinapi=/home/travis/build/Marwes/gluon/c-api/target/debug/deps/libwinapi-0889532d327ff4e2.rlib" "--extern" "libmopa=/home/travis/build/Marwes/gluon/c-api/target/debug/deps/libmopa-995c59fb09027a13.rlib" "--extern" "libencode_unicode=/home/travis/build/Marwes/gluon/c-api/target/debug/deps/libencode_unicode-d74a1a9ff97304d6.rlib" "--extern" "libkernel32=/home/travis/build/Marwes/gluon/c-api/target/debug/deps/libkernel32-df86a08647459244.rlib" "--extern" "libgluon_vm=/home/travis/build/Marwes/gluon/c-api/target/debug/deps/libgluon_vm-d4d697a89cb8fa3c.rlib" "--extern" "libbitflags=/home/travis/build/Marwes/gluon/c-api/target/debug/deps/libbitflags-0e272044714c8076.rlib" "--extern" "librand=/home/travis/build/Marwes/gluon/c-api/target/debug/deps/librand-49a08859d086fffe.rlib" "--extern" "libutf8_ranges=/home/travis/build/Marwes/gluon/c-api/target/debug/deps/libutf8_ranges-5c6a6dacba3be7ce.rlib" "--extern" "liblibc=/home/travis/build/Marwes/gluon/c-api/target/debug/deps/liblibc-1bd8847afb79f283.rlib" "--extern" "libthread_id=/home/travis/build/Marwes/gluon/c-api/target/debug/deps/libthread_id-bcd46c79a620a618.rlib" "--extern" "liblazy_static=/home/travis/build/Marwes/gluon/c-api/target/debug/deps/liblazy_static-359f5533c970cd71.rlib" "--extern" "libnix=/home/travis/build/Marwes/gluon/c-api/target/debug/deps/libnix-e3c0cb7ca8e41f17.rlib" "--extern" "libunicode_width=/home/travis/build/Marwes/gluon/c-api/target/debug/deps/libunicode_width-bcfd50e6d35d16e0.rlib"', /home/travis/.cargo/git/checkouts/rust-skeptic-a55258b0bcefe102/master/src/skeptic/lib.rs:390

[brson]

I'm afraid I still don't see anything obvious. It seems like the rustc command isn't printing anything to stdout or stderr? That's quite unexpected since rustc prints something when it errors.

[Marwes]

Yes, it does seem a bit bit odd. Looked at librustc_driver https://github.com/rust-lang/rust/blob/824000aee3b9fd594a3da3eb38462fa341f0fa81/src/librustc_driver/lib.rs#L1083, if there is a FatalError nothing seem to be printed directly. It might be printed somewhere else though... Would submit an issue on rust-lang/rust if there was something to actually go on.

[Marwes]

Started enabling skeptic tests again and haven't seen this again. This might very well be because I have to do cargo clean before every build due to #18 though (I think it is likely that this issue and #18 is related).

Closing, but will reopen if I see it happen again.