Travis fails but stays green

[rijobro]

Travis shows that all builds are green for the Superbuild (e.g., most recent build).

However, if you check one of the OSX builds (e.g., this one), you'll notice:

3: The following tests FAILED:
3: 	  1 - test_all (Failed)
3: Errors while running CTest
3/3 Test #3: Gadgetron_TESTS ..................***Failed   20.27 sec

These tests are also failing on SIRF, but they are actually showing up as having failed (e.g., link).

The reason they should be red is that they are failing the Gagetron unit tests (added in PR #182). This is a problem to be addressed elsewhere, but Travis should be flagging up the fact that it fails.

[casperdcl]

possibly unrelated, but:

• https://stackoverflow.com/questions/51289456/test-for-non-zero-exit-status-using-ctest-cmake
• https://stackoverflow.com/questions/33693486/how-can-i-use-cmake-to-test-processes-that-are-expected-to-fail-with-an-exceptio

[rijobro]

I think these are unrelated, as they are for cases that are expected to fail, so we want to return the opposite of the exit code of the script.

In the case of Gadgetron, if the test fails, we want to report that it fails.

Since the ctest fails, but travis is happy, my guess is that the answer lies in the way we've set up the .travis.yml (not sure though).

[KrisThielemans]

I guess this then needs

ctest -VV || travis_terminate 1

in the .travis.yml

[rijobro]

Ok, waiting to see what travis has to say about that: https://travis-ci.org/CCPPETMR/SIRF-SuperBuild/builds/483358628.

[casperdcl]

doesn't explain why SIRF fails without needing the travis_terminate

[rijobro]

This seems to fix it (link), but also shows up the docker builds in red. @casperdcl is this normal?

[casperdcl]

ah I didn't look at the correct .travis.yml - was looking at SIRF, not the superBuild. The PR needs some modification, will do

[rijobro]

This has started happening again. Most recent error that should be causing it to be red happened during the cmake of SIRF:

CMake Error at /usr/local/Cellar/cmake/3.11.4/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:137 (message):

  Could NOT find FFTW3 (missing: FFTW3F_LIBRARY FFTW3_INCLUDE_DIR)

Call Stack (most recent call first):

  /usr/local/Cellar/cmake/3.11.4/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:378 (_FPHSA_FAILURE_MESSAGE)

  /Users/travis/build/CCPPETMR/SIRF-SuperBuild/INSTALL/lib/cmake/ISMRMRD/FindFFTW3.cmake:114 (find_package_handle_standard_args)

  src/xGadgetron/CMakeLists.txt:36 (find_package)





-- Configuring incomplete, errors occurred!

See also "/Users/travis/build/CCPPETMR/SIRF-SuperBuild/builds/SIRF/build/CMakeFiles/CMakeOutput.log".

make[2]: *** [builds/SIRF/stamp/SIRF-configure] Error 1

make[1]: *** [CMakeFiles/SIRF.dir/all] Error 2

make[1]: *** Waiting for unfinished jobs....

[paskino]

on what OS?

[rijobro]

both unfortunately...

[KrisThielemans]

we won't update Travis things anymore