no error when a browser fails to start

[IgorMinar]

latest canay 0.3.8 does abort a run if a browser fails to start, but there is no indication as to which browser failed and with what error:

info: Testacular server started at http://localhost:9876/
info (launcher): Starting  browser "Chrome"
info (launcher): Starting  browser "Firefox"
info (launcher): Starting  browser "Opera"
info (launcher): Starting  browser "/Users/jenkins/bin/safari.sh"
info (launcher): Starting  browser "/Users/jenkins/bin/ie8.sh"
info (launcher): Starting  browser "/Users/jenkins/bin/ie9.sh"
info: Disconnecting all browsers

ideally, I'd like to see an error log statying that e.g. ie9.sh exited before a browser was captured and that the exit code was 12. or maybe even print the actual error message from stderr if there is anything there.

[vojtajina]

Does the script really exit with non zero code ? Because if so, testacular should show error, including the stderr output:
https://github.com/vojtajina/testacular/blob/master/lib/launchers/Base.js#L115

$ testacular start test/client/testacular.conf.js --browsers /non-existing/browser.sh --single-run
info: Testacular server started at http://localhost:9876/
info (launcher): Starting browser /non-existing/browser.sh
error (launcher): Cannot start /non-existing/browser.sh
    execvp(): No such file or directory

info (launcher): Trying to start /non-existing/browser.sh again.
error (launcher): Cannot start /non-existing/browser.sh
    execvp(): No such file or directory

info (launcher): Trying to start /non-existing/browser.sh again.
error (launcher): Cannot start /non-existing/browser.sh
    execvp(): No such file or directory

info: Disconnecting all browsers

This is actually log with retrying the browser 3 times....

[IgorMinar]

it's possible that the browser somehow the script ended up returning 0 even though there was an error. Try the same thing with a existing file that just returns 0 without starting a browser.

[vojtajina]

I think that's the case:

#!/bin/bash

echo "ERROR" >&2
exit 0

Will produce:

$ testacular start test/client/testacular.conf.js --browsers /Users/vojta/Code/testacular/fake-browser.sh 
info: Testacular server started at http://localhost:9876/
info (launcher): Starting browser /Users/vojta/Code/testacular/fake-browser.sh
info (launcher): Trying to start /Users/vojta/Code/testacular/fake-browser.sh again.
info (launcher): Trying to start /Users/vojta/Code/testacular/fake-browser.sh again.

If you change the script to exit non zero status, you will get:

$ testacular start test/client/testacular.conf.js --browsers /Users/vojta/Code/testacular/fake-browser.sh 
info: Testacular server started at http://localhost:9876/
info (launcher): Starting browser /Users/vojta/Code/testacular/fake-browser.sh
error (launcher): Cannot start /Users/vojta/Code/testacular/fake-browser.sh
    ERROR

info (launcher): Trying to start /Users/vojta/Code/testacular/fake-browser.sh again.
error (launcher): Cannot start /Users/vojta/Code/testacular/fake-browser.sh
    ERROR

info (launcher): Trying to start /Users/vojta/Code/testacular/fake-browser.sh again.
error (launcher): Cannot start /Users/vojta/Code/testacular/fake-browser.sh
    ERROR

We can ignore the exit code and just print out error log, whenever there is any stderr output.

[vojtajina]

Reverted, see vojtajina/testacular@825a61d
I think, if the script exits with zero code in the case of failure, it's wrong.