optionaltest and shtest should be made to pass on Windows builds

[Mindavi]

Describe the bug
Failed test on appveyor don't trigger a 'fail' on the test run.

To Reproduce

Go to appveyor and look at the test logs. These contain lines like this

============================================================================
Testsuite summary for jq 1.6-51-g9a0d5be
============================================================================
# TOTAL: 1
# PASS:  0
# SKIP:  0
# XFAIL: 0
# FAIL:  1
# XPASS: 0
# ERROR: 0
============================================================================
See ./test-suite.log

https://ci.appveyor.com/api/buildjobs/9jhag16jw73arl27/log (text log)
https://ci.appveyor.com/project/stedolan/jq/build/job/9jhag16jw73arl27 (test page)

Expected behavior
Failed test within appveyor will give a failure status for the tests instead of seemingly passing.

Environment (please complete the following information):

• OS and Version: Windows
• jq version: jq 1.6-51-g9a0d5be (master branch)

[nicowilliams]

For optionaltest and shtest this is on purpose. Failures of any of the others do cause appveyor build failures.

See appveyor.yml:

test_script:
  # tests/optionaltest and tests/shtest fail on Windows; run them
  # anyways but ignore their failures.  Also, trace shtest.
  - bash -lc "exec 0</dev/null && cd $APPVEYOR_BUILD_FOLDER && make -j3 SUBDIRS= 'TESTS=tests/mantest tests/jqtest tests/onigtest tests/base64test' check"
  - bash -lc "exec 0</dev/null && cd $APPVEYOR_BUILD_FOLDER && make SUBDIRS= TESTS=tests/optionaltest check || true"
  - bash -lc "exec 0</dev/null && cd $APPVEYOR_BUILD_FOLDER && make SUBDIRS= TRACE_TESTS=1 TESTS=tests/shtest check || true"

Basically, we have to do some work to get these to pass on Windows.

[Mindavi]

I guess this can be closed

[nicowilliams]

I think we should leave this open. We really should fix these failures.

[nicowilliams]

Thanks for creating this issue!

[itchyny]

Closing as we now run Windows CI on GitHub Actions, limiting the target by TESTS=. Portability of shtest is tracked by #1079.