Solve syntax error reporting and other minor improvements #101
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This is required, because PHPUnit 9.6+ no longer fails for syntax errors with exit code 255. Instead, it converts the errors to exceptions and handles them the same as every other exception (like the ones we use in students stubs). Use "all-fail" to show, that exceptions are now reported as individual failing tests (for all tests here). This was based on a syntax error, too. But that does not work like this test expected anymore.
This is recommended by test runner specification.
mk-mxp
requested review from
ErikSchierboom and
neenjaw
and removed request for
neenjaw
neenjaw
approved these changes
Mar 30, 2024
mk-mxp
added
x:action/fix
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Solves this forum discussion.
This replaces #97, which got no updates for a while. It is solving the problem, that syntax errors in students code fail in a way, that cannot be distinguished from failing tests, but tests are not reported by PHPUnit as failing.
I added a syntax linting step before invoking PHPUnit, which adds minimal overhead and only fails for hard syntactical errors.
While reading into the test runner specification I realized that we shouldn't write absolute pathes into the result messages. So I adjusted output to only have file names and no path at all. That fits well with the online editor only showing the file names on editor tabs, too.
I also improved the shell script and updated the README.