Add integration tests for Maven Surefire provider

[marcphilipp]

In order to ensure that the Maven Surefire provider is working correctly we need a basic set of integration tests that use an actual POM, execute tests via Maven, and verify their results.

[lutovich]

Hello @marcphilipp,

I've made some initial progress on such tests and it would be nice to get some feedback.
Commit lutovich@6ea1ebc adds a simple IT with real pom.xml that checks maven exit code and test execution results.

The main complication is that ITs like this need junit-platform-surefire-provider jar to include into the pom. Same applies to transitive dependencies. Prototype scans classpath and creates pom.xml with needed jars using system scope and systemPath. This can probably be made prettier by using some template engine for pom.xml generation. Also IT lives in platform-tests module to be able to access surefire provider jar.

Please let me know what you think about this approach. Thanks!

[marcphilipp]

Thanks, that looks like it has been a lot of work! Maybe we should really think about a plan to submit a pull request to Surefire soon so we can leverage their integration test infrastructure. But for now I think you're heading into the right direction. 👍

[lutovich]

@marcphilipp thanks for the feedback. I'll try to shape up a PR this week.

[britter]

See #31 : We'll be using the infrastructure already provided by the Surefire Project.

[lutovich]

@britter @marcphilipp ok, this sounds like a good plan. I'll stop working on this task than.
Here is my latest version of ITs: lutovich@6e94d9d and it is not pretty. Plus I was not able to figure out how to retrieve surefire events to assert on them.

[britter]

Please see apache/maven-surefire#118

[britter]

apache/maven-surefire#118 has been fixed. The Apache Maven team has set up a branch in the surefire repository for working on the surefire integration. I'm going to add more integration tests. I'm currently not sure when/how we're going to handle the contribution of the provider implementation. I'll comment again, when I have a better overview of surefire.

[marcphilipp]

@britter Awesome, thanks for your perseverence!

[marcphilipp]

Will be addressed within Surefire.