pitest-junit5-plugin doesn't support JUnit 5.10.x

[ThomGeG]

Hey all,

I've got a handful of Gradle projects using SpringBoot and have bumped them to the latest versions (which is also managing the JUnit version up from 5.9.x to 5.10.x) only to find that it's broken PIT. I've done some sluething to figure out the problem, so for the sake of future travelers it starts here:

10:37:15 am PIT >> INFO : MINION : 10:37:15 am PIT >> INFO : Checking environment
10:37:16 am PIT >> FINE : MINION : org.junit.platform.commons.JUnitException: TestEngine with ID 'junit-jupiter' failed to discover tests
...
Caused by: java.lang.NoSuchMethodError: 'java.util.stream.Stream org.junit.platform.commons.support.ReflectionSupport.streamNestedClasses(java.lang.Class, java.util.function.Predicate)'
10:37:16 am PIT >> INFO : MINION :      at org.junit.jupiter.engine.discovery.ClassSelectorResolver.lambda$toResolution$12(ClassSelectorResolver.java:138)
...
10:37:16 am PIT >> SEVERE : Coverage generator Minion exited abnormally due to UNKNOWN_ERROR
Exception in thread "main" org.pitest.util.PitError: Coverage generation minion exited abnormally! (UNKNOWN_ERROR)

The method it can't find is this one. It was introduced in the 5.10 release, however the shaded JUnit classes within pitest-junit5-plugin are still on version 5.9.2 and therefore predates those changes.

Looking at the arguments being passed into PIT itself, my classpath looks like this:

C:\Users\thomasgaskell\.gradle\caches\modules-2\files-2.1\org.pitest\pitest-junit5-plugin\1.1.2\d9e9db8766ed43ebe2209b35f95a9c091d4f5885\pitest-junit5-plugin-1.1.2.jar
C:\Users\thomasgaskell\.gradle\caches\modules-2\files-2.1\org.pitest\pitest\1.15.8\7d0e02f2c7fa71fc217aef154450782932e4af74\pitest-1.15.8.jar
...
C:\Users\thomasgaskell\.gradle\caches\modules-2\files-2.1\org.mockito\mockito-junit-jupiter\5.7.0\ac2d6a3431747a7986b8f4abef465f72bf3a21ae\mockito-junit-jupiter-5.7.0.jar
C:\Users\thomasgaskell\.gradle\caches\modules-2\files-2.1\org.junit.jupiter\junit-jupiter-params\5.10.2\359132c82a9d3fa87a325db6edd33b5fdc67a3d8\junit-jupiter-params-5.10.2.jar
C:\Users\thomasgaskell\.gradle\caches\modules-2\files-2.1\org.junit.jupiter\junit-jupiter-engine\5.10.2\f1f8fe97bd58e85569205f071274d459c2c4f8cd\junit-jupiter-engine-5.10.2.jar
C:\Users\thomasgaskell\.gradle\caches\modules-2\files-2.1\org.junit.jupiter\junit-jupiter-api\5.10.2\fb55d6e2bce173f35fd28422e7975539621055ef\junit-jupiter-api-5.10.2.jar
C:\Users\thomasgaskell\.gradle\caches\modules-2\files-2.1\org.junit.platform\junit-platform-engine\1.10.2\d53bb4e0ce7f211a498705783440614bfaf0df2e\junit-platform-engine-1.10.2.jar
C:\Users\thomasgaskell\.gradle\caches\modules-2\files-2.1\org.junit.platform\junit-platform-commons\1.10.2\3197154a1f0c88da46c47a9ca27611ac7ec5d797\junit-platform-commons-1.10.2.jar
C:\Users\thomasgaskell\.gradle\caches\modules-2\files-2.1\org.junit.jupiter\junit-jupiter\5.10.2\831c0b86ddc2ce38391c5b81ea662b0cfdc02cce\junit-jupiter-5.10.2.jar
C:\Users\thomasgaskell\.gradle\caches\modules-2\files-2.1\org.mockito\mockito-core\5.7.0\a1c258331ab91d66863c983aff7136357e9de056\mockito-core-5.7.0.jar
...

My classpath contains its own JUnit 5.10.x dependencies because that's what I'm otherwise using to run my tests 'normally', however the shaded classes embedded into pitest-junit5-plugin are being picked up first. The mix of 5.10.x and 5.9.x is therefore causing me grief I presumably won't have if I move all my projects back down to 5.9.x.

From what I understand PIT needs this JUnit plugin to be able to discover the tests (all that code lives in org.pitest.junit5) so I can't simply omit it to get everything working. Instead I need to drop myself back down to 5.9.x and wait for a 5.10.x compatible version of this plugin to be released.

With that said, are there plans to release a new version of this plugin that supports 5.10.x?

What I'm seeing sounds similar to szpak/gradle-pitest-plugin#300, so apologies if I'm raising this in the wrong spot. The way PIT is split into the Gradle/Maven projects, this JUnit plugin, and PIT itself makes it difficult to pick the right one sometimes. One could maybe argue the heart of the issue is more to do with how the Gradle plugin is putting JUnit on the classpath even though it's already provided by the pitest-junit5-plugin. Ideally I would have thought this plugin wouldn't shade JUnit so bumping versions doesn't require actually re-releasing the plugin like it currently does.

[ThomGeG]

For any future travelers that just want a quick fix to the problem, I've simply managed my JUnit version back down to 5.9.3 via the gradle.properties.

# TODO remove once pitest-junit5-plugin supports JUnit 5.10.x
junit-jupiter.version=5.9.3

[hcoles]

What versions are you using of

• pitest
• info.solidsoft.pitest
• pitest-junit5-plugin

Recent releases of the junit 5 plugin do not include shaded junit classes and should work with 5.10 without issue.

[ThomGeG]

Ah... upon closer inspection despite specifying junit5PluginVersion = '1.2.1' in my projects I am infact pulling in version 1.1.2. Apologies, my dyslexia has betrayed me and I hadn't noticed I was decompiling an older version than the latest. I should have put two and two together when I couldn't see any shading in the pom on master...

I'll take another look at why I'm pulling in the wrong version of the JUnit plugin (I vaguely remember there being a bug with the Gradle plugin along these lines) and close this off when I eventually find alls well.

[ThomGeG]

It's indeed working when I force it to use the correct version of the JUnit plugin, so I'll close this off.

There are 2 issues I'm encountering however:

1. The wrong version of the JUnit plugin is being picked up
   I use a binary plugin to manage PIT, SpotBugs, Checkstyle, and a bunch of other boilerplate across my projects from a single source of truth. That binary plugin is what's updating the extension managing the JUnit plugin version. I imagine the source of my problem is something in how gradle-pitest-plugin works under the bonnet. There was a similar issue a while back, gradle-pitest-plugin#244, with the specified versions being ignored.
2. I'm missing the junit-platform-launcher
   When the shaded JUnit JARs where removed it created the need to otherwise have junit-platform-launcher on the classpath to launch the tests with. Reading through gradle-pitest-plugin#337, support was added to automatically add the dependency however for whatever reason it's not working for me. I'll have to poke around the logs to see where it's fallen over.

I'll raise issues in gradle-pitest-plugin if I can figure out what's wrong.

[szpak]

Reading through szpak/gradle-pitest-plugin#337, support was added to automatically add the dependency however for whatever reason it's not working for me. I'll have to poke around the logs to see where it's fallen over.
I'll raise issues in gradle-pitest-plugin if I can figure out what's wrong.

The support for it in Gradle was very tricky and it is an experimental feature. If you are able to debug the Gradle plugin to determine why it was not added or - even better - provide a small project to reproduce a problem, there is a chance to have it improved.

[szpak]

I imagine the source of my problem is something in how gradle-pitest-plugin works under the bonnet. There was a similar issue a while back, szpak/gradle-pitest-plugin#244, with the specified versions being ignored.

Yes, but only in the Kotlin project and problematic to reproduce. Again, a small project as a reproducer, might help to find the problem.

[szpak]

Ah... upon closer inspection despite specifying junit5PluginVersion = '1.2.1' in my projects I am infact pulling in version 1.1.2.

Btw, it's very strange. junit5 jars might be managed by something, but - unless you have some extra internal (company) plugin to manage the versions across the projects - I don't know why anything else could manager the pitest-junit5-plugin version...

[ThomGeG]

@szpak it turned out to be a bug on my end in my binary plugin. It wasn't actually trying to set the versions and leaving them to the old defaults. No need for concern, everything among the JUnit plugin, Gradle plugin, and PIT itself are working as expected.

All's well, sorry for wasting everybodies time.

[szpak]

No problem. Thanks for the update!