Unsupported or missing version of JUnit 4 should fail the build #1944

nielsbasjes · 2019-07-03T08:13:37Z

I'm using junit and vintage from the bom 5.5.0 in a Maven based project.
https://github.com/nielsbasjes/yauaa

In this project I have created a udf for Apache Pig.
In the pig dependency for this udf I also get junit 4.11.

The consequence of this older version of junit is that I get this perfectly fine stack trace during my build:

[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
Jul 02, 2019 11:42:33 PM org.junit.platform.launcher.core.DefaultLauncher handleThrowable
WARNING: TestEngine with ID 'junit-vintage' failed to discover tests
org.junit.platform.commons.JUnitException: Unsupported version of junit:junit: 4.11. Please upgrade to version 4.12 or later.
	at org.junit.vintage.engine.JUnit4VersionCheck.checkSupported(JUnit4VersionCheck.java:39)
	at org.junit.vintage.engine.JUnit4VersionCheck.checkSupported(JUnit4VersionCheck.java:32)
	at org.junit.vintage.engine.VintageTestEngine.discover(VintageTestEngine.java:61)

and later on

[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0

The bug I'm reporting is that I think this is should be a fatal error and not just a warning.

None of the tests I have ran and the ONLY reason I noticed this is because I saw a sudden drop in the code coverage of the tests.

So if this problem happens then please fail the build.

The text was updated successfully, but these errors were encountered:

sbrannen · 2019-07-03T08:23:01Z

Tentatively slated for 5.6 M1 for team discussion

marcphilipp · 2019-07-03T10:17:47Z

Related to #242.

marcphilipp · 2019-07-05T10:54:51Z

Team Decision: We agree that this should fail during test execution. 🙂

sbrannen · 2019-09-23T10:51:33Z

I updated the title to point out that a missing JUnit 4 artifact should also fail the build for use cases where someone excluded JUnit 4 from the classpath but left junit-vintage-engine in the classpath.

The logic in JUnit4VersionCheck should already handle this use case as well, but I wanted to ensure we cover both use cases explicitly: missing and unsupported JUnit 4 artifacts.

marcphilipp · 2019-09-23T11:03:25Z

This was fixed in #1909.

@nielsbasjes Would you care to verify using the latest snapshot?

nielsbasjes · 2019-09-24T10:27:26Z

The problem has not been fixed in junit-vintage-engine-5.6.0-20190924.094617-81.jar.

I get

WARNING: TestEngine with ID 'junit-vintage' failed to discover tests
org.junit.platform.commons.JUnitException: Unsupported version of junit:junit: 4.11. Please upgrade to version 4.12 or later.

then

[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0

and

[INFO] BUILD SUCCESS

To facilitate you guys repeating what I have I created a testing branch on my project
https://github.com/nielsbasjes/yauaa/tree/ReproduceJunit5Issue1944

I have the sonatype snapshot repo enabled using my ~/.m2/settings.xml

To limit the time needed to reproduce this (some of my tests take a while) do:

mvn clean install -DskipTests=true

verify I actually use the right version in the pom.xml

fgrep junit5.version ~/.m2/repository/nl/basjes/parse/useragent/yauaa-parent/5.13-SNAPSHOT/yauaa-parent-5.13-SNAPSHOT.pom 
    <junit5.version>5.6.0-SNAPSHOT</junit5.version>
    <version>${junit5.version}</version>

then

cd udfs/pig/

verify the actual dependency versions for this module

$ mvn dependency:list | fgrep junit
[INFO]    org.junit.jupiter:junit-jupiter-api:jar:5.6.0-SNAPSHOT:test
[INFO]    org.junit.platform:junit-platform-commons:jar:1.6.0-SNAPSHOT:test
[INFO]    org.junit.jupiter:junit-jupiter-engine:jar:5.6.0-SNAPSHOT:test
[INFO]    org.junit.platform:junit-platform-engine:jar:1.6.0-SNAPSHOT:test
[INFO]    org.junit.vintage:junit-vintage-engine:jar:5.6.0-SNAPSHOT:test
[INFO]    junit:junit:jar:4.11:provided

then

mvn clean test

I see this (which says BUILD SUCCESS):

$ mvn clean test
[INFO] Scanning for projects...
[INFO] 
[INFO] ------------------------------------------------------------------------
[INFO] Building Yauaa : UDF : Apache Pig 5.13-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO] 
[INFO] --- maven-clean-plugin:3.1.0:clean (default-clean) @ yauaa-pig ---
[INFO] Deleting /home/nbasjes/workspace/Prive/yauaa_/udfs/pig/target
[INFO] 
[INFO] --- git-commit-id-plugin:3.0.0:revision (get-the-git-infos) @ yauaa-pig ---
[INFO] 
[INFO] --- maven-enforcer-plugin:3.0.0-M2:enforce (enforce-maven) @ yauaa-pig ---
[INFO] 
[INFO] --- jacoco-maven-plugin:0.8.4:prepare-agent (pre-unit-test) @ yauaa-pig ---
[INFO] jacoco.surefireArgLine set to -javaagent:/home/nbasjes/.m2/repository/org/jacoco/org.jacoco.agent/0.8.4/org.jacoco.agent-0.8.4-runtime.jar=destfile=/home/nbasjes/workspace/Prive/yauaa_/udfs/pig/target/jacoco.exec
[INFO] 
[INFO] --- maven-resources-plugin:3.1.0:resources (default-resources) @ yauaa-pig ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /home/nbasjes/workspace/Prive/yauaa_/udfs/pig/src/main/resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.8.1:compile (default-compile) @ yauaa-pig ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 1 source file to /home/nbasjes/workspace/Prive/yauaa_/udfs/pig/target/classes
[INFO] 
[INFO] --- maven-resources-plugin:3.1.0:testResources (default-testResources) @ yauaa-pig ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 1 resource
[INFO] 
[INFO] --- maven-compiler-plugin:3.8.1:testCompile (default-testCompile) @ yauaa-pig ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 1 source file to /home/nbasjes/workspace/Prive/yauaa_/udfs/pig/target/test-classes
[INFO] 
[INFO] --- maven-surefire-plugin:3.0.0-M3:test (default-test) @ yauaa-pig ---
[INFO] 
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
Sep 24, 2019 12:24:27 PM org.junit.platform.launcher.core.DefaultLauncher handleThrowable
WARNING: TestEngine with ID 'junit-vintage' failed to discover tests
org.junit.platform.commons.JUnitException: Unsupported version of junit:junit: 4.11. Please upgrade to version 4.12 or later.
	at org.junit.vintage.engine.JUnit4VersionCheck.checkSupported(JUnit4VersionCheck.java:39)
	at org.junit.vintage.engine.JUnit4VersionCheck.checkSupported(JUnit4VersionCheck.java:32)
	at org.junit.vintage.engine.VintageTestEngine.discover(VintageTestEngine.java:61)
	at org.junit.platform.launcher.core.DefaultLauncher.discoverEngineRoot(DefaultLauncher.java:168)
	at org.junit.platform.launcher.core.DefaultLauncher.discoverRoot(DefaultLauncher.java:155)
	at org.junit.platform.launcher.core.DefaultLauncher.discover(DefaultLauncher.java:120)
	at org.apache.maven.surefire.junitplatform.TestPlanScannerFilter.accept(TestPlanScannerFilter.java:56)
	at org.apache.maven.surefire.util.DefaultScanResult.applyFilter(DefaultScanResult.java:102)
	at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.scanClasspath(JUnitPlatformProvider.java:143)
	at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:124)
	at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:384)
	at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:345)
	at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:126)
	at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:418)

Sep 24, 2019 12:24:27 PM org.junit.platform.launcher.core.DefaultLauncher handleThrowable
WARNING: TestEngine with ID 'junit-vintage' failed to discover tests
org.junit.platform.commons.JUnitException: Unsupported version of junit:junit: 4.11. Please upgrade to version 4.12 or later.
	at org.junit.vintage.engine.JUnit4VersionCheck.checkSupported(JUnit4VersionCheck.java:39)
	at org.junit.vintage.engine.JUnit4VersionCheck.checkSupported(JUnit4VersionCheck.java:32)
	at org.junit.vintage.engine.VintageTestEngine.discover(VintageTestEngine.java:61)
	at org.junit.platform.launcher.core.DefaultLauncher.discoverEngineRoot(DefaultLauncher.java:168)
	at org.junit.platform.launcher.core.DefaultLauncher.discoverRoot(DefaultLauncher.java:155)
	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:128)
	at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:150)
	at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:124)
	at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:384)
	at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:345)
	at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:126)
	at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:418)

[INFO] 
[INFO] Results:
[INFO] 
[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0
[INFO] 
[INFO] 
[INFO] --- maven-checkstyle-plugin:3.1.0:check (checkstyle-check) @ yauaa-pig ---
[INFO] Starting audit...
Audit done.
[INFO] 
[INFO] --- jacoco-maven-plugin:0.8.4:report (post-unit-test) @ yauaa-pig ---
[INFO] Skipping JaCoCo execution due to missing execution data file.
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 5.658 s
[INFO] Finished at: 2019-09-24T12:24:28+02:00
[INFO] Final Memory: 48M/599M
[INFO] ------------------------------------------------------------------------

marcphilipp · 2019-09-24T12:41:09Z

@nielsbasjes Thanks for testing! Must be Surefire swallowing the error. I'll take a closer look myself.

nielsbasjes · 2019-09-24T15:06:57Z

Just by looking at the messages it seems the JUnitException is classified as a WARNING.

And a warning should not fail the build...

marcphilipp · 2019-09-24T15:47:43Z

@nielsbasjes We still log this as a warning when Surefire calls discover(). However, if Surefire called execute() with the same TestPlan, it would throw an exception. I'll add an integration test.

marcphilipp · 2019-09-25T16:32:14Z

@nielsbasjes The missing piece was that Surefire bundles an old version of junit-platform-launcher. If you add the snapshot version to the test scoped dependencies, it fails.

I've created #2028 to ensure Surefire and other build tools and IDEs upgrade their bundled Launcher once 5.6 is released.

nielsbasjes · 2019-09-29T14:13:23Z

@marcphilipp Thanks. I've check it and when I make my maven look like this

  <dependencyManagement>
    <dependencies>
      <dependency>
        <groupId>org.junit</groupId>
        <artifactId>junit-bom</artifactId>
        <version>5.6.0-SNAPSHOT</version>
        <type>pom</type>
        <scope>import</scope>
      </dependency>
    </dependencies>
  </dependencyManagement>

  <dependencies>

    <dependency>
      <groupId>org.junit.jupiter</groupId>
      <artifactId>junit-jupiter-engine</artifactId>
      <scope>test</scope>
    </dependency>

    <dependency>
      <groupId>org.junit.vintage</groupId>
      <artifactId>junit-vintage-engine</artifactId>
      <scope>test</scope>
    </dependency>

    <dependency>
      <groupId>org.junit.platform</groupId>
      <artifactId>junit-platform-launcher</artifactId>
      <scope>test</scope>
    </dependency>

  </dependencies>

it correctly fails with

[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
Sep 29, 2019 4:12:16 PM org.junit.platform.launcher.core.DefaultLauncher discoverEngineRoot
SEVERE: TestEngine with ID 'junit-vintage' failed to discover tests
org.junit.platform.commons.JUnitException: Unsupported version of junit:junit: 4.11. Please upgrade to version 4.12 or later.
	at org.junit.vintage.engine.JUnit4VersionCheck.checkSupported(JUnit4VersionCheck.java:39)
	at org.junit.vintage.engine.JUnit4VersionCheck.checkSupported(JUnit4VersionCheck.java:32)
	at org.junit.vintage.engine.VintageTestEngine.discover(VintageTestEngine.java:61)
	at org.junit.platform.launcher.core.DefaultLauncher.discoverEngineRoot(DefaultLauncher.java:177)
	at org.junit.platform.launcher.core.DefaultLauncher.discoverRoot(DefaultLauncher.java:166)
	at org.junit.platform.launcher.core.DefaultLauncher.discover(DefaultLauncher.java:122)
	at org.apache.maven.surefire.junitplatform.TestPlanScannerFilter.accept(TestPlanScannerFilter.java:56)
	at org.apache.maven.surefire.util.DefaultScanResult.applyFilter(DefaultScanResult.java:102)
	at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.scanClasspath(JUnitPlatformProvider.java:143)
	at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:124)
	at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:384)
	at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:345)
	at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:126)
	at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:418)

Sep 29, 2019 4:12:16 PM org.junit.platform.launcher.core.DefaultLauncher discoverEngineRoot
SEVERE: TestEngine with ID 'junit-vintage' failed to discover tests
org.junit.platform.commons.JUnitException: Unsupported version of junit:junit: 4.11. Please upgrade to version 4.12 or later.
	at org.junit.vintage.engine.JUnit4VersionCheck.checkSupported(JUnit4VersionCheck.java:39)
	at org.junit.vintage.engine.JUnit4VersionCheck.checkSupported(JUnit4VersionCheck.java:32)
	at org.junit.vintage.engine.VintageTestEngine.discover(VintageTestEngine.java:61)
	at org.junit.platform.launcher.core.DefaultLauncher.discoverEngineRoot(DefaultLauncher.java:177)
	at org.junit.platform.launcher.core.DefaultLauncher.discoverRoot(DefaultLauncher.java:166)
	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:130)
	at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:150)
	at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:124)
	at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:384)
	at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:345)
	at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:126)
	at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:418)

[INFO] Running junit-vintage
[ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.006 s <<< FAILURE! - in junit-vintage
[ERROR] junit-vintage  Time elapsed: 0.003 s  <<< ERROR!
org.junit.platform.commons.JUnitException: TestEngine with ID 'junit-vintage' failed to discover tests
	at org.junit.vintage.engine.VintageTestEngine.discover(VintageTestEngine.java:61)

[INFO] 
[INFO] Results:
[INFO] 
[ERROR] Errors: 
[ERROR]   VintageTestEngine » JUnit TestEngine with ID 'junit-vintage' failed to discove...
[INFO] 
[ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0
[INFO] 
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 3.972 s
[INFO] Finished at: 2019-09-29T16:12:17+02:00
[INFO] Final Memory: 24M/396M
[INFO] ------------------------------------------------------------------------

Thanks for fixing this.

marcphilipp · 2019-09-29T17:50:00Z

@nielsbasjes Thanks for double-checking! 👍

raul1991 · 2020-04-26T12:13:34Z

I had this problem when using junit 4.8.2 and then changing it to 4.13 worked (see doc). I am just commenting it here so as to validate my understanding that vintage engine requires junit 4.13 and not older to be able to detect tests.

Please correct me if I am wrong.

sbrannen · 2020-04-26T13:22:11Z

Please correct me if I am wrong.

JUnit Vintage requires JUnit 4.12 or higher.

raul1991 · 2020-04-26T13:26:00Z

Please correct me if I am wrong.

JUnit Vintage requires JUnit 4.12 or higher.

I directly used 4.13 so would have missed that. Thanks for clearing it out.

sbrannen added component: Vintage theme: diagnostics labels Jul 3, 2019

sbrannen added this to the 5.6 M1 milestone Jul 3, 2019

sbrannen added the status: team discussion label Jul 3, 2019

sbrannen changed the title ~~"Unsupported version of junit:junit:" should fail the build.~~ "Unsupported version of junit:junit:" should fail the build Jul 3, 2019

marcphilipp mentioned this issue Jul 5, 2019

Introduce error handling mechanism for validation and discovery errors #242

Open

5 tasks

marcphilipp modified the milestones: 5.6 M1, 5.6 Backlog Jul 5, 2019

marcphilipp removed the status: team discussion label Jul 5, 2019

marcphilipp modified the milestones: 5.6 Backlog, 5.6 M1 Jul 15, 2019

marcphilipp mentioned this issue Jul 15, 2019

No warning/error is present in case of error executing JUnit Platform tests gradle/gradle#8956

Closed

sbrannen changed the title ~~"Unsupported version of junit:junit:" should fail the build~~ Unsupported version of JUnit 4 should fail the build Jul 16, 2019

sbrannen changed the title ~~Unsupported version of JUnit 4 should fail the build~~ Unsupported or missing version of JUnit 4 should fail the build Sep 23, 2019

marcphilipp added the status: waiting-for-feedback label Sep 23, 2019

sbrannen removed the status: waiting-for-feedback label Sep 24, 2019

marcphilipp self-assigned this Sep 24, 2019

marcphilipp closed this as completed Sep 25, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Unsupported or missing version of JUnit 4 should fail the build #1944

Unsupported or missing version of JUnit 4 should fail the build #1944

nielsbasjes commented Jul 3, 2019

sbrannen commented Jul 3, 2019

marcphilipp commented Jul 3, 2019

marcphilipp commented Jul 5, 2019

sbrannen commented Sep 23, 2019

marcphilipp commented Sep 23, 2019

nielsbasjes commented Sep 24, 2019 •

edited

Loading

marcphilipp commented Sep 24, 2019

nielsbasjes commented Sep 24, 2019

marcphilipp commented Sep 24, 2019

marcphilipp commented Sep 25, 2019

nielsbasjes commented Sep 29, 2019

marcphilipp commented Sep 29, 2019

raul1991 commented Apr 26, 2020 •

edited

Loading

sbrannen commented Apr 26, 2020

raul1991 commented Apr 26, 2020

Unsupported or missing version of JUnit 4 should fail the build #1944

Unsupported or missing version of JUnit 4 should fail the build #1944

Comments

nielsbasjes commented Jul 3, 2019

sbrannen commented Jul 3, 2019

marcphilipp commented Jul 3, 2019

marcphilipp commented Jul 5, 2019

sbrannen commented Sep 23, 2019

marcphilipp commented Sep 23, 2019

nielsbasjes commented Sep 24, 2019 • edited Loading

marcphilipp commented Sep 24, 2019

nielsbasjes commented Sep 24, 2019

marcphilipp commented Sep 24, 2019

marcphilipp commented Sep 25, 2019

nielsbasjes commented Sep 29, 2019

marcphilipp commented Sep 29, 2019

raul1991 commented Apr 26, 2020 • edited Loading

sbrannen commented Apr 26, 2020

raul1991 commented Apr 26, 2020

nielsbasjes commented Sep 24, 2019 •

edited

Loading

raul1991 commented Apr 26, 2020 •

edited

Loading