Silently skipped tests when last line of Kotlin test method returns a coroutine

[simondean]

The issue I'm seeing is with JUnit 5 and Kotlin. When I have a unit test like the following, JUnit 5 skips the test and outputs nothing to indicate that it has been skipped. I've seen this issue in a real codebase were about 10 unit tests were skipped in this way. The example below is a contrived one. When I've seen it happening for real, it was a call to a Kotlin version of Mockito's verify() method that was causing the issue. The method call was the last line in the test and the method call being verified just happened to return a Kotlin coroutine (which is what the verify() method then returns).

    @Test
    // Function is missing an explicit return type
    fun `will silently be skipped`() = runBlocking {
        // Last step is something that returns a Kotlin coroutine
        launch { doSomething() }
    }

I'm using JUnit v5.8.1, the latest version

Steps to reproduce

Here's a minimal reproduction repo, including a GitHub actions build that archives a copy of the JUnit test report that shows the issue.

https://github.com/simondean/kotlin-junit-run-blocking-issue

Here's an example test report: https://github.com/simondean/kotlin-junit-run-blocking-issue/suites/3996726768/artifacts/100745995

Context

• Used versions (Jupiter/Vintage/Platform): org.junit:junit-bom:5.8.1 - see https://github.com/simondean/kotlin-junit-run-blocking-issue for more info
• Build Tool/IDE: Gradle from command line and IntelliJ

Deliverables

• ...

[sbrannen]

Related to:

• Introduce error handling mechanism for validation and discovery errors #242
• Forbid return values from test and lifecycle methods #835
• IsTestableMethod silently ignores @Test annotated methods that return a value #2244

[sbrannen]

Closing as a duplicate of #2244 and #242.

See #2244 (comment) for details.

[simondean]

Here's what the code looks like in one of the real world codebases where I'm seeing this issue:

@Test
fun `some test`() = runBlocking {
  // Stuff happens here
  
  // verify() is verifying a Koklin suspending function.  The generic return type of the verify() method is affected by this 
  verify(someObject).someSuspendFunction()
}