Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor DiktatMavenPluginIntegrationTest to ease troubleshooting in case of test failures #1304

Merged
Merged
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -5,60 +5,105 @@ import com.soebes.itf.jupiter.extension.MavenJupiterExtension
import com.soebes.itf.jupiter.extension.MavenTest
import com.soebes.itf.jupiter.maven.MavenExecutionResult
import org.junit.jupiter.api.Assertions
import org.junit.jupiter.api.BeforeEach
import org.junit.jupiter.api.TestInfo
import java.io.File
import kotlin.io.path.ExperimentalPathApi
import kotlin.io.path.Path
import kotlin.io.path.deleteIfExists
import kotlin.io.path.div
import kotlin.io.path.readText

/**
* Integration tests for diktat-maven-plugin. Run against the project from diktat-examples.
* The whole pipeline is as follows:
* * For each test case, test data is copied from examples with respect to maven-itf requirements, .mvn/jvm.config and .mvn/maven.config are copied too
* Note: for maven itf test name should equal example project's directory name, which we have in pom.xml.
* * maven-failsafe-plugin launches tests; for each test case a separate maven process is spawned. diktat.version is taken from .mvn/maven.config
* * For each test case, test data is copied from examples with respect to maven-itf requirements, .mvn/jvm.config and .mvn/maven.config are copied, too.
* **Note**: for maven its test name should equal example project's directory name, which we have in `pom.xml`.
unix-junkie marked this conversation as resolved.
Show resolved Hide resolved
* * maven-failsafe-plugin launches tests; for each test case a separate maven process is spawned. `diktat.version` is taken from `.mvn/maven.config`
* and the exact value is written when maven copies resources.
* * maven execution results are analyzed here; .mvn/jvm.config is used to attach jacoco java agent to every maven process and generate individual execution reports
* * maven execution results are analyzed here; `.mvn/jvm.config` is used to attach jacoco java agent to every maven process and generate individual execution reports.
*
* When within an IDE (e.g.: _IDEA_), don't run this test directly: it's
* expected to be executed from a forked JVM. Instead, run a `verify` _lifecycle
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you have a chance to try to run this forked JVM with remote debugging support? Such instructions would be useful here too

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, some notes about the remote debugger in the case of diktat will be great here.

But actually I love ultimate debugging tool called print

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's on my TODO list, will update the KDoc.

* phase_ for the `diktat-maven-plugin` submodule, as if you were running
*
* ```console
* $ mvn -pl diktat-maven-plugin verify
* ```
*
* from your terminal. If multiple JDKs are installed, be sure to pass
* `JAVA_HOME` to the _run configuration_, so that the parent and the forked
* JVMs have the same version.
*/
@OptIn(ExperimentalPathApi::class)
@MavenJupiterExtension
class DiktatMavenPluginIntegrationTest {
@BeforeEach
fun beforeEach(testInfo: TestInfo) {
val method = testInfo.testMethod.orElse(null) ?: return

(Path("target") / "jacoco-it-${method.name}.exec").deleteIfExists()
}

@MavenTest
@MavenGoal("diktat:check@diktat")
fun diktatCheck(result: MavenExecutionResult) {
fun diktatCheck(testInfo: TestInfo, result: MavenExecutionResult) {
Assertions.assertEquals(1, result.returnCode)
Assertions.assertFalse(result.isSuccessful)
Assertions.assertTrue(result.isFailure)

val mavenLog = result.mavenLog.stdout.readText()
Assertions.assertTrue(
mavenLog.contains("[FILE_NAME_MATCH_CLASS]")
)

mavenLog.assertContains("[FILE_NAME_MATCH_CLASS]")

val method = testInfo.testMethod.get()
File(result.mavenProjectResult.targetProjectDirectory, "target/jacoco-it.exec").copyTo(
File("target/jacoco-it-1.exec")
File("target/jacoco-it-${method.name}.exec")
)
}

@MavenTest
@MavenGoal("diktat:fix@diktat")
fun diktatFix(result: MavenExecutionResult) {
fun diktatFix(testInfo: TestInfo, result: MavenExecutionResult) {
Assertions.assertEquals(1, result.returnCode)
Assertions.assertFalse(result.isSuccessful)
Assertions.assertTrue(result.isFailure)

val mavenLog = result.mavenLog.stdout.readText()

Assertions.assertTrue(
mavenLog.contains(Regex("""Original and formatted content differ, writing to [:\w/\\]+Test\.kt\.\.\."""))
)
Assertions.assertTrue(
mavenLog.contains(Regex("There are \\d+ lint errors"))
)
Assertions.assertTrue(
mavenLog.contains("[MISSING_KDOC_TOP_LEVEL]")
)
mavenLog.assertContains(Regex("""Original and formatted content differ, writing to [:\w/\\]+Test\.kt\.\.\."""))
mavenLog.assertContains(Regex("There are \\d+ lint errors"))
mavenLog.assertContains("[MISSING_KDOC_TOP_LEVEL]")

val method = testInfo.testMethod.get()
File(result.mavenProjectResult.targetProjectDirectory, "target/jacoco-it.exec").copyTo(
File("target/jacoco-it-2.exec")
File("target/jacoco-it-${method.name}.exec")
)
}

/**
* Asserts that this string contains a [substring][other].
*
* @param other the expected substring.
*/
private inline fun String.assertContains(other: CharSequence,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought that assertContains is there in kotlin.test. Why do we need these?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed. But we don't have it added to our pom.xml yet. Should I add kotlin-test instead?

crossinline lazyMessage: () -> String = {
"The string: \"$this\" doesn't contain the substring: \"$other\""
}) {
Assertions.assertTrue(contains(other)) {
lazyMessage()
}
}

/**
* Asserts that this string contains a substring matching the [regex].
*/
private inline fun String.assertContains(regex: Regex,
crossinline lazyMessage: () -> String = {
"The string: \"$this\" doesn't contain any substring matching the regex: \"$regex\""
}) {
Assertions.assertTrue(contains(regex)) {
lazyMessage()
}
}
}