Parent project classloader scopes should be locked before evaluating any children project

[eskatos]

When using evaluationDependsOn(p) or Configure-On-Demand, it can happen a project is evaluated before its parent.

When this happens, parent classloader scopes aren't locked. This works in most cases with Groovy scripts because Gradle doesn't run with -Dorg.gradle.classloaderscope.strict=true. It currently fails with Kotlin scripts because the Kotlin DSL provider requires parent scopes to be locked in order to deterministically compute script compilation classpaths.

Groovy or Kotlin, letting the classpath vary depending on the order the projects are evaluated lead to hard to debug errors particularly when combined with configure-on-demand:

Kotlin
Why my script fails to compile when I run ./gradlew :foo:bar but compiles cleanly when I run ./gradlew bar?

Groovy
Why my script fails to execute when I run ./gradlew :foo:bar but executes properly when I run ./gradlew bar?

As suggested by @adammurdoch, this should be fixed by introducing some sort of implicit script classpath evaluation dependency between a child project and its parent.

See gradle/kotlin-dsl-samples#782 for some context

[JLLeitschuh]

Is this relationship something that would be specified in the settings.gradle file or would it simply be some sort of standard that can't be modified?

[eskatos]

Gradle class loader scope hierarchy follows the project hierarchy. Note that configure-on-demand (COD) or not, the root project is always configured first.

evaluationDependsOn() and COD don't respect that hierarchy, they would configure parents projects first if they did. It could mean more projects configured.

If you are looking for workarounds see gradle/kotlin-dsl-samples#607 (comment)

[eskatos]

The issue can be reproduced without COD with Groovy scripts as follows.

// settings.gradle
include(":a", ":b:c")

// a/build.gradle
evaluationDependsOn(":b:c")

// b/build.gradle
plugins {
    id("org.gradle.hello-world") version "0.2" apply false
}

// b/c/build.gradle
import org.gradle.plugin.HelloWorldTask

https://scans.gradle.com/s/h3p6l5js65fig

If you rename :a to :d, no more failure.

[eskatos]

What adds to the difficulty to track that kind of issues with Groovy scripts is that if, in the reproducer above, b/c/build.gradle was using the org.gradle.plugin.HelloWorldTask without making a static reference to the type (the import above), then the Groovy script would compile, but would be relying on Groovy dynamics and reflection at runtime. Add COD to the mix and it also depends on what task you request on the command line.

Another manifestation can be observed by replacing the content of b/c/build.gradle in the reproducer above by:

// b/c/build.gradle
plugins {
    id("org.gradle.hello-world")
}

Then you get a different error for the same issue: Plugin [id: 'org.gradle.hello-world'] was not found
https://scans.gradle.com/s/snxz65x332enm

Again, if you rename :a to :d, no more failure, because then :b gets evaluated first.

[SUPERCILEX]

Any updates? I just encountered this, kinda unpleasant.

[mkobit]

We are running into this in one of our builds that is mixed Groovy/Kotlin build scripts but use neither COD nor evaluationDependsOn. We removed both as we ran into this a few times in converting, but see it now in a medium-sized multi-project build where most of the configuration is done in the root build.gradle.kts

[mkobit]

I can't share the full project but I do see it happen where we have something like this:

build.gradle.kts

allprojects {
  // some configuration of plugins.withId, tasks, other stuff
}

tasks {
  val copyBunchOfStuff by creating(Copy::class) {
    from(tasks.findByPath(":code:in:sub:assembleTar")) {
        rename { "renamed-assembly.tar.gz" }
    }
  }
}

Getting rid of the tasks.findByPath seems to get around the classloader scope exception. Hopefully this gives some more insight!

[eskatos]

@mkobit I can't reproduce with your snippet, any chance you could narrow it down to a reproducer?

Just a gut feeling, but does from(provider { tasks.findByPath(":code:in:sub:assembleTar") }) { .. } fix it?

[ZacSweers]

This issue is several years old now. We have a somewhat consistent repro, can we help report anything?

[andy-maca]

We made a workground, hope it will work also for some of you.

We have many projects, some of them use kotlin script, we have problem when run task of kotlin project with CoD(it can be easily reproduced by deleting the gradle cache and stopping the deamon), the root project is using groovy, so we add the following code to the build script of the root project, the idea is, for each kotlin project, evaluate its parent projects first.

StartParameter startParameter = gradle.getStartParameter();
if(startParameter.isConfigureOnDemand()) {
  subprojects.each {s ->
    if(s.buildscript.sourceFile.name.endsWith(".kts")){ //kotlin project
      def current = s
      def depPath = current.path
      while(current.path != rootProject.path && current.parent.path != rootProject.path) {
        depPath += " -> ${current.parent.path}"
        current = current.evaluationDependsOn(current.parent.path)
      }
      logger.info("Adding kotlin project dependency for CoD: ${depPath}")
    }
  }
}

[mikejhill]

Thanks @andy-maca for your solution. We've sporadically run into this issue for our projects since switching over to Kotlin build scripts. We had been re-ordering dependencies to try to encourage Gradle to resolve the parent projects first, which worked in most cases but was inconsistent and painful. I implemented your changes today and it seems to have fixed things for us so far.

subprojects {
	// Address https://github.com/gradle/gradle/issues/4823: Force parent project evaluation before sub-project evaluation for Kotlin build scripts
	if (gradle.startParameter.isConfigureOnDemand
			&& buildscript.sourceFile?.extension?.toLowerCase() == "kts"
			&& parent != rootProject) {
		generateSequence(parent) { project -> project.parent.takeIf { it != rootProject } }
				.forEach { evaluationDependsOn(it.path) }
	}
}

[tasomaniac]

There is not much change when running the whole app from the root. But there would be a difference when running unit tests of a leaf module. In that case, the whole repo will be configured first before running the tests.

[chrisjenx]

There is not much change when running the whole app from the root. But there would be a difference when running unit tests of a leaf module. In that case, the whole repo will be configured first before running the tests.

That was my understanding too, when working on feature modules you would only configure your direct modules required. In some cases that could be a large difference?

[eskatos]

This issue is about using evaluationDependsOn and configure-on-demand. Other use cases should be fixed by #14616 already.

@Tolriq, sorry you are getting into troubles with dependencies catalogs and thanks for the report. The fact that this happens without configure-on-demand enabled make it look like a bug in how dependencies catalogs accessors work. Could you please open a separate issue, ideally with a self contained reproducer?

[JavierSegoviaCordoba]

@Tolriq your issue is via CLI or syncing the IDE

[abhishekdewan101]

Disabling configureOnDemand helped fixed my issue

[nayash]

In my case, it was happening while creating & adding a new module to the Android project. Turns out, it was happening because Android Studio was adding these 2 lines to project level gradle file:

id("com.android.library") version "7.1.2" apply false
id("org.jetbrains.kotlin.android") version "1.6.10" apply false

Once I removed these it worked. I already had many modules added to the project without any issues, so I looked for what was different for this new one and that's how I found the issue. Would suggest anyone with this issue to do the same.

[dsvoronin]

Another case i found today:

root/settings.gradle.kts:

pluginManagement {
    apply(from = "myplugin.settings.gradle.kts")
}

any exception thrown inside this custom settings script during IDE gradle sync leads to same:

java.lang.IllegalArgumentException: org.gradle.api.internal.initialization.DefaultClassLoaderScope@568c7753 must be locked before it can be used to compute a classpath!
	at org.gradle.kotlin.dsl.provider.KotlinScriptClassPathProvider.exportClassPathFromHierarchyOf(KotlinScriptClassPathProvider.kt:152)
	at org.gradle.kotlin.dsl.provider.KotlinScriptClassPathProvider.computeCompilationClassPath(KotlinScriptClassPathProvider.kt:148)
	at org.gradle.kotlin.dsl.provider.KotlinScriptClassPathProvider.access$computeCompilationClassPath(KotlinScriptClassPathProvider.kt:95)
	at org.gradle.kotlin.dsl.provider.KotlinScriptClassPathProvider$compilationClassPathOf$1.invoke(KotlinScriptClassPathProvider.kt:144)
	at org.gradle.kotlin.dsl.provider.KotlinScriptClassPathProvider$compilationClassPathOf$1.invoke(KotlinScriptClassPathProvider.kt:95)
	at org.gradle.kotlin.dsl.provider.KotlinScriptClassPathProviderKt$sam$java_util_function_Function$0.apply(KotlinScriptClassPathProvider.kt)

running ./gradlew help in terminal shows correct stack trace

[sschuberth]

I believe I'm running into the same bug with Gradle 8.0.1 (interestingly only on Windows) when inputs / from for a Copy-task depend on Jars produced by tasks in another project, see this:

val processResources = tasks.named<Copy>("processResources").configure {
    val gradleModelProject = project(":plugins:package-managers:gradle-model")
    val gradleModelJarTask = gradleModelProject.tasks.named<Jar>("jar")
    val gradleModelJarFile = gradleModelJarTask.get().outputs.files.singleFile

    val gradlePluginProject = project(":plugins:package-managers:gradle-plugin")
    val gradlePluginJarTask = gradlePluginProject.tasks.named<Jar>("jar")
    val gradlePluginJarFile = gradlePluginJarTask.get().outputs.files.singleFile

    // As the Copy-task simply skips non-existing files, add explicit dependencies on the Jar-tasks.
    dependsOn(gradleModelJarTask, gradlePluginJarTask)

    // Bundle the model and plugins JARs as resources, so the inspector can copy them at runtime to the init script's
    // classpath.
    from(gradleModelJarFile, gradlePluginJarFile)

    // Ensure constant file names without a version suffix.
    rename(gradleModelJarFile.name, "gradle-model.jar")
    rename(gradlePluginJarFile.name, "gradle-plugin.jar")
}

Running

./gradlew :plugins:package-managers:gradle-inspector:processResources --rerun-tasks --no-build-cache -Dorg.gradle.configureondemand=true

fails on Windows with a lot of unresolved referenced and

* What went wrong:
Execution failed for task ':plugins:package-managers:gradle-plugin:compileKotlin'.
> A failure occurred while executing org.jetbrains.kotlin.compilerRunner.GradleCompilerRunnerWithWorkers$GradleKotlinCompilerWorkAction
   > Compilation error. See log for more details

But when running the failed task :plugins:package-managers:gradle-plugin:compileKotlin manually, it succeeds.

Also, running with configure-on-demand disabled works:

./gradlew :plugins:package-managers:gradle-inspector:processResources --rerun-tasks --no-build-cache -Dorg.gradle.configureondemand=false

Wrapping inputs in provider {} as suggested here unfortunately does not help.

[eskatos]

Here's a summary of complete reproducers shared in this issue and their outcome as of Today.

As noted in the comment for the second one, they are about using evaluationDependsOn() even without enabling Configure On Demand.
They also all fail in Groovy when they fail in Kotlin and respectively succeed in Groovy when they succeed in Kotlin. The error messages are different though but that's not the most important thing here.

---

Reproducers:

1/ #4823 (comment)

• Kotlin DSL https://github.com/eskatos/issue-reproducers/tree/main/gradle/issue-4823/repro-1-kotlin
• Groovy DSL https://github.com/eskatos/issue-reproducers/tree/main/gradle/issue-4823/repro-1-groovy

2/ #4823 (comment)

• Kotlin DSL https://github.com/eskatos/issue-reproducers/tree/main/gradle/issue-4823/repro-2-kotlin
• Groovy DSL https://github.com/eskatos/issue-reproducers/tree/main/gradle/issue-4823/repro-2-groovy

---

Results with links to build scans:

Reproducer	Last version failing	First version fixed
repro-1-kotlin	🔴 7.3	🟢 7.4
repro-1-groovy	🔴 7.3	🟢 7.4
repro-2-kotlin	🔴 7.3	🟢 7.4
repro-2-groovy	🔴 7.3	🟢 7.4

---

All the actionable reproducers from this issue are fixed since Gradle 7.4.
I could not figure out what PR in 7.4.x fixed those use cases though.
I'm going to close this as fixed in 7.4.

There are other open issues with Configure On Demand, see https://github.com/gradle/gradle/issues?q=is%3Aopen+is%3Aissue+label%3Ain%3Aconfigure-on-demand

If you come across this issue, please look for existing ones and add your 👍 there or create a new issue.

---

Also note that most of the classloading issues were already fixed in 6.8 by

• Always configure intermediate parents when a child project is configured #14616

[unoexperto]

I see the PR merged but I'm confused by how to use it. Could you please elaborate ? I'm on Gradle 8.5 and my faulty script looks like this:

tasks {
    wrapper {
        gradleVersion = "8.5"
        distributionType = Wrapper.DistributionType.BIN
    }

    create("publishSdkLocal") {
        val publicationTasks = this.project.flattenChildren().filter { sdkProjects.contains(it.name) }.map { p ->
            p.getTasksByName("publishToMavenLocal", false)
        }
        dependsOn(publicationTasks)
    }
}

fun Project.flattenChildren(): List<Project> {

    return if (this.childProjects.isNotEmpty()) {
        childProjects.values.flatMap { it.flattenChildren() } + this
    } else
        listOf(this)
}