WIP: Make QuarkusBuild not pollute Gradle's build cache

[snazy]

Currently the QuarkusBuild task implementation adds even large build artifacts and unmodified dependency jars to Gradle's build cache. This pollutes the Gradle build cache quite a lot and therefore lets archived build caches become unnecessary huge.

This change updates the build logic to fix this behavior by not adding dependencies and large build artifacts, like uber-jar and native binary, to the Gradle build cache.

The QuarkusBuild task has been split into three tasks:

1. a new QuarkusBuildDependencies task that only collects the contents for build/quarkus-app/lib
2. a new QuarkusBuildApp task that to collect everything else from a Quarkus build (everything except the build/quarkus-app/lib)
3. the QuarkusBuild task now combines the outputs of the above two tasks

QuarkusBuildDependencies (named 'quarkusDependenciesBuild) is not cacheable, because it only collects dependencies, which come either from a repository (and are already available locally elsewhere) or are built by other Gradle tasks. This task is only executed if the configured Quarkus package type requires the "quarkus-app" directory (fast-jar+native). It's "build working directory" is build/quarkus-build/dep`.

QuarkusBuildApp (named quarkusAppBuild) collects the contents of the "quarkus-app" directory excluding the lib/ directory are cacheable, which is the default for CI environments. Non-CI environments still cache all outputs, even uber-jars and native binaries to retain the existing behavior for developers and keep build turn-around times low. CI environments can opt-in to add even huge artifacts to Gradle's build cache by explicitly setting the cacheUberAndNativeRunners property in the Quarkus extension to true. It's "build working directory" is build/quarkus-build/app.

Since QuarkusBuild only combines the outputs of the above two tasks, the same "CI vs local" caching behavior as for the QuarkusBuildApp task applies. To make "up to date" checks (kind of) more reliable, all outputs are removed first. This means, that for example an existing uber-jar in build/ will disappear, when the build's package type is "fast-jar". This behavior can be disabled by setting the cleanupBuildOutput property on the Quarkus extension to false.

Both QuarkusBuildDependencies and QuarkusBuildApp can trigger an actual Quarkus application build. That Quarkus app build will only be triggered when needed and only once per Gradle build.

The task names quarkusDependenciesBuild and quarkusAppBuild are intentionally "that way around". Letting the names of these tasks begin with quarkusBuild... could confuse users, who use abbreviated task names on the command line (for example ./gradlew qB is automagically expanded to ./gradlew quarkusBuild, which would become ambiguous with quarkusBuildDependencies and quarkusBuildApp).

Relates to: #30852

Gradle caching in CI

Unless the cacheLargeArtifacts property on the quarkus extension is set to true, the output of/for the package type fast-jar minus the dependency jars is cached by Gradle's build cache (similar for legacy-jar).

Gradle caching (non CI)

Basically everything is cacheable to allow fast(er) local development turn-around cycles.

[snazy]

/cc @geoand this is a working draft. No integration tests or the like yet. I've chosen 2.16 as the base branch, because we have no choice to move to Jakarta EE 10 quickly :(

[snazy]

Remaining TODOs for this PR:

• Tests
• Docs

[geoand]

Thanks a lot for this!

I've chosen 2.16 as the base branch

I am pretty certain we won't be merging changes of this magniture to 2.16

[aloubyansky]

Also worth taking into account that we also support application.yaml, config profiles, env vars, etc.

[snazy]

Also worth taking into account that we also support application.yaml, config profiles, env vars, etc.

Good point. I actually thought quite a lot about the way the "effective configuration" is calculated in "all the places". One thing that I think is a bug in the current implementation is that configuration in application.[properties|yaml] has priority over the configuration in the build scripts. I think, the proper priority should be:

1. system properties (quarkus.*)
2. environment (QUARKUS_*)
3. config in Gradle build.gradle[.kts] plugin extension
4. config in Gradle build.gradle[.kts] QuarkusBuild task
5. config from application.[properties|yaml]

WDYT?

[aloubyansky]

Sounds good @snazy

[snazy]

Hum - changing the priority of the configurations as described above is much trickier than I expected. Looks like an existing src/main/resources/application.(properties|yaml|yml) takes precedence in the augmented build (hope I got that term right) over the build-properties passed to it. I could push the effective properties via System.setProperties, but that's really bad practice in Gradle (think: parallel builds). Need to look a bit deeper into that. So far, I've not updated the priorities yet.

Gradle integration tests are passing locally.

Opened #31165 to replace this one, because this PR is based on 2.16