Memory usage increased significantly in 1.9.10

[sgrimm]

Describe the bug
Dokka 1.9.10 requires over 30% more memory than 1.9.0.

Expected behaviour
The memory requirements stay roughly the same across patch versions.

To Reproduce
Example: https://github.com/terraware/terraware-server/

This project's gradle.properties sets the maximum JVM heap size to 4096MB.

./gradlew dokkaHtml succeeds on both x86 and ARM systems using Dokka 1.9.0.

Upgrade to Dokka 1.9.10 as in terraware/terraware-server#1420 and ./gradlew dokkaHtml fails with a "Java heap space" error.

Editing gradle.properties to set the maximum heap size to 5.5GB (-Xmx5632m) causes Dokka to run successfully.

Dokka configuration
Configuration of dokka used to reproduce the bug

tasks.withType<DokkaTask>().configureEach {
  dokkaSourceSets {
    named("main") {
      outputDirectory = file("docs/dokka")
      moduleName = "Terraware Server"
      includes.from(fileTree("src/main/kotlin") { include("**/Package.md") })
      sourceLink {
        localDirectory = file("src/main/kotlin")
        remoteUrl =
            URI("https://github.com/terraware/terraware-server/tree/main/src/main/kotlin").toURL()
        remoteLineSuffix = "#L"
      }
    }
  }
}

Installation

• Operating system: macOS 13.5.2 (ARM)
• Build tool: Gradle 8.4
• Dokka version: 1.9.10
• JVM version: Corretto 20.0.2.9.1

Additional context
The example project uses a Dokka plugin to add Mermaid support, but commenting out that plugin in build.gradle.kts doesn't help.

Are you willing to provide a PR?
Yes, but I have no idea where to start looking.

[ianbrandt]

I just had to raise my Gradle Daemon max heap from 11G to 16G due to Dokka 1.9.20 OutOfMemoryErrors:

[ianbrandt]

Just an FYI, I noticed this past comment on #1405 from @Kordyjan:

Dokka is indeed memory-intensive in some parts, which are not yet optimized, but it should only affect heap space.

[whyoleg]

Hey @sgrimm and @ianbrandt!
In Dokka 2.0.0 we slightly improved memory usage, could you check how it affects your projects?
For reference, the PR with some measurements is #3800
Also, one more fix with potential improvements to memory usage will be available in next version (#4008)

[ianbrandt]

Hi @whyoleg,

A colleague looked into upgrading our project from Dokka 1.9.20 to 2.0.0. Unless there was a misunderstanding, they found that the dokka(project(...)) dependencies in our report aggregation project didn't result in transitive project dependencies being included (like they are with Gradle's Test Report Aggregation Plugin and JaCoCo Report Aggregation Plugin).

Is that correct regarding how Dokka 2.0.0 documentation aggregation currently works?

We have over 450 subprojects in our build. Maintaining a flat list of the ones that should be included in documentation aggregation wouldn't be very practical for us. I believe any scripting to try to accumulate subprojects that have the Dokka plugin applied may end up violating the requirements for Gradle's upcoming Project Isolation and Parallel Configuration features. We wouldn't want to do that because parallel configuration is probably going to offer us significant build performance gains.