-
Notifications
You must be signed in to change notification settings - Fork 455
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
spotless gradle plugin leaks memory (massively) #1194
Comments
Thanks for the bug report, I should have a fix published by tomorrow. |
Thanks, much appreciated. |
Ok, I after dealing with lineEndings as above, the next issue seems to be This affects After disabling those from my spotless config, another issue is shown with After disabling the So it appears the issue is a bit more wide spread than initially suggested, hinting to a bit of a conceptual "tug-of-war" between caching and lazyness. |
Thanks for the fix, I tested it. Unfortunately, if one of the projects in a multi-build fails (e.g. during |
…lity to give it a chance to null-out its initializing lambda. Addresses diffplug#1194 (comment)
Thanks for testing this, the commit above makes sure that |
Thanks, re-tested, unfortunately there seems to be still a reference that slips by in the "broken build" scenario, coming from JVisualVM seems to pick one of the uncollectable reference chains when displaying GC roots, so they were hidden by the earlier issue. Sorry for the "drop by drop" reporting... |
Are you kidding!? Thanks for sticking with this! I'm giving drop-by-drop fixes, and I think our incremental approach is the fastest way we could do this. The code design has definitely picked up some warts as the Gradle model has changed, but the lazyness is still helpful to how Spotless is used in Maven builds, so I think the little kludges we've had to accept are our best option. |
I think this is fixed in |
Thanks again, I tested the 6.6.0 release but still see the leak (in the "broken build" scenario) via the same reference chain as above (#1194 (comment)) ~~ As fair as I understand the code, the latest changes do not yet apply there, bc. spotless/lib/src/main/java/com/diffplug/spotless/java/RemoveUnusedImportsStep.java Line 32 in 49099f3
DelegateFormatterStep as per the lastest commit. ~~
|
Pls disregard my earlier analysis attempt - I see that |
Ok, did some more debugging and it appears that |
After some local patching it seems that losing the
I couldn't reproduce any leaks anymore after applying that change |
Hmm, while the "unlazying" cleans up the memory leaks, I have now a new problem at hand to deal with. We wrap spotless-gradle (among others) in our own inhouse convention plugin, which configures all of our projects consistently. However, now with the fixes above in place - spotless tries to resolve all its configuration file (headers, import order...) earlier than before, even before our custom task had a chance to export all those files into the project. The stacktrace looks like this:
So it looks like this all happens in This means I could now:
A better solution (in our use case) would be if I could configure the spotless extension directly with classpath resources, eliminating the need to export the spotless profile into each project directory altogether. |
Yeah, a consequence of fixing the memory leaks is that the |
Here's the issue re: loading resources from the classpath. |
I'm sorry, but would you mind to still apply the final fix (or something similar) suggested here? #1194 (comment) Edit: added a PR for that #1206 |
diffplug#1194 unlazy collections/delegates
diffplug#1194 unlazy collections/delegates
Thank you, fix confirmed - I don't see any leaks anymore 👍 |
Happy to reopen and squash this if you have time to trace GC roots, or if you want to wait for the issues below to close first that works too |
Hi there, while upgrading to the latest spotless I revisited the issue and indeed it seems to have regressed. Here are some GC roots holding project references, though I didn't quite wrap my head around whats happening. Edit: spotless-plugin-gradle-6.21.0 on gradle-8.3 |
bump |
Have the same problem |
We have started the process of removing It will take a while for this effort to fully complete, but there is progress on the horizon here. |
From my preliminary analysis of instabilities in our build environment I suspect that spotless gradle plugin is massively leaking memory in the gradle daemon.
It seems that via
GitAttributesLinieEndings$RelocatablePolicy
theJVMLocalCache
holds onto complete project references, which cannot be collected by a full gc of the daemon process.The image shows a full gc after the build of one of our projects (admittedly one having about 35 subprojects), with spotless applied vs. spotless not applied.
Also shown a GC root analysis on the daemon heap dump, showing the suspected offender.
The expectation would be that spotless only caches calculated values (and using weak cache values that could be collected if need be).
Using spotless-gradle-plugin 6.5.2
The text was updated successfully, but these errors were encountered: