Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Speed up goimports usage #19740

Conversation

modular-magician
Copy link
Collaborator

Now that the go compiler is live I've started toying with bazel, and the extra time that goimports adds to each compile run is irking me. This PR speeds things up by running goimports once as a last compilation step rather than running it for each generated source file.

goimports performs two functions for us: removing imports that are declared but not used, and adding imports that are used but not declared. The former is fast, the latter is slow - like, a couple of orders of magnitude slower than the former. The difference exists because removing imports requires parsing only the go file that's being fixed, while adding imports potentially parses every go file in the project, the standard library, and in your GOPATH. goimports has an internal cache that avoids some repetitive parsing, and by using a single execution that runs over all of the generated files at once we're able to leverage that cache.

Here's some example timings on my M1 Macbook Pro. Before:

$ time make provider VERSION=beta OUTPUT_PATH="../terraform-provider-google-beta"
real	1m1.976s
user	1m33.130s
sys	2m37.034s

$ time make provider VERSION=ga OUTPUT_PATH="../terraform-provider-google"
real	1m0.521s
user	1m23.280s
sys	2m25.612s

After:

$ time make provider VERSION=beta OUTPUT_PATH="../terraform-provider-google-beta"
real	0m24.659s
user	0m38.855s
sys	0m18.785s

$ time make provider VERSION=ga OUTPUT_PATH="../terraform-provider-google"
real	0m22.059s
user	0m36.664s
sys	0m19.448s

YMMV of course, and full builds benefit a lot more than single product builds.

Ultimately the best way to speed up the compiler will be to include all imports that we're using, and the change to resource.go.tmpl picks the lowest hanging fruit (for context, 10 seconds of the speed up on my machine came from just those five lines). The new --show-import-diffs flag is useful for identifying the files that need import fixes.

Derived from GoogleCloudPlatform/magic-modules#11896

[upstream:73acb5d34b59a547b026e0a76618aa8d00e6e8de]

Signed-off-by: Modular Magician <magic-modules@google.com>
@modular-magician modular-magician merged commit eae436d into hashicorp:main Oct 3, 2024
4 checks passed
Copy link

github-actions bot commented Nov 8, 2024

I'm going to lock this pull request because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Nov 8, 2024
@modular-magician modular-magician deleted the downstream-pr-73acb5d34b59a547b026e0a76618aa8d00e6e8de branch November 17, 2024 00:53
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant