Dependency is causing 100% CPU use and preventing program startup

[mholt]

Sorry for abandoning the issue template here, it's mostly an "FYI" and a request for more information.

I have reasons to believe -- but I can't prove -- that one of the dependencies of this repo is causing all importers of it and thus all importers of these certificates packages to spin the CPU and hang at startup, sometimes. I filed an issue here with the relevant data: census-instrumentation/opencensus-go#1200

Have you experienced any issues with your CLI hanging at startup at all?

I ask, because when I eliminate the smallstep and opencensus dependencies, the CPU no longer spins, after many many trials.

I am not saying the bug is for sure in opencensus (or in smallstep for that matter) nor am I saying what is causing it, I'm just going from the limited data provided to me in the stack trace...

[maraino]

Hi @mholt, I've never seen this happening to me, neither in Go 1.14 nor 1.13. That dependency was automatically added by packages in cloud.google.com/go, in our case only cloud.google.com/go/kms/apiv1.

I'm not sure if you want to add kms support at this moment, but a workaround, that might fix your issue would be to import directly github.com/smallstep/certificates/kms/softkms instead of github.com/smallstep/certificates/kms.

Let me know if it works.

[mholt]

@maraino

I've never seen this happening to me, neither in Go 1.14 nor 1.13.

Hmm, very odd. Is it just me? (Are you on a Mac?)

That dependency was automatically added by packages in cloud.google.com/go, in our case only cloud.google.com/go/kms/apiv1.

I can confirm that. For the record:

$ go mod why go.opencensus.io
# go.opencensus.io
github.com/caddyserver/caddy/v2/modules/caddytls
github.com/smallstep/certificates/authority
github.com/smallstep/certificates/kms
github.com/smallstep/certificates/kms/cloudkms
cloud.google.com/go/kms/apiv1
cloud.google.com/go/kms/apiv1.test
cloud.google.com/go/internal/testutil
go.opencensus.io/trace
go.opencensus.io/internal
go.opencensus.io

Thanks for the workaround idea! But, I don't import the kms package directly. As you can see, it's github.com/smallstep/certificates/authority which imports it.

For now I've been testing by eliminating the smallstep stuff completely, on a branch.

As far as I can tell (after many many trials), it does work when smallstep & opencensus is not imported.

[mholt]

Are there any package-level var ... = ... initializations in smallstep that you know of that involve a function call? (Kinda hard to do a search for that, heh.) Just trying to cover all the bases as to what could be causing a CPU spin at init time... and only sometimes, at that...

[maraino]

Hmm, very odd. Is it just me? (Are you on a Mac?)

I'm on a Mac too. And I haven't heard anything from other people running mac or linux.

Thanks for the workaround idea! But, I don't import the kms package directly. As you can see, it's github.com/smallstep/certificates/authority which imports it.

I missed that.

Are there any package-level var ... = ... initializations in smallstep that you know of that involve a function call?

I'm not sure at the moment, we have some init too, but I'm not sure how that is relevant. As long as go.opencensus.io is imported, it will run their init methods.

[mholt]

Yeah, that's a good point. I did try importing just that package but wasn't able to reproduce the bug, but maybe it needs more trials in that configuration. Or, I wonder if it's some combination of imports? I dunno.

I'm gonna try for another day or through this week... and will just close this issue if I haven't been able to pin it down more.

[mholt]

Someone much smarter than me was able to reproduce the bug, and it appears fairly certain to be an issue with the Go 1.14 scheduler or, at least, the Go 1.14 runtime. Sorry for the noise!