Allow non-global metrics factory for workqueues

[austince]

Currently, all workqueues share the same metrics provider (see util/workqueue/metrics.go). For similar reasons to
kubernetes-sigs/controller-runtime#210 + kubernetes/community#2909, having a single global metrics provider for all consumers is inflexible.

Concretely, I'm trying to instrument my apps with OpenTelemetry — we have some dependencies (controller-runtime) that set a provider, but we also use workqueues in our code where I'd like to have a different provider. I don't see a way around this currently.

A fix for only the workqueue package could be done in a non-breaking way I'd think, by allowing the provider to be plumbed through the constructors, e.g. the ...rest arg option function pattern.

---

Has there been any discussions around the global metrics in the past? I can't find anything, but think this applies to other packages in this repo as well e.g. tools/metrics.go

[maherbeg]

+1 to this. We just saw an issue where transitively importing the controller-runtime package causes subsequent workqueue metric registration to fail. This is partially because that package uses init functions to register things rather than explicitly doing so, but again does it on the global factory which is shared.

[austince]

Discussed on the ML a bit: https://groups.google.com/g/kubernetes-sig-api-machinery/c/rs9aIppwWE0/m/TZ5TSHJ5BAAJ

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle stale
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[austince]

@ash2k @maherbeg if this is important to you guys, can you add your support here? kubernetes/kubernetes#114242

Otherwise, I'll let this issue + fix will go stale.