MachinePool.Spec.ProviderIDList may not scale well

[mboersma]

What steps did you take and what happened:

In working on the MachinePool Machines proposal in #6088, we collectively rejected the idea of attaching an infrastructureRefsList to each MachinePool spec.

While this would be convenient, the overall size of a MachinePool could become large if the number of replicas / instances grows. Its representation is transferred over the network and stored in etcd. It's easy to imagine a long list of structs causing performance problems with a thousand-node cluster, for example. (No, we didn't test this, we are simply anticipating a possible problem.)

Similarly, the existing ProviderIDList on a MachinePool spec could become large in a large cluster. See this comment.

We should investigate if a large ProviderIDList actually causes problems–maybe this could be tested with DockerMachinePool? It might also be possible to deprecate use of ProviderIDList after #6088 has been implemented.

What did you expect to happen:

Anything else you would like to add:

Environment:

• Cluster-api version:
• Minikube/KIND version:
• Kubernetes version: (use kubectl version):
• OS (e.g. from /etc/os-release):

/kind bug

[fabriziopandini]

/milestone v1.2

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle stale
• Mark this issue or PR as rotten with /lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[fabriziopandini]

/lifecycle frozen

[fabriziopandini]

/triage accepted
/help

[k8s-ci-robot]

@fabriziopandini:
This request has been marked as needing help from a contributor.

Guidelines

Please ensure that the issue body includes answers to the following questions:

• Why are we solving this issue?
• To address this issue, are there any code changes? If there are code changes, what needs to be done in the code and what places can the assignee treat as reference points?
• Does this issue have zero to low barrier of entry?
• How can the assignee reach out to you for help?

For more details on the requirements of such an issue, please see here and ensure that they are met.

If this request no longer meets these requirements, the label can be removed
by commenting with the /remove-help command.

In response to this:

/triage accepted
/help

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[k8s-triage-robot]

This issue has not been updated in over 1 year, and should be re-triaged.

You can:

• Confirm that this issue is still relevant with /triage accepted (org members only)
• Close this issue with /close

For more details on the triage process, see https://www.kubernetes.dev/docs/guide/issue-triage/

/remove-triage accepted

[fabriziopandini]

/priority important-longterm

[fabriziopandini]

@mboersma @willie-yao @Jont828 to re-asses this issue and check if this is in the critical path for MP graduation

[fabriziopandini]

kind ping

[mboersma]

We should investigate if a large ProviderIDList actually causes problems

I think this is still a potential issue, but we still need to do some testing to see if it's a real-world limitation. I am doing some large cluster testing currently in Azure and I'll try to orient that toward MachinePools and see if I can collect any useful data.

Thanks for the nudge @fabriziopandini!

[sbueringer]

/triage accepted