Implement the CCM/CSI migration

[xmudrii]

What this PR does / why we need it:

This PR is a follow-up of #1468. The following has been done in this PR:

• Improve stability of the CCM migration process implemented in Initial CCM migration implementation for OpenStack #1468 by:
  • Verifying that both kube-apiserver and kube-controller-manager are ready before updating Kubelet config and restarting it
  • Verifying that Kubelet has successfully restarted before proceeding to the next node
• Implement the CSI migration process
  • This assumes applying the appropriate feature gates on kube-apiserver, kube-controller-manager, and kubelet, as well as, deploying the CSI plugin
• Implement the --complete flag used to complete the CCM/CSI migration
  • The CCM/CSI migration process is done in two phases, where:
    • Phase 1 assumes disabling cloud-related in-tree controllers in favor of external CCM. The volume controllers are still left enabled until the worker nodes are not rolled out
    • Phase 2 assumes fully disabling the in-tree cloud provider

Known limitations and issues:

• The CCM/CSI migration currently works only for OpenStack
  • vSphere support will be added in a follow-up PR (Implement CCM/CSI migration for vSphere #1471 tracks vSphere support)
  • Support for other providers will be added in future KubeOne releases
• The OpenStack CCM/CSI migration document states that PersistentVolumes must be annotated, so the CSI driver can properly clean them up. This part isn't done. It's still unclear can this be automated, so I'll leave it for a follow up (Implement CCM/CSI migration for OpenStack #1452 tracks this issue)
• The migration process has been tested only with Kubernetes 1.22.1. I'll create a dedicated Epic for in-depth testing of this feature.

On top of that, this PR also removes the kubeadm v1beta1, which has been used for Kubernetes clusters up to 1.15. Since we're not supporting clusters older than 1.19 (#1466), it doesn't make much sense to maintain the unused API/code.

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #1451
Relevant to #1452

Does this PR introduce a user-facing change?:

Implement the CCM/CSI migration. The migration is currently available for OpenStack clusters. Support for other cloud providers will be added in future KubeOne releases. Make sure to familiarize yourself with the migration process before getting started by checking the command description.

/assign @kron4eg

[kubermatic-bot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: xmudrii

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [xmudrii]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[xmudrii]

/retest

[kubermatic-bot]

LGTM label has been added.

Git tree hash: 5168cb17024e8d850f63e42d505d0aa13f30bc8a