Add CSI driver for AWS EBS

[embik]

What this PR does / why we need it:
Adds the CSI driver for AWS EBS volumes to kubeone, offering an out-of-tree alternative to the built-in driver.

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #1565

Special notes for your reviewer:

• The documentation for the CSI driver says that the csi-snapshotter needs to be installed as a prerequisite if the snapshotting feature is supposed to be used. It's not included in the YAML files published by the CSI driver. Should we embed it into this addon?
• The resources have been adapted to a) run on the control plane nodes only and b) use the system's identity. I can change this to using the AWS credentials already passed to machine-controller.

Does this PR introduce a user-facing change?:

Support for AWS EBS via CSI driver added

[kubermatic-bot]

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

[embik]

/assign @xmudrii

I don't understand the failing test case tbh. It seems to be related to YAML processing, which this PR doesn't touch as far as I can say.

[kron4eg]

/retest

[kron4eg]

@embik this is a flake fail, it's based on the iteration over the map ordering and of course in Go this is randomized so sometimes we get this error. nothing to worry about.

[xmudrii]

/assign
I’ll take a look and test this on Monday.

[xmudrii]

The documentation for the CSI driver says that the csi-snapshotter needs to be installed as a prerequisite if the snapshotting feature is supposed to be used. It's not included in the YAML files published by the CSI driver. Should we embed it into this addon?

That's not all. The docs state that we should also enable a bunch of feature gates on Kubelet and API server. Please take care of those feature gates as well.

Regarding the CSI snapshotter, we deploy it for some other providers, so I think it makes sense to do it for AWS too. Take a look at Azure or vSphere CSI addons as an example.

The resources have been adapted to a) run on the control plane nodes only and b) use the system's identity. I can change this to using the AWS credentials already passed to machine-controller.

I think this is fine.

As a side note, the problem with hostnames for the worker nodes will be resolved by kubermatic/machine-controller#1087.

[embik]

@xmudrii the feature gates listed there should be all enabled by default in the Kubernetes versions we support. I assume the documentation lists them for completeness' sake or hasn't been updated:

https://kubernetes.io/docs/reference/command-line-tools-reference/feature-gates/

• CSINodeInfo enabled by default since 1.14 as Beta (GA since 1.17)
• CSIDriverRegistry enabled by default since 1.14 as Beta (GA since 1.18)
• CSIBlockVolume enabled by default since 1.14 as Beta (GA since 1.18)
• VolumeSnapshotDataSource enabled by default since 1.17 as Beta (GA since 1.20)

So I think we don't need to change anything about the feature flags.

[kubermatic-bot]

LGTM label has been added.

Git tree hash: 9e103a1a4bffeae279076bf08d61e0cd50584288

[xmudrii]

/approve

[kubermatic-bot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: embik, xmudrii

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [xmudrii]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment