Why are Kubelet extra mounts for Longhorn needed?

[M4t7e]

I'm currently working on integrating Longhorn to a Terraform based Talos deployment and I'm just wondering why this extra mounts for Kubelet are needed according to the Longhorn Talos documentation:

machine:
  kubelet:
    extraMounts:
      - destination: /var/lib/longhorn
        type: bind
        source: /var/lib/longhorn
        options:
          - bind
          - rshared
          - rw

They say:

These mounts are necessary to provide access to the host directories, and attach volumes required by Longhorn components.

I'm trying to understand the background behind this requirement. Why does Kubelet need access to these paths when Longhorn already has access to the host path?

[smira]

My answer is only about hostPath mounts (mounts of host directories into running Kubernetes pods and containers).

When a container is started with a hostPath mount, the actual mount operation is performed by the containerd (via CRI plugin), which run on the host in the host namespace (including mount namespace). So in terms of actual mount operation, kubelet is not involved into that.

In Talos Linux, kubelet itself runs in a container in a different mount namespace, so by default it doesn't see mounted user disks (volumes), neither it sees full contents of /var directory. For most basic volume mount, this is fine, as kubelet doesn't need to inspect the mount source or do anything about it.

But kubelet supports e.g. subPath mounts, which require kubelet to look into the actual host path. In this and similar cases, kubelet has to see the actual host path inside its own mount namespace (inside kubelet container). This is why machine.kubelet.extraMounts is important - it ensures that kubelet sees same picture as the actual mounter (e.g. containerd).

Same way, e.g. DirectoryOrCreate requires kubelet to see the actual host path.

[M4t7e]

Thank you so much! This was really helpful. So I can assume Longhorn offloads some actions to Kubelet, where these or similar functionalities are required?

I think I just needed a concrete example, as my initial naive thought was that Longhorn handles everything by itself.

[smira]

I explained above, this has nothing to do with Longhorn-provided PVs. It might only be related to the way Longhorn itself is deployed, and how it uses hostPath volumes. You don't have control over Longhorn deployment (I guess you just helm install it), so you need to inspect every hostPath mount right now and over any upgrades if you don't want to add extra mounts to the kubelet (which is totally safe).