mount static cephFS pvc via fuse inside Kubernetes pod with fsName=""

[apetrunev]

Hi! i need a little help. I cannot figure out how to mount cephFS static pvc with empty fsName paramteter. I have a volume on a ceph cluster and directory i want to mount via ceph-fuse. It works fine on linux cli but i cannot mount the dir inside a pod. It looks like it is ok to have fsName defined as an empty string.

Example

This works fine on cli:

ceph-fuse -c /etc/ceph/ceph.conf -n client.user-rnd /mnt/ceph/rnd/test -r /rnd/test

PV manifest:

apiVersion: v1
kind: PersistentVolume
metadata:
  name: ceph-rnd-test-pv
spec:
  accessModes:
  - ReadWriteMany
  capacity:
    storage: 1Gi
  csi:
    driver: cephfs.csi.ceph.com
    nodeStageSecretRef:
      # node stage secret name
      name: csi-cephfs-secret
      # node stage secret namespace where above secret is created
      namespace: default
    volumeAttributes:
      # Required options from storageclass parameters need to be added in volumeAttributes
      clusterID: "-cluster-id-"
      fsName: ""
      staticVolume: "true"
      rootPath: /rnd/test
      mounter: fuse
    # volumeHandle can be anything, need not to be same
    # as PV name or volume name. keeping same for brevity
    volumeHandle: ceph-rnd-test-pv
  persistentVolumeReclaimPolicy: Retain
  volumeMode: Filesystem

When i mount this PV inside a pod i get error:

MountVolume.MountDevice failed for volume "ceph-rnd-test-pv" : rpc error: code = Internal desc = rpc error: code = Internal desc = parameter 'fsName' cannot be empty

Environment details

ceph-csi chart version 3.8.1 (3.9.0)
kubernetes version: 1.26.6

[apetrunev]

Is it possible to set parameters such that option --client_mds_namespace is not appended?

[wolgod]

The default is to mount it through the kernel. Do you want to mount it through fuse through pv?

[apetrunev]

The default is to mount it through the kernel. Do you want to mount it through fuse through pv?

I want to be able to use it both ways. One problem is that options --client_mds_namespace is added implicitly and i don't know what value i should pass there. I have only settings that allow me mount ceph path via cephfs(example https://github.com/kubernetes/examples/tree/master/volumes/cephfs/): username, secret, monitors, path. Ceph-csi plugin requires more knowledge about ceph volume then i may have. As an example I don't know what to put as value for fsName (i should probably ask a team that support ceph about that). My use case seems to more suitable for cephfs (deprecated). Having configuration options for cepfs seem not enough for using ceph-csi plugin.

[Madhu-1]

@apetrunev what is the output of ceph fs ls

[apetrunev]

@Madhu-1

2024-01-12T10:11:59.610+0300 7f1548c1a700 -1 auth: unable to find a keyring on /etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin,: (2) No such file or directory
2024-01-12T10:11:59.610+0300 7f1548c1a700 -1 AuthRegistry(0x7f154405ea88) no keyring found at /etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin,, disabling cephx
2024-01-12T10:11:59.610+0300 7f1548c1a700 -1 auth: unable to find a keyring on /etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin,: (2) No such file or directory
2024-01-12T10:11:59.610+0300 7f1548c1a700 -1 AuthRegistry(0x7f1548c19130) no keyring found at /etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin,, disabling cephx
2024-01-12T10:11:59.746+0300 7f1541d9b700 -1 monclient(hunting): handle_auth_bad_method server allowed_methods [2] but i only support [1]
2024-01-12T10:12:02.746+0300 7f1542d9d700 -1 monclient(hunting): handle_auth_bad_method server allowed_methods [2] but i only support [1]
2024-01-12T10:12:02.790+0300 7f1548c1a700 -1 monclient: authenticate NOTE: no keyring found; disabled cephx authentication

I use unprivileged roles for myself and for accessing ceph dirs within kubernetes pods. These roles alose have access only to certain dirs.

[Madhu-1]

@apetrunev that should be fine, can you use the output of ceph fs ls output (which is fsname in PV yaml )` that you will get when you run this command in your ceph cluster.

[Madhu-1]

@apetrunev i made #4372 to see if that is enough

[apetrunev]

@apetrunev that should be fine, can you use the output of ceph fs ls output (which is fsname in PV yaml )` that you will get when you run this command in your ceph cluster.

unfortunately i cannot access them. I got the fsName but did not test how it works because my point here is to move smoothly from kuber cephfs to ceph-csi with the same settings.

[apetrunev]

@apetrunev i made #4372 to see if that is enough

looks nice, that way --client_mds_namespace is not used. It seems that is enough for my case.