Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Provide capacity of block-mode RBD volumes #1831

Merged
merged 3 commits into from
May 25, 2021

Conversation

nixpanic
Copy link
Member

@nixpanic nixpanic commented Jan 12, 2021

Describe what this PR does

Metrics from block-mode volumes are not available from Kubelet. CephFS and RBD-with-filesystem do return metrics. This adds support for the capacity attribute for block-mode volumes.

Related issues

It seems Kubernetes does not call NodeGetVolumeStats got block-mode volumes yet. This will get added through kubernetes/kubernetes#97972 (merged)

Without the change in Kubernetes, this can not be tested (and we do not have tests for metrics either yet).


Show available bot commands

These commands are normally not required, but in case of issues, leave any of
the following bot commands in an otherwise empty comment in this PR:

  • /retest ci/centos/<job-name>: retest the <job-name> after unrelated
    failure (please report the failure too!)
  • /retest all: run this in case the CentOS CI failed to start/report any test
    progress or results

@nixpanic nixpanic added enhancement New feature or request component/rbd Issues related to RBD dependency/k8s depends on Kubernetes features labels Jan 12, 2021
@nixpanic nixpanic force-pushed the rbd/NodeGetVolumeStats branch 9 times, most recently from 8741757 to a4d1031 Compare January 18, 2021 10:01
e2e/pvc.go Show resolved Hide resolved
@nixpanic nixpanic force-pushed the rbd/NodeGetVolumeStats branch 5 times, most recently from 1b3a58a to 73c5826 Compare January 18, 2021 17:16
@nixpanic nixpanic marked this pull request as ready for review January 18, 2021 17:16
@nixpanic
Copy link
Member Author

/retest ci/centos/mini-e2e-helm/k8s-1.19

@nixpanic
Copy link
Member Author

/retest all

@nixpanic nixpanic force-pushed the rbd/NodeGetVolumeStats branch 3 times, most recently from d5a3ec9 to 73d9428 Compare January 26, 2021 16:27
@nixpanic
Copy link
Member Author

nixpanic commented May 6, 2021

ci/centos/upgrade-tests-cephfs failed again with #1969

ci/centos/mini-e2e/k8s-1.20 failed to bring up the minikube VM

@nixpanic
Copy link
Member Author

nixpanic commented May 6, 2021

/retest ci/centos/mini-e2e/k8s-1.20

@nixpanic
Copy link
Member Author

nixpanic commented May 7, 2021

/retest ci/centos/mini-e2e/k8s-1.20

@nixpanic
Copy link
Member Author

nixpanic commented May 7, 2021

/retest ci/centos/mini-e2e/k8s-1.20

Yet again #1969. But the job has been updated, so this is hopefully the last time 🤞

@nixpanic
Copy link
Member Author

nixpanic commented May 7, 2021

/retest ci/centos/mini-e2e/k8s-1.20

@nixpanic
Copy link
Member Author

nixpanic commented May 7, 2021

/retest ci/centos/mini-e2e/k8s-1.20

Hit #1969 for cephfs testing.

@nixpanic
Copy link
Member Author

/retest ci/centos/mini-e2e/k8s-1.20

@nixpanic
Copy link
Member Author

/retest ci/centos/mini-e2e/k8s-1.20

Again failed due to #1969

@nixpanic
Copy link
Member Author

/retest ci/centos/mini-e2e/k8s-1.20

@nixpanic
Copy link
Member Author

/retest ci/centos/mini-e2e/k8s-1.20

So annoying, #1969 yet again... We need more debugging of the minikube VM (get libvirt status?) to find out the issue...

@nixpanic
Copy link
Member Author

/retest ci/centos/mini-e2e-helm/k8s-1.20

@nixpanic
Copy link
Member Author

/retest ci/centos/mini-e2e/k8s-1.20

@nixpanic
Copy link
Member Author

Both ci/centos/mini-e2e/k8s-1.20 and ci/centos/mini-e2e-helm/k8s-1.20 failed due to #1969

@nixpanic
Copy link
Member Author

/retest ci/centos/mini-e2e-helm/k8s-1.20

@nixpanic
Copy link
Member Author

/retest ci/centos/mini-e2e/k8s-1.20

@nixpanic
Copy link
Member Author

/retest ci/centos/mini-e2e-helm/k8s-1.20

@nixpanic
Copy link
Member Author

/retest ci/centos/mini-e2e/k8s-1.20

1 similar comment
@nixpanic
Copy link
Member Author

/retest ci/centos/mini-e2e/k8s-1.20

@nixpanic
Copy link
Member Author

/retest ci/centos/mini-e2e-helm/k8s-1.20

@nixpanic
Copy link
Member Author

/retest ci/centos/mini-e2e/k8s-1.20

@Madhu-1
Copy link
Collaborator

Madhu-1 commented May 25, 2021

@Mergifyio rebase

The CephFS NodeServer should handle the CephFS specific requests. This
is not something that the NodeServer for RBD should handle.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
The NodeGetVolumeStats procedure can now be used to fetch the capacity
of the RBD block-device. By default this is a thin-provisioned device,
which means that the capacity is not reserved in the Ceph cluster. This
makes it possible to over-provision the cluster.

In order to detect the amount of storage used by the RBD block-device
(when thin-provisioned), it is required to connect to the Ceph cluster.
Unfortunately, the NodeGetVolumeStats CSI procedure does not provide
enough parameters to connect to the Ceph cluster and fetch more details
about the RBD image.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
Test if metrics are available at all. The actual values are a little
difficult to validate.

BlockMode volumes support metrics since Kubernetes 1.22.

See-also: kubernetes/kubernetes#97972
Signed-off-by: Niels de Vos <ndevos@redhat.com>
@mergify
Copy link
Contributor

mergify bot commented May 25, 2021

Command rebase: success

Branch has been successfully rebased

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component/rbd Issues related to RBD dependency/k8s depends on Kubernetes features enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants