Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] Operator is unable to create RayClusters on OpenShift due to RBAC #630

Closed
2 tasks done
AlessandroPomponio opened this issue Oct 12, 2022 · 1 comment · Fixed by #631
Closed
2 tasks done
Labels
bug Something isn't working

Comments

@AlessandroPomponio
Copy link
Contributor

Search before asking

  • I searched the issues and found no similar issues.

KubeRay Component

ray-operator

What happened + What you expected to happen

The operator is unable to create the example RayCluster on OpenShift due to missing RBAC permissions, with the error being:

2022-10-11T14:45:04.448Z	ERROR	controllers.RayCluster	Pod Service Account create error!	{"Pod.ServiceAccount.Error": "serviceaccounts \"ray-disorch\" is forbidden: cannot set blockOwnerDeletion if an ownerReference refers to a resource you can't set finalizers on: , <nil>", "error": "serviceaccounts \"ray-disorch\" is forbidden: cannot set blockOwnerDeletion if an ownerReference refers to a resource you can't set finalizers on: , <nil>"}
github.com/ray-project/kuberay/ray-operator/controllers/ray.(*RayClusterReconciler).rayClusterReconcile
	/workspace/controllers/ray/raycluster_controller.go:172
github.com/ray-project/kuberay/ray-operator/controllers/ray.(*RayClusterReconciler).Reconcile
	/workspace/controllers/ray/raycluster_controller.go:95
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile
	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.11.1/pkg/internal/controller/controller.go:114
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.11.1/pkg/internal/controller/controller.go:311
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.11.1/pkg/internal/controller/controller.go:266
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.11.1/pkg/internal/controller/controller.go:227
2022-10-11T14:45:05.054Z	ERROR	controller.raycluster-controller	Reconciler error	{"reconciler group": "ray.io", "reconciler kind": "RayCluster", "name": "ray-disorch", "namespace": "discovery-prod", "error": "serviceaccounts \"ray-disorch\" is forbidden: cannot set blockOwnerDeletion if an ownerReference refers to a resource you can't set finalizers on: , <nil>"}
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.11.1/pkg/internal/controller/controller.go:266
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.11.1/pkg/internal/controller/controller.go:227

Expected behavior: Creating the example clusters provided in the documentation should work without issues.

Additional details:

$ oc version
Client Version: 4.11.6
Kustomize Version: v4.5.4
Server Version: 4.10.34
Kubernetes Version: v1.23.5+8471591

Reproduction script

kubectl apply -f https://ray-project.github.io/kuberay/components/config/samples/ray-cluster.autoscaler.yaml

Anything else

The problem occurs every time

Are you willing to submit a PR?

  • Yes I am willing to submit a PR!
@AlessandroPomponio AlessandroPomponio added the bug Something isn't working label Oct 12, 2022
@AlessandroPomponio
Copy link
Contributor Author

The issue is due to a typo in the kubebuilder instructions, as reported here:
https://sdk.operatorframework.io/docs/faqs/#after-deploying-my-operator-why-do-i-see-errors-like-is-forbidden-cannot-set-blockownerdeletion-if-an-ownerreference-refers-to-a-resource-you-cant-set-finalizers-on-

In a few instances, it is spelled finalizer instead of finalizers

DmitriGekhtman pushed a commit that referenced this issue Oct 13, 2022
These changes address issue #630

When trying to create one of the example RayCluster on OpenShift, it will fail with the operator reporting "cannot set blockOwnerDeletion if an ownerReference refers to a resource you can’t set finalizers on:".

As per https://sdk.operatorframework.io/docs/faqs/#after-deploying-my-operator-why-do-i-see-errors-like-is-forbidden-cannot-set-blockownerdeletion-if-an-ownerreference-refers-to-a-resource-you-cant-set-finalizers-on-, we were able to fix the issue by applying the new role.yaml that is created running make manifests after applying the changes in this PR.
lowang-bh pushed a commit to lowang-bh/kuberay that referenced this issue Sep 24, 2023
These changes address issue ray-project#630

When trying to create one of the example RayCluster on OpenShift, it will fail with the operator reporting "cannot set blockOwnerDeletion if an ownerReference refers to a resource you can’t set finalizers on:".

As per https://sdk.operatorframework.io/docs/faqs/#after-deploying-my-operator-why-do-i-see-errors-like-is-forbidden-cannot-set-blockownerdeletion-if-an-ownerreference-refers-to-a-resource-you-cant-set-finalizers-on-, we were able to fix the issue by applying the new role.yaml that is created running make manifests after applying the changes in this PR.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant