[YUNIKORN-3016] Cleanup cluster resources after persistent_volume e2e test failed

[kaichiachen]

What is this PR for?

In the e2e test persistent_volume_test.go, the test case Verify_dynamic_binding_with_nfs_server may leave behind cluster resources (e.g., StorageClasses, ClusterRoles, Deployments, etc.) when the test fails. These resources could cause subsequent tests using the same namespace to encounter unexpected errors.

What type of PR is it?

• - Bug Fix
• - Improvement
• - Feature
• - Documentation
• - Hot Fix
• - Refactoring

Todos

• - Task

What is the Jira issue?

https://issues.apache.org/jira/browse/YUNIKORN-3016

How should this be tested?

1. Inject an error to the end of testcase "Verify_dynamic_binding_with_nfs_server" in persistent_volume_test
2. Run e2e test, verify resources are all cleaned up

#> kubectl get clusterrolebindings nfs-cluster-role-binding -n dev-hkp8i
Error from server (NotFound): clusterrolebindings.rbac.authorization.k8s.io "nfs-cluster-role-binding" not found
#> kubectl get ClusterRole nfs-cluster-role -n dev-hkp8i          
Error from server (NotFound): clusterroles.rbac.authorization.k8s.io "nfs-cluster-role" not found
#> kubectl get storageclasses nfs-sc -n dev-hkp8i                  
Error from server (NotFound): storageclasses.storage.k8s.io "nfs-sc" not found
#> kubectl get serviceaccount nfs-service-account -n dev-hkp8i          
Error from server (NotFound): serviceaccounts "nfs-service-account" not found
#> kubectl get deployment nfs-provisioner -n dev-hkp8i             
Error from server (NotFound): deployments.apps "nfs-provisioner" not found

Screenshots (if appropriate)

Questions:

• - The licenses files need update.
• - There is breaking changes for older versions.
• - It needs documentation.

[ryankert01]

Thank you for you patch! Having a simple question.

[ryankert01]

Why assert only when k8serror is found?

[kaichiachen]

Thanks Ryan for helping me review! k8serrors.IsNotFound returns true if the resource is not found
At line 147/148
147 createNfsRbac(saName, crName, crbName)
148 createNfsProvisioner(saName, serverName, scName)
At the beginning of the test, resources are created sequentially while some of them might be created successfully and some of not. We don't want to show resource not found error when cleaning up resources.