Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Alias mode to export a share to other namespaces #47

Closed
wants to merge 1 commit into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -119,6 +119,8 @@ spec:
path: /var/nfs
```

**Alias mode:** use the provisioner in this mode to share the same existing NFS claim to multiple namespaces, without propagating manually the server/path in each namespace's claim. For example, first create a `data-original` claim as normal, through any provisioner such as `example.com/efs-aws` or the `fuseim.pri/ifs` example below. In the same namespace of your choice, run a new NFS client provisioner that uses the claim. Set NFS_SERVER to the magic value of `--alias`. Give the new deployment a clearer name, `nfs-alias-provisioner`, and set PROVISIONER_NAME to `foo.com/nfs-alias-provisioner`. Then create a StorageClass `nfs-alias` with its provisioner set to `foo.com/nfs-alias-provisioner`. Now, every new `nfs-alias` claim you create in any namespace will have the same `server:path` as the `data-original` volume.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the fuseim.pri/ifs soon will be changed to k8s-sigs.io/nfs-subdir-external-provisioner (#37)


You may also want to change the PROVISIONER_NAME above from `fuseim.pri/ifs` to something more descriptive like `nfs-storage`, but if you do remember to also change the PROVISIONER_NAME in the storage class definition below.

To disable leader election, define an env variable named ENABLE_LEADER_ELECTION and set its value to false.
Expand Down
100 changes: 71 additions & 29 deletions cmd/nfs-subdir-external-provisioner/provisioner.go
Original file line number Diff line number Diff line change
Expand Up @@ -36,17 +36,20 @@ import (
"k8s.io/client-go/rest"
"k8s.io/client-go/tools/clientcmd"
"k8s.io/kubernetes/pkg/apis/core/v1/helper"
mnt "k8s.io/mount-utils"
"sigs.k8s.io/sig-storage-lib-external-provisioner/v6/controller"
)

const (
provisionerNameKey = "PROVISIONER_NAME"
magicAliasHostname = "--alias"
)

type nfsProvisioner struct {
client kubernetes.Interface
server string
path string
alias bool
}

type pvcMetadata struct {
Expand Down Expand Up @@ -79,40 +82,17 @@ const (
var _ controller.Provisioner = &nfsProvisioner{}

func (p *nfsProvisioner) Provision(ctx context.Context, options controller.ProvisionOptions) (*v1.PersistentVolume, controller.ProvisioningState, error) {
var path string
var err error

if options.PVC.Spec.Selector != nil {
return nil, controller.ProvisioningFinished, fmt.Errorf("claim Selector is not supported")
}
glog.V(4).Infof("nfs provisioner: VolumeOptions %v", options)

pvcNamespace := options.PVC.Namespace
pvcName := options.PVC.Name

pvName := strings.Join([]string{pvcNamespace, pvcName, options.PVName}, "-")

metadata := &pvcMetadata{
data: map[string]string{
"name": pvcName,
"namespace": pvcNamespace,
},
labels: options.PVC.Labels,
annotations: options.PVC.Annotations,
}

fullPath := filepath.Join(mountPath, pvName)
path := filepath.Join(p.path, pvName)

pathPattern, exists := options.StorageClass.Parameters["pathPattern"]
if exists {
customPath := metadata.stringParser(pathPattern)
path = filepath.Join(p.path, customPath)
fullPath = filepath.Join(mountPath, customPath)
}

glog.V(4).Infof("creating path %s", fullPath)
if err := os.MkdirAll(fullPath, 0777); err != nil {
return nil, controller.ProvisioningFinished, errors.New("unable to create directory to provision new pv: " + err.Error())
if path, err = p.getOrMakeDir(options); err != nil {
return nil, controller.ProvisioningFinished, err
}
os.Chmod(fullPath, 0777)

pv := &v1.PersistentVolume{
ObjectMeta: metav1.ObjectMeta{
Expand All @@ -129,14 +109,51 @@ func (p *nfsProvisioner) Provision(ctx context.Context, options controller.Provi
NFS: &v1.NFSVolumeSource{
Server: p.server,
Path: path,
ReadOnly: false,
ReadOnly: false, // Pass ReadOnly through if in alias mode?
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

afaik it have no real effect, you can try to set it to true and it will be writable anyway.

},
},
},
}
return pv, controller.ProvisioningFinished, nil
}

// If in alias mode, forward the server:path details from the PVC we mounted.
// If not, create a new directory.
func (p *nfsProvisioner) getOrMakeDir(options controller.ProvisionOptions) (string, error) {
if p.alias {
return p.path, nil
}

pvcNamespace := options.PVC.Namespace
pvcName := options.PVC.Name

pvName := strings.Join([]string{pvcNamespace, pvcName, options.PVName}, "-")
fullPath := filepath.Join(mountPath, pvName)
path := filepath.Join(p.path, pvName)

pathPattern, exists := options.StorageClass.Parameters["pathPattern"]
if exists {
metadata := &pvcMetadata{
data: map[string]string{
"name": pvcName,
"namespace": pvcNamespace,
},
labels: options.PVC.Labels,
annotations: options.PVC.Annotations,
}
customPath := metadata.stringParser(pathPattern)
path = filepath.Join(p.path, customPath)
fullPath = filepath.Join(mountPath, customPath)
}

glog.V(4).Infof("creating path %s", fullPath)
if err := os.MkdirAll(fullPath, 0777); err == nil {
return "", errors.New("unable to create directory to provision new pv: " + err.Error())
}
os.Chmod(fullPath, 0777)
return path, nil
}

func (p *nfsProvisioner) Delete(ctx context.Context, volume *v1.PersistentVolume) error {
path := volume.Spec.PersistentVolumeSource.NFS.Path
relativePath := strings.Replace(path, p.path, "", 1)
Expand Down Expand Up @@ -200,6 +217,17 @@ func (p *nfsProvisioner) getClassForVolume(ctx context.Context, pv *v1.Persisten
return class, nil
}

// Return the server and path parts for the given NFS mount
func getDetailsForMountPoint(m string) (server, path string, err error) {
if path, _, err = mnt.GetDeviceNameFromMount(mnt.New(""), m); err == nil {
if parts := strings.Split(path, ":"); len(parts) == 2 {
return parts[0], parts[1], err
}
err = errors.New("Can't parse server:path from device string: " + path)
}
return
}

func main() {
flag.Parse()
flag.Set("logtostderr", "true")
Expand Down Expand Up @@ -256,10 +284,24 @@ func main() {
}
}

// If $NFS_SERVER=="--alias", just pass through the server/path that is
// mounted in the provisioner's pod under /persistentvolumes and never
// make a new directory for each volume we are asked to provision.
var alias bool
if server == magicAliasHostname {
Copy link
Contributor

@yonatankahana yonatankahana Feb 4, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

imo its not very intuitive way and --alias sounds like command line option.

what do you think about another environment variable?
or StorageClass parameter where the same provisioner can be used both with alias and without?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was something I cooked up quickly last year as a proof of concept. I agree that supporting both modes with a class parameter is better, but if we use that to reference a volume claim rather than a server:path pair, resolving that reference might involve another round trip to the API server with potential failure modes. I'll rework the code and try both kinds of class parameters.

// Figure just once and store the server/path pair
if server, path, err = getDetailsForMountPoint(mountPath); err != nil {
glog.Fatalf("Error getting server details for %v: %v", mountPath, err)
}
glog.Infof("Aliasing all new volumes to %v::%v", server, path)
alias = true
}

clientNFSProvisioner := &nfsProvisioner{
client: clientset,
server: server,
path: path,
alias: alias,
}
// Start the provision controller which will dynamically provision efs NFS
// PVs
Expand Down
9 changes: 1 addition & 8 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -4,18 +4,11 @@ go 1.14

require (
github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b
github.com/miekg/dns v1.1.29 // indirect
github.com/onsi/ginkgo v1.12.0 // indirect
github.com/onsi/gomega v1.9.0 // indirect
github.com/prometheus/client_golang v1.5.1 // indirect
golang.org/x/oauth2 v0.0.0-20200107190931-bf48bf16ab8d // indirect
golang.org/x/time v0.0.0-20191024005414-555d28b269f0 // indirect
k8s.io/api v0.18.0
k8s.io/apimachinery v0.18.0
k8s.io/client-go v0.18.0
k8s.io/klog v1.0.0 // indirect
k8s.io/kubernetes v1.18.0
k8s.io/utils v0.0.0-20200327001022-6496210b90e8 // indirect
k8s.io/mount-utils v0.20.2
sigs.k8s.io/sig-storage-lib-external-provisioner/v6 v6.0.0
)

Expand Down
Loading