Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Out-of-tree CSI Volume Plugins #178

Closed
saad-ali opened this issue Jan 24, 2017 · 49 comments · Fixed by kubernetes/kubernetes#71020
Closed

Out-of-tree CSI Volume Plugins #178

saad-ali opened this issue Jan 24, 2017 · 49 comments · Fixed by kubernetes/kubernetes#71020
Assignees
Labels
kind/feature Categorizes issue or PR as related to a new feature. sig/storage Categorizes an issue or PR as relevant to SIG Storage. stage/stable Denotes an issue tracking an enhancement targeted for Stable/GA status tracked/no Denotes an enhancement issue is NOT actively being tracked by the Release Team
Milestone

Comments

@saad-ali
Copy link
Member

saad-ali commented Jan 24, 2017

  • One-line feature description (can be used as a release note): Add support for Out-of-Tree CSI Volume Plugins in Kubernetes
  • Primary contact (assignee): @saad-ali
  • Responsible SIGs: storage
  • Design proposal link (community repo): CSI Volume Plugins in Kubernetes Design Doc community#1258
  • Reviewer(s) - (for LGTM) recommend having 2+ reviewers (at least one from code-area OWNERS file) agreed to review. Reviewers from multiple companies preferred:
    @childsb
    @jsafrane
  • Approver (likely from SIG/area to which feature belongs):
    @thockin
  • Feature target (which target equals to which milestone):
    • Alpha release target (x.y): 1.9
    • Beta release target (x.y): 1.10
    • Stable release target (x.y): 1.13

Background

Kubernetes volume plugins are currently all "in-tree" meaning that their source code is included in the main Kubernetes repo. All volume plugins are compiled and ship along with kubernetes binaries.

The drawback to this approach is that it requires third-party storage vendors wanting to support Kubernetes to commit code to the Kubernetes repo, and thus be locked into Kubernetes release schedules. It requires them to make their source code public/open-source.

While the Flex volume plugin already provides a mechanism for Plugin developers to experiment with out-of-tree plugins, it provides no guarantees of backwards compatibility (since it is alpha), and is completely exec based (driver installation requires ability to deploy files to specific locations on node and master machines).

This feature aims to create a new (or adopt an existing) API for Volume Plugins in Kubernetes that:

  • Will allow volume plugins to be developed out-of-tree.
  • Not require building volume plugins (or their dependencies) into Kubernetes binaries.
  • Not requiring direct machine access to deploy new volume plugins (drivers).

CC @kubernetes/sig-storage-feature-requests @kubernetes/sig-storage-proposals

@saad-ali saad-ali added the sig/storage Categorizes an issue or PR as relevant to SIG Storage. label Jan 24, 2017
@saad-ali saad-ali added this to the next-milestone milestone Jan 24, 2017
@saad-ali
Copy link
Member Author

We aim to have a design doc for this by 1.6.

@calebamiles calebamiles modified the milestones: v1.6, next-milestone Feb 7, 2017
@idvoretskyi
Copy link
Member

@saad-ali any update on this feature? Docs and release notes are required (please, provide them to the features spreadsheet.

@saad-ali saad-ali removed this from the v1.6 milestone Mar 13, 2017
@saad-ali
Copy link
Member Author

This does not need to be tracked for 1.6. It's design only. No bits shipping as part of 1.6. Removing 1.6 label.

@idvoretskyi idvoretskyi added this to the next-milestone milestone Mar 14, 2017
@idvoretskyi
Copy link
Member

@saad-ali thank you for clarifying.

@jeffvance
Copy link
Contributor

jeffvance commented Mar 21, 2017

@saad-ali is the idea behind the 3rd bullet Not requiring direct machine access to deploy new volume plugins (drivers) to be able to pull plugin dependencies (eg. client libraries) from a defined registry or repo and push them to each node?

@krmayankk
Copy link

@saad-ali also its not clear, how is this different from out -of tree dynamic provisioners ?

@saad-ali saad-ali changed the title Out-of-tree Volume Plugins (CVI) Out-of-tree Volume Plugins (CSI) May 18, 2017
@lpabon
Copy link
Contributor

lpabon commented Jun 14, 2017

@saad-ali How does this compare with your CSI work on the CNCF ?

@saad-ali
Copy link
Member Author

@saad-ali is the idea behind the 3rd bullet Not requiring direct machine access to deploy new volume plugins (drivers) to be able to pull plugin dependencies (eg. client libraries) from a defined registry or repo and push them to each node?

Yep!

@saad-ali also its not clear, how is this different from out -of tree dynamic provisioners ?

This feature (CSI) will enable completely out-of-tree volume plugins. Out-of-tree dynamic provisioners exist today as an alpha mechinism to enable volume provisioning (one functionality of volume plugins) to 1) be implmented out-of-tree, and 2) be decoupled from the rest of the volume plugin (potentially enabling multiple provisioners per volume type). Once Kubernetes supports CSI volume plugins, the only remaining benefit of Out-of-tree dynamic provisioners will be 2 (decoupled from the rest of the volume plugin, potentially enabling multiple provisioners per volume type). We can decide at that point if it makes sense to continue to support them or deprecate them.

@saad-ali How does this compare with your CSI work on the CNCF ?

CSI is the proposed protocol spec. The intention (or at least one of the intentions) of this feature is to implement that interface in Kubernetes.

@luxas
Copy link
Member

luxas commented Sep 1, 2017

@saad-ali Are you targeting alpha for v1.9? Is that information up-to-date?
Thanks!

@saad-ali saad-ali modified the milestones: next-milestone, 1.9 Oct 20, 2017
@saad-ali saad-ali changed the title Out-of-tree Volume Plugins (CSI) Out-of-tree CSI Volume Plugins Oct 23, 2017
@saad-ali
Copy link
Member Author

@saad-ali Are you targeting alpha for v1.9? Is that information up-to-date?

Yes! All info is up to date.

@saad-ali
Copy link
Member Author

This feature (CSI) will enable completely out-of-tree volume plugins. Out-of-tree dynamic provisioners exist today as an alpha mechinism to enable volume provisioning (one functionality of volume plugins) to 1) be implmented out-of-tree, and 2) be decoupled from the rest of the volume plugin (potentially enabling multiple provisioners per volume type). Once Kubernetes supports CSI volume plugins, the only remaining benefit of Out-of-tree dynamic provisioners will be 2 (decoupled from the rest of the volume plugin, potentially enabling multiple provisioners per volume type). We can decide at that point if it makes sense to continue to support them or deprecate them.

An update on this: we've incorporated external-provisioners in to the CSI design!

@govint
Copy link

govint commented Oct 24, 2017

@saad-ali can you point to the CSI design that has the external provisioners incorporated. Is it here kubernetes/community#1258?

@luxas
Copy link
Member

luxas commented Oct 24, 2017

@saad-ali Will this work be in v1.9? I assume as alpha as the initial step.

@kokhang
Copy link

kokhang commented Nov 9, 2017

@luxas @saad-ali does "alpha" mean that csi would need to be enabled via a feature gate like all the other alpha features?

Also the CSI plugin depends on other features, which are currently in alpha, like mount-propagation and perhaps containerized-mount? Will these be promoted to beta? or does this mean that in order to consume the CSI plugin, these features have to be enabled via the feature gates?

@AishSundar
Copy link

@saad-ali whats left of this work to go to GA in 1.13? Are kubernetes/kubernetes#69690 and kubernetes/kubernetes#69688 the only 2 pending PRs for this feature?

@jimangel
Copy link
Member

@saad-ali when this goes GA will there need to be any doc changes (1.13)?

@jimangel
Copy link
Member

jimangel commented Nov 5, 2018

@saad-ali will there be any updates to the docs necessary for 1.13? The deadline for placeholder PRs for the 1.13 release is November 8. So it's important to make a docs PR as soon as possible if needed. Thanks! cc @idvoretskyi @AishSundar @tfogo

@claurence
Copy link

Hi @saad-ali I'm an enhancements shadow checking in on how this issue is tracking. Code slush is on 11/9 and code freeze is coming up on 11/15 do you have a status update on the likelihood that this will make the the code freeze date?

@msau42
Copy link
Member

msau42 commented Nov 6, 2018

Tasks for 1.13:

We are still on track for 1.13, although timeline will be very tight. The longest pull item is moving the CSI spec to 1.0. All other tasks are close to being merged.

@AishSundar
Copy link

@saad-ali @msau42 I understand you are working actively on getting this to GA. I see the 2 open PRs are close, but is there a tracking issue for the 1.0 spec?

Can either of you attend Release burndown meeting (Zoom: https://zoom.us/j/611312756) this Wedsnesday (11/14)? We would like the discuss the latest update Go/No-GO for 1.13. Thanks

@saad-ali
Copy link
Member Author

CSI 1.0.0-rc1 Spec was cut on Monday (https://github.com/container-storage-interface/spec/releases/tag/v1.0.0-rc1). We are working to pick it up in Kubernetes (kubernetes/kubernetes#71020). @msau42 will attend the burn down.

@kacole2
Copy link

kacole2 commented Nov 16, 2018

/reopen

@k8s-ci-robot
Copy link
Contributor

@kacole2: Reopened this issue.

In response to this:

/reopen

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@k8s-ci-robot k8s-ci-robot reopened this Nov 16, 2018
@spiffxp
Copy link
Member

spiffxp commented Jan 10, 2019

@saad-ali @kacole2 What remains to be done here? I think this shipped as GA in 1.13 and can be closed

@msau42
Copy link
Member

msau42 commented Jan 10, 2019

We have some post GA work, like updating the in-tree controllers to use the new v1 objects.

@kacole2
Copy link

kacole2 commented Jan 10, 2019

@msau42 but is that "enhancement" related? or is that more updating?

@msau42
Copy link
Member

msau42 commented Jan 10, 2019

It's cleanup/refactoring work that should not have any user-visible effects. So I guess this can be closed.

@claurence
Copy link

@msau42 @saad-ali Hello - I'm the enhancements lead for 1.14 - it sounds like there is no work to be tracked for the 1.14 release so I can remove it from being tracked but let me know if that is an incorrect interpretation. Thanks

@msau42
Copy link
Member

msau42 commented Jan 15, 2019

We can close this and open up issues for remaining cleanup work. There are already individual enhancement issues opened for csi alpha features
/close

@k8s-ci-robot
Copy link
Contributor

@msau42: Closing this issue.

In response to this:

We can close this and open up issues for remaining cleanup work. There are already individual enhancement issues opened for csi alpha features
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@kacole2 kacole2 added tracked/no Denotes an enhancement issue is NOT actively being tracked by the Release Team and removed tracked/yes Denotes an enhancement issue is actively being tracked by the Release Team labels Jul 15, 2019
ingvagabund pushed a commit to ingvagabund/enhancements that referenced this issue Apr 2, 2020
user-workload-monitoring: be more accurate about alerts & silences enpoints
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/feature Categorizes issue or PR as related to a new feature. sig/storage Categorizes an issue or PR as relevant to SIG Storage. stage/stable Denotes an issue tracking an enhancement targeted for Stable/GA status tracked/no Denotes an enhancement issue is NOT actively being tracked by the Release Team
Projects
None yet