-
Notifications
You must be signed in to change notification settings - Fork 1.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add csi snapshot data mover doc #6637
Add csi snapshot data mover doc #6637
Conversation
bb4f222
to
2b71d97
Compare
5eee0fc
to
704f0f5
Compare
Codecov Report
@@ Coverage Diff @@
## main #6637 +/- ##
==========================================
+ Coverage 60.36% 60.44% +0.07%
==========================================
Files 239 242 +3
Lines 25499 26023 +524
==========================================
+ Hits 15392 15729 +337
- Misses 9036 9190 +154
- Partials 1071 1104 +33 |
bfa944b
to
8c08c2d
Compare
8c08c2d
to
2250378
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This looks good to me after the first glance.
2250378
to
6d37203
Compare
6d37203
to
dd379e2
Compare
dd379e2
to
55301c0
Compare
165d2ff
to
c5eadc7
Compare
@sseago @shubham-pampattiwar |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Overall looks good ! Added a few comments/nits here and there. Thank you @Lyndon-Li !!
c5eadc7
to
6f8167d
Compare
I just started going through the doc but one thing that is missing is changes to "Backup" spec. The fields "snapshotMoveData" and "datamover" need to be added at https://velero.io/docs/main/api-types/backup/. |
6f8167d
to
801e40c
Compare
542981e
to
5e3bd18
Compare
Done the changes. Added them for both backup and schedule api-types. |
- CSI plugin first takes a CSI snapshot to the PVC by creating the `VolumeSnapshot` and `VolumeSnapshotContent`. | ||
- CSI plugin checks if a data movement is required, if so it creates a `DataUpload` CR and then returns to Velero backup. | ||
- Velero now is able to back up other resources, including other PVC objects. | ||
- Velero backup periodically queries the data movement status from CSI plugin, the period is configurable through the Velero server parameter `--item-operation-sync-frequency`, by default it is 10s. On the call, CSI plugin turns to check the phase of the `DataUpload` CRs. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
backup -> backup controller
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done, thanks
- CSI plugin expects the same data mover for the backup to handle the `DataDownload` CR. If no data mover was configured for the backup, Velero built-in data mover will handle it. | ||
- If the `DataDownload` CR does not reach to the terminal state with in the given time, the `DataDownload` CR will be cancelled. You can set the timeout value per backup through the same `--item-operation-timeout` parameter. | ||
|
||
- Velero built-in data mover creates a volume with the same specification of the source volume. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Shall we stress that the volume creation is done via dynamic provision, i.e. it only creates the PVC and it has dependency on storageclass on the target cluster?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We have mentioned how to configure the storage class on the target in section Configure storage class on target cluster
.
I have rephrased the sentence in the same section as below:
On the other hand, Velero built-in data movement creates a PVC with the same specification as it in the source cluster and expects the volume to be provisioned similarly. For example, the same storage class should be working in the target cluster
|
||
### Cancellation | ||
|
||
At present, Velero backup and restore doesn't support end to end cancellation that is launched by users. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
But a user can patch a dataupload/datadownload CR's spec to cancel the action. Shall we also mention that in the doc?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We don't intend to expose this approach to users officially -- when there are many datauploads/downloads in a backup/restore, users may not able to correctly take care of everyone of the datauploads/downloads, and an unexpected result may happen.
For example, a dataupload/datadownload is matched to a wrong backup/restore, as a result, the backup/restore will get to a ParitialFailed status unexpectedly.
For Velero side configurations mentioned above, the installation and configuration of node-agent may not be required. | ||
|
||
|
||
## To back up |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For both backup and restore, shall we mention how to monitor the progress of the data movement of each volume via velero backup/restore describe --details
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We have suggested the other way to watch the progress:
You can see the phase changes as well as the data upload progress by watching the
DataUpload CRs
This is the most direct way, while velero backup/restore describe --details
needs to retrieve data from backup store and is not timely (we have discussed this problem and the possible fix during the Async Operation PR, but not fixed it yet).
cba029b
5e3bd18
to
cba029b
Compare
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
cba029b
to
1adf842
Compare
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
Add CSI snapshot data movement doc