diff --git a/docs-2.0/nebula-operator/1.introduction-to-nebula-operator.md b/docs-2.0/nebula-operator/1.introduction-to-nebula-operator.md index b8be718103a..a2ee228ca11 100644 --- a/docs-2.0/nebula-operator/1.introduction-to-nebula-operator.md +++ b/docs-2.0/nebula-operator/1.introduction-to-nebula-operator.md @@ -21,6 +21,8 @@ The following features are already available in NebulaGraph Operator: - **Scale clusters**: NebulaGraph Operator calls NebulaGraph's native scaling interfaces in a control loop to implement the scaling logic. You can simply perform scaling operations with YAML configurations and ensure the stability of data. For more information, see [Scale clusters with Kubectl](3.deploy-nebula-graph-cluster/3.1create-cluster-with-kubectl.md) or [Scale clusters with Helm](3.deploy-nebula-graph-cluster/3.2create-cluster-with-helm.md). - **Cluster Upgrade**: NebulaGraph Operator supports cluster upgrading from version {{operator.upgrade_from}} to version {{operator.upgrade_to}}. + +- **Backup and Recovery**：NebulaGraph supports data backup and recovery. Users can use NebulaGraph Operator to backup the data of the NebulaGraph cluster to storage services that are compatible with the S3 protocol, and can also restore data to the cluster from the storage service. For details, see [Backup and restore using NebulaGraph Operator](10.backup-restore-using-operator.md). - **Self-Healing**: NebulaGraph Operator calls interfaces provided by NebulaGraph clusters to dynamically sense cluster service status. Once an exception is detected, NebulaGraph Operator performs fault tolerance. For more information, see [Self-Healing](5.operator-failover.md). @@ -34,8 +36,7 @@ NebulaGraph Operator does not support the v1.x version of NebulaGraph. NebulaGra | NebulaGraph |NebulaGraph Operator | | ------------------- | ----------- | -| 3.0.0 ~ 3.4.0 |1.4.0| -| 3.0.0 ~ 3.3.x |1.3.0| +| 3.0.0 ~ 3.4.0 |1.3.0, 1.4.0| | 3.0.0 ~ 3.3.x |1.0.0, 1.1.0, 1.2.0| | 2.5.x ~ 2.6.x |0.9.0| | 2.5.x |0.8.0| diff --git a/docs-2.0/nebula-operator/10.backup-restore-using-operator.md b/docs-2.0/nebula-operator/10.backup-restore-using-operator.md new file mode 100644 index 00000000000..9c6d33cdd1d --- /dev/null +++ b/docs-2.0/nebula-operator/10.backup-restore-using-operator.md @@ -0,0 +1,230 @@ +# Backup and restore data using NebulaGraph Operator + +This article introduces how to back up and restore data of the NebulaGraph cluster on Kubernetes. + +!!! enterpriseonly + + This feature is only for the enterprise edition NebulaGraph clusters on Kubernetes. + +## Overview + +[NebulaGraph BR (Enterprise Edition)](../backup-and-restore/nebula-br-ent/1.br-ent-overview.md) is a command line tool for data backup and recovery of NebulaGraph enterprise edition. NebulaGraph Operator is based on the BR tool to achieve data backup and recovery for NebulaGraph clusters on Kubernetes. + +When backing up data, NebulaGraph Operator creates a Job to back up the data in the NebulaGraph cluster to the specified storage service. + +When restoring data, NebulaGraph Operator checks the specified backup NebulaGraph cluster for existence, and whether the access to remote storage is normally based on the information defined in the NebulaRestore resource object. It then creates a new cluster and restores the backup data to the new NebulaGraph cluster. For more information, see [restore flowchart](https://github.com/vesoft-inc/nebula-operator/blob/v{{operator.release}}/doc/user/br_guide.md#restore-nebulagraph-cluster). + + +## Prerequisites + +To backup and restore data using NebulaGraph Operator, the following conditions must be met: + +- Nebula Operator version >= 1.4.0. +- The enterprise edition NebulaGraph cluster deployed on Kubernetes is running. +- In the YAML file used to create the cluster, `spec.enableBR` is set to true. + + ``` + // Partial content of a sample cluster YAML file. + apiVersion: apps.nebula-graph.io/v1alpha1 + kind: NebulaCluster + metadata: + name: nebula + spec: + enableBR: true // Set to true to enable the backup and restore function. + ... + ``` + +- Only storage services that use the S3 protocol (such as AWS S3, Minio, etc.) can be used to back up and restore data. +- Sufficient computing resources are available in the cluster to restore data. + +## Backup + +### Notes + +- NebulaGraph Operator supports full and incremental backups. +- During data backup, DDL and DML statements in the specified graph space will be blocked. We recommend performing the operation during off-peak hours, such as from 2:00 am to 5:00 am. +- The cluster executing incremental backups and the cluster specified for the last backup must be the same, and the (storage bucket) path for the last backup must be the same. +- Ensure that the time between each incremental backup and the last backup is less than a `wal_ttl`. +- Specifying the backup data of a specified graph space is not supported. + +### Full backup + +When backing up data to a storage service compatible with the S3 protocol, you need to create a backup Job, which will back up the full NebulaGraph data to the specified storage location. + +Here is an example of the YAML file for a full backup Job: + +```yaml +apiVersion: batch/v1 +kind: Job +metadata: + name: nebula-full-backup +spec: + parallelism: 1 + ttlSecondsAfterFinished: 60 + template: + spec: + restartPolicy: OnFailure + containers: + - image: vesoft/br-ent:v{{br_ent.release}} + imagePullPolicy: Always + name: backup + command: + - /bin/sh + - -ecx + - exec /usr/local/bin/nebula-br backup full + - --meta $META_ADDRESS:9559 + - --storage s3://$BUCKET + - --s3.access_key $ACCESS_KEY + - --s3.secret_key $SECRET_KEY + - --s3.region $REGION + - --s3.endpoint https://s3.$REGION.amazonaws.com +``` + +### Incremental backup + +Except for the name of the Job and the command specified in `spec.template.spec.containers[0].command`, the YAML file for incremental backup is the same as that for a full backup. Here is an example of the YAML file for incremental backup: + +```yaml +apiVersion: batch/v1 +kind: Job +metadata: + name: nebula-incr-backup +spec: + parallelism: 1 + ttlSecondsAfterFinished: 60 + template: + spec: + restartPolicy: OnFailure + containers: + - image: vesoft/br-ent:v{{br_ent.release}} + imagePullPolicy: Always + name: backup + command: + - /bin/sh + - -ecx + - exec /usr/local/bin/nebula-br backup incr + - --meta $META_ADDRESS:9559 + - --base $BACKUP_NAME + - --storage s3://$BUCKET + - --s3.access_key $ACCESS_KEY + - --s3.secret_key $SECRET_KEY + - --s3.region $REGION + - --s3.endpoint https://s3.$REGION.amazonaws.com +``` + +### Parameter description + +The main parameters are described as follows: + + +| Parameter |Default value | Description | +| ------------- | ---- | ---- | +| `spec.parallelism` |1 |The number of tasks executed in parallel. | +| `spec.ttlSecondsAfterFinished` | 60 | The time to keep task information after the task is completed. | +| `spec.template.spec.containers[0].image` | `vesoft/br-ent:{{br_ent.release}}`|The image address of the NebulaGraph BR Enterprise Edition tool. | +| `spec.template.spec.containers[0].command`| - | The command for backing up data to the storage service compatible with the S3 protocol.
For descriptions of the options in the command, see [Parametr description](../backup-and-restore/nebula-br-ent/3.backup-data.md#_12). | + + +For more settings of the Job, see [Kubernetes Jobs](https://kubernetes.io/docs/concepts/workloads/controllers/job/). + +After the YAML file for the backup Job is set, run the following command to start the backup Job: + + + +```bash +kubectl apply -f .yaml +``` + +When the data backup succeeds, a backup file is generated in the specified storage location. For example, the backup file name is `BACKUP_2023_02_12_10_04_16`. + + +## Restore + +### Notes + +- After the data recovery is successful, a new cluster will be created, and the old cluster will not be deleted. Users can decide whether to delete the old cluster themselves. +- There will be a period of service unavailability during the data recovery process, so it is recommended to perform the operation during a low period of business activity. + + + +### Process + +When restoring data from a compatible S3 protocol service, you need to create a Secret to store the credentials for accessing the compatible S3 protocol service. Then create a resource object (NebulaRestore) for restoring the data, which will instruct the Operator to create a new NebulaGraph cluster based on the information defined in this resource object and restore the backup data to the newly created cluster. + +Here is an example YAML for restoring data based on the backup file `BACKUP_2023_02_12_10_04_16`: + +```yaml +apiVersion: v1 +kind: Secret +metadata: + name: aws-s3-secret +type: Opaque +data: + access-key: QVNJQVE0WFlxxx + secret-key: ZFJ6OEdNcDdxenMwVGxxx +--- +apiVersion: apps.nebula-graph.io/v1alpha1 +kind: NebulaRestore +metadata: + name: restore1 +spec: + br: + clusterName: nebula + backupName: "BACKUP_2023_02_12_10_04_16" + concurrency: 5 + s3: + region: "us-west-2" + bucket: "nebula-br-test" + endpoint: "https://s3.us-west-2.amazonaws.com" + secretName: "aws-s3-secret" +``` + +### Parameter Description + +- Secret + + |Parameter|Default Value|Description| + |:---|:---|:---| + |`metadata.name`|-|The name of the Secret.| + |`type`|`Opaque`|The type of the Secret. See [Types of Secret](https://kubernetes.io/docs/concepts/configuration/secret/#secret-types) for more information.| + |`data.access-key`|-|The AccessKey for accessing the S3 protocol-compatible storage service.| + |`data.secret-key`|-|The SecretKey for accessing the S3 protocol-compatible storage service.| + +- NebulaRestore + + |Parameter|Default Value|Description| + |:---|:---|:---| + |`metadata.name`|-|The name of the resource object NebulaRestore.| + |`spec.br.clusterName`|-|The name of the backup cluster.| + |`spec.br.backupName`|-|The name of the backup file. Restore data based on this backup file.| + |`spec.br.concurrency`|`5`|The number of concurrent downloads when restoring data. The default value is `5`.| + |`spec.br.s3.region`|-| The geographical region where the S3 storage bucket is located.| + |`spec.br.s3.bucket`|-|The path of the S3 storage bucket where backup data is stored.| + |`spec.br.s3.endpoint`|-|The access address of the S3 storage bucket.| + |`spec.br.s3.secretName`|-|The name of the Secret that is used to access the S3 storage bucket.| + +After setting up the YAML file for restoring the data, run the following command to start the restore job: + +```bash +kubectl apply -f .yaml +``` + +Run the following command to check the status of the NebulaRestore object. + +```bash +kubectl get rt -w + diff --git a/mkdocs.yml b/mkdocs.yml index e1004620cc6..e6e5daa6d8b 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -81,16 +81,17 @@ plugins: - nebula-cloud.md # When publishing a version of a document that includes Enterprise Edition, annotation the following page # ent.begin - - 3.ngql-guide/6.functions-and-expressions/17.ES-function.md - - 4.deployment-and-installation/deploy-license.md - - 5.configurations-and-logs/2.log-management/audit-log.md - - 7.data-security/1.authentication/4.ldap.md - - nebula-operator/8.custom-cluster-configurations/8.3.balance-data-when-scaling-storage.md - - synchronization-and-migration/replication-between-clusters.md - - 20.appendix/release-notes/nebula-ent-release-note.md - - nebula-dashboard-ent/4.cluster-operator/operator/scale.md - - backup-and-restore/nebula-br-ent/* - - 6.monitor-and-metrics/3.bbox/* + # - 3.ngql-guide/6.functions-and-expressions/17.ES-function.md + # - 4.deployment-and-installation/deploy-license.md + # - 5.configurations-and-logs/2.log-management/audit-log.md + # - 7.data-security/1.authentication/4.ldap.md + # - nebula-operator/8.custom-cluster-configurations/8.3.balance-data-when-scaling-storage.md + # - nebula-operator/10.backup-restore-using-operator.md + # - synchronization-and-migration/replication-between-clusters.md + # - 20.appendix/release-notes/nebula-ent-release-note.md + # - nebula-dashboard-ent/4.cluster-operator/operator/scale.md + # - backup-and-restore/nebula-br-ent/* + # - 6.monitor-and-metrics/3.bbox/* # ent.end # comm.begin @@ -694,22 +695,23 @@ nav: - Export data from NebulaGraph: nebula-exchange/use-exchange/ex-ug-export-from-nebula.md - Exchange FAQ: nebula-exchange/ex-ug-FAQ.md -# - NebulaGraph Operator: -# - What is NebulaGraph Operator: nebula-operator/1.introduction-to-nebula-operator.md -# - Overview of using NebulaGraph Operator: nebula-operator/6.get-started-with-operator.md -# - Deploy NebulaGraph Operator: nebula-operator/2.deploy-nebula-operator.md -# - Deploy clusters: -# - Deploy clusters with Kubectl: nebula-operator/3.deploy-nebula-graph-cluster/3.1create-cluster-with-kubectl.md -# - Deploy clusters with Helm: nebula-operator/3.deploy-nebula-graph-cluster/3.2create-cluster-with-helm.md -# - Configure clusters: -# - Custom configuration parameters for a NebulaGraph cluster: nebula-operator/8.custom-cluster-configurations/8.1.custom-conf-parameter.md -# - Reclaim PVs: nebula-operator/8.custom-cluster-configurations/8.2.pv-reclaim.md -# #ent -# - Balance storage data after scaling out: nebula-operator/8.custom-cluster-configurations/8.3.balance-data-when-scaling-storage.md -# - Upgrade NebulaGraph clusters: nebula-operator/9.upgrade-nebula-cluster.md -# - Connect to NebulaGraph databases: nebula-operator/4.connect-to-nebula-graph-service.md -# - Self-healing: nebula-operator/5.operator-failover.md -# - FAQ: nebula-operator/7.operator-faq.md + - NebulaGraph Operator: + - What is NebulaGraph Operator: nebula-operator/1.introduction-to-nebula-operator.md + - Overview of using NebulaGraph Operator: nebula-operator/6.get-started-with-operator.md + - Deploy NebulaGraph Operator: nebula-operator/2.deploy-nebula-operator.md + - Deploy clusters: + - Deploy clusters with Kubectl: nebula-operator/3.deploy-nebula-graph-cluster/3.1create-cluster-with-kubectl.md + - Deploy clusters with Helm: nebula-operator/3.deploy-nebula-graph-cluster/3.2create-cluster-with-helm.md + - Configure clusters: + - Custom configuration parameters for a NebulaGraph cluster: nebula-operator/8.custom-cluster-configurations/8.1.custom-conf-parameter.md + - Reclaim PVs: nebula-operator/8.custom-cluster-configurations/8.2.pv-reclaim.md +#ent + - Balance storage data after scaling out: nebula-operator/8.custom-cluster-configurations/8.3.balance-data-when-scaling-storage.md + - Upgrade NebulaGraph clusters: nebula-operator/9.upgrade-nebula-cluster.md + - Connect to NebulaGraph databases: nebula-operator/4.connect-to-nebula-graph-service.md + - Backup and restore: nebula-operator/10.backup-restore-using-operator.md + - Self-healing: nebula-operator/5.operator-failover.md + - FAQ: nebula-operator/7.operator-faq.md - Graph computing: