Skip to content

Commit

Permalink
add tutorial doc in website repo about how to do migration rollback
Browse files Browse the repository at this point in the history
Signed-off-by: wulemao <1194736083@qq.com>
  • Loading branch information
wulemao committed Oct 28, 2024
1 parent a612b36 commit b5c52dc
Show file tree
Hide file tree
Showing 2 changed files with 127 additions and 209 deletions.
173 changes: 67 additions & 106 deletions docs/tutorials/resource-migration.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
title: Resource Migration
title: Resource Migration and Rollback
---

## Objectives
Expand All @@ -9,8 +9,8 @@ you want to migrate the existing resource to Karmada and then achieve multi-clus

So, this section will guide you to cover:

- Migrate all the existing resource from original cluster to Karmada based on resource granularity.
- Apply higher priority `PropagationPolicy` to meet more propagate demands based on application granularity.
- Migrate application from Kubernetes to Karmada.
- Rollback application migration.

## Prerequisites

Expand All @@ -32,31 +32,13 @@ export KUBECONFIG=~/.kube/karmada.config:~/.kube/members.config
>
> After the above command executed, you will see Karmada control plane installed with multi member clusters.
### Enable PropagationPolicyPreemption in karmada-controller-manager

#### Step 2: Run the command

```shell
kubectl --context karmada-host get deploy karmada-controller-manager -n karmada-system -o yaml | sed '/- --failover-eviction-timeout=30s/{n;s/- --v=4/- --feature-gates=PropagationPolicyPreemption=true\n &/g}' | kubectl --context karmada-host replace -f -
```

> **Note:**
>
> The feature `PropagationPolicy Priority and Preemption` was introduced in v1.7, and it is controlled by the feature gate `PropagationPolicyPreemption` which is disabled by default.
>
> You can just execute the above one command to enable this feature gate. Or, if you want to use a more cautious approach, you can do like this:
>
> 1. execute `kubectl --context karmada-host edit deploy karmada-controller-manager -n karmada-system`
> 2. check if feature gate `--feature-gates=PropagationPolicyPreemption=true` is existed in `spec.template.spec.containers[0].command` field.
> 3. If not, you shall add `--feature-gates=PropagationPolicyPreemption=true` into that field.
### Preset resource in a member cluster

To simulate resources already exist in member cluster, we deploy some simple Deployments and Services to `member1` cluster.
To simulate resources already exist in member cluster, we deploy simple Deployment to `member1` cluster.

#### Step 3: Write the code
#### Step 2: Write the code

Create new file `/tmp/deployments-and-services.yaml` and copy text below to it:
Create a new file `/tmp/deployments.yaml` and copy text below to it:

```yaml
apiVersion: apps/v1
Expand All @@ -78,54 +60,47 @@ spec:
image: nginx:latest
ports:
- containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
name: nginx-svc
spec:
selector:
app: nginx
type: NodePort
ports:
- port: 80
nodePort: 30000
targetPort: 80
```
#### Step 4: Run the command
### Step 3: Run the command
```shell
$ kubectl --context member1 apply -f /tmp/deployments-and-services.yaml
$ kubectl --context member1 apply -f /tmp/deployments.yaml
deployment.apps/nginx-deploy created
service/nginx-svc created
deployment.apps/hello-deploy created
service/hello-svc created

$ karmadactl --karmada-context karmada-apiserver get deploy --operation-scope=all
NAME CLUSTER READY UP-TO-DATE AVAILABLE AGE ADOPTION
nginx-deploy member1 2/2 2 2 15m N
```

Thus, we can use `member1` as the cluster with existing resources, while `member2` as a bare cluster.
You will see the current `nginx-deploy` deployment in the member1 cluster,
with `ADOPTION=N` indicating it is not currently managed by Karmada.

## Tutorials

### Migrate all the resources to Karmada
### Migrate application to Karmada

To make the tutorial simpler and easier to understand,
here we assume the application example contains only a single simple deployment.

#### Step 1: Run the command

```shell
$ kubectl --context karmada-apiserver apply -f /tmp/deployments-and-services.yaml
$ kubectl --context karmada-apiserver apply -f /tmp/deployments.yaml
deployment.apps/nginx-deploy created
service/nginx-svc created
deployment.apps/hello-deploy created
service/hello-svc created

$ karmadactl --karmada-context karmada-apiserver get deploy --operation-scope=all
NAME CLUSTER READY UP-TO-DATE AVAILABLE AGE ADOPTION
nginx-deploy Karmada 0/2 0 0 7s -
nginx-deploy member1 2/2 2 2 16m N
```

> **Note:**
>
> Same Deployments and Services should be deployed to Karmada control plane as [ResourceTemplate](https://karmada.io/docs/core-concepts/concepts#resource-template).
You will see that the Deployment has been deployed to the Karmada control plane as a [ResourceTemplate](https://karmada.io/docs/core-concepts/concepts#resource-template),
but there is currently no matching PropagationPolicy to propagate it.

#### Step 2: Write the code

Create new file `/tmp/pp-for-migrating-deployments-and-services.yaml` and copy text below to it:
Create new file `/tmp/pp-for-migrating-deployments.yaml` and copy text below to it:

```yaml
apiVersion: policy.karmada.io/v1alpha1
Expand All @@ -138,98 +113,84 @@ spec:
clusterAffinity:
clusterNames:
- member1
priority: 0
resourceSelectors:
- apiVersion: apps/v1
kind: Deployment
- apiVersion: v1
kind: Service
name: nginx-deploy
schedulerName: default-scheduler
```
> **Note:**
>
> You should pay attention to two fields:
> You should pay attention to two fields:
>
> * `spec.conflictResolution: Overwrite`:the value must be [Overwrite](https://github.com/karmada-io/karmada/blob/master/docs/proposals/migration/design-of-seamless-cluster-migration-scheme.md#proposal).
> * `spec.resourceSelectors`:selecting resources to migrate, you can define your custom [ResourceSelector](https://karmada.io/docs/userguide/scheduling/override-policy/#resource-selector).

#### Step 3: Run the command
#### Step 3: Run the Command

Apply this PropagationPolicy to Karmada control plane.
Apply the above PropagationPolicy to the Karmada control plane:

```shell
$ kubectl --context karmada-apiserver apply -f /tmp/pp-for-migrating-deployments-and-services.yaml
$ kubectl --context karmada-apiserver apply -f /tmp/pp-for-migrating-deployments.yaml
propagationpolicy.policy.karmada.io/migrate-pp created
```

#### Step 4: Verification

```shell
$ kubectl --context karmada-apiserver get deploy
NAME READY UP-TO-DATE AVAILABLE AGE
nginx-deploy 2/2 2 2 38s
$ kubectl --context karmada-apiserver get rb
NAME SCHEDULED FULLYAPPLIED AGE
nginx-deploy-deployment True True 13s
nginx-svc-service True True 13s
$ karmadactl --karmada-context karmada-apiserver get deploy --operation-scope=all
NAME CLUSTER READY UP-TO-DATE AVAILABLE AGE ADOPTION
nginx-deploy Karmada 2/2 2 2 34s -
nginx-deploy member1 2/2 2 2 16m Y
$ karmadactl --karmada-context karmada-apiserver get pods --operation-scope=all
NAME CLUSTER READY STATUS RESTARTS AGE
nginx-deploy-54b9c68f67-4qjrz member1 1/1 Running 0 16m
nginx-deploy-54b9c68f67-f4qpm member1 1/1 Running 0 16m
```

You shall see the Deployments in Karmada are all ready and the `aggregatedStatus` of `ResourceBinding` is applied,
which means the existing resources in `member1` cluster has been taken over by Karmada.
You will see that all Deployments are ready,
which means `nginx-deploy` in the member1 cluster is successfully taken over by Karmada:

So far, you have finished the migration, isn't it so easy?
* `AGE=16m` indicates the resource is taken over rather than recreated, with corresponding pods no need to be restarted.
* `ADOPTION=Y` indicates that the resource is now managed by Karmada.

### Apply higher priority PropagationPolicy
You have now completed the migration.

#### Step 5: Write the code
### Rollback Migration

Create new file `/tmp/pp-for-nginx-app.yaml` and copy text below to it:
#### Step 5: Set preserveResourcesOnDeletion for PropagationPolicy

```yaml
apiVersion: policy.karmada.io/v1alpha1
kind: PropagationPolicy
metadata:
name: nginx-pp
spec:
conflictResolution: Overwrite
placement:
clusterAffinity:
clusterNames:
- member1
- member2 ## propagate to more clusters other than member1
priority: 10 ## priority greater than above PropagationPolicy (10 > 0)
preemption: Always ## preemption should equal to Always
resourceSelectors:
- apiVersion: apps/v1
kind: Deployment
name: nginx-deploy
- apiVersion: v1
kind: Service
name: nginx-svc
schedulerName: default-scheduler
Run the following command to modify `PropagationPolicy/migrate-pp`, setting `spec.preserveResourcesOnDeletion=true`:

```shell
$ kubectl --context karmada-apiserver patch pp migrate-pp --type='json' -p '[{"op": "replace", "path": "/spec/preserveResourcesOnDeletion", "value": true}]'
propagationpolicy.policy.karmada.io/nginx-pp patched
```

#### Step 6: Run the command
#### Step 6: Delete the Resource Template from the Control Plane

Apply this higher priority PropagationPolicy to Karmada control plane.
Run the following command:

```shell
$ kubectl --context karmada-apiserver apply -f /tmp/pp-for-nginx-app.yaml
propagationpolicy.policy.karmada.io/nginx-pp created
$ kubectl --context karmada-apiserver delete -f /tmp/deployments.yaml
deployment.apps "nginx-deploy" deleted
```

#### Step 7: Verification

Run the following command:

```shell
$ kubectl --context member2 get deploy -o wide
NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR
nginx-deploy 2/2 2 2 5m24s nginx nginx:latest app=nginx
$ kubectl --context member2 get svc -o wide
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
nginx-svc NodePort 10.13.161.255 <none> 80:30000/TCP 54s app=nginx
$ karmadactl --karmada-context karmada-apiserver get deploy --operation-scope=all
NAME CLUSTER READY UP-TO-DATE AVAILABLE AGE ADOPTION
nginx-deploy member1 2/2 2 2 17m N
...
```

As you see, you shall find `nginx` application related resource are all propagated to `member2` cluster,
which means the higher priority `PropagationPolicy` does work.
You will see that the resource template in the control plane has been deleted,
and the resources in the member clusters remain intact,
with `ADOPTION=N` indicating that the resource is not managed by Karmada.

You have now completed the migration rollback.
Loading

0 comments on commit b5c52dc

Please sign in to comment.