Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add tutorial doc in website repo about how to do migration rollback #698

Merged
merged 1 commit into from
Oct 29, 2024
Merged

add tutorial doc in website repo about how to do migration rollback #698

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
174 changes: 67 additions & 107 deletions docs/tutorials/resource-migration.md
View file
Open in desktop
wulemao marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
title: Resource Migration
title: Resource Migration and Rollback
---

## Objectives
Expand All @@ -9,8 +9,8 @@ you want to migrate the existing resource to Karmada and then achieve multi-clus

So, this section will guide you to cover:

- Migrate all the existing resource from original cluster to Karmada based on resource granularity.
- Apply higher priority `PropagationPolicy` to meet more propagate demands based on application granularity.
- Migrate application from Kubernetes to Karmada.
- Rollback application migration.

## Prerequisites

Expand All @@ -32,31 +32,13 @@ export KUBECONFIG=~/.kube/karmada.config:~/.kube/members.config
>
> After the above command executed, you will see Karmada control plane installed with multi member clusters.

### Enable PropagationPolicyPreemption in karmada-controller-manager

#### Step 2: Run the command

```shell
kubectl --context karmada-host get deploy karmada-controller-manager -n karmada-system -o yaml | sed '/- --failover-eviction-timeout=30s/{n;s/- --v=4/- --feature-gates=PropagationPolicyPreemption=true\n &/g}' | kubectl --context karmada-host replace -f -
```

> **Note:**
>
> The feature `PropagationPolicy Priority and Preemption` was introduced in v1.7, and it is controlled by the feature gate `PropagationPolicyPreemption` which is disabled by default.
>
> You can just execute the above one command to enable this feature gate. Or, if you want to use a more cautious approach, you can do like this:
>
> 1. execute `kubectl --context karmada-host edit deploy karmada-controller-manager -n karmada-system`
> 2. check if feature gate `--feature-gates=PropagationPolicyPreemption=true` is existed in `spec.template.spec.containers[0].command` field.
> 3. If not, you shall add `--feature-gates=PropagationPolicyPreemption=true` into that field.

### Preset resource in a member cluster

To simulate resources already exist in member cluster, we deploy some simple Deployments and Services to `member1` cluster.
To simulate resources already exist in member cluster, we deploy simple Deployment to `member1` cluster.

#### Step 3: Write the code
#### Step 2: Write the code

Create new file `/tmp/deployments-and-services.yaml` and copy text below to it:
Create new file `/tmp/deployments.yaml` and copy text below to it:

```yaml
apiVersion: apps/v1
Expand All @@ -78,54 +60,47 @@ spec:
image: nginx:latest
ports:
- containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
name: nginx-svc
spec:
selector:
app: nginx
type: NodePort
ports:
- port: 80
nodePort: 30000
targetPort: 80
```

#### Step 4: Run the command
### Step 3: Run the command

```shell
$ kubectl --context member1 apply -f /tmp/deployments-and-services.yaml
$ kubectl --context member1 apply -f /tmp/deployments.yaml
deployment.apps/nginx-deploy created
service/nginx-svc created
deployment.apps/hello-deploy created
service/hello-svc created

$ karmadactl --karmada-context karmada-apiserver get deploy --operation-scope=all
NAME CLUSTER READY UP-TO-DATE AVAILABLE AGE ADOPTION
nginx-deploy member1 2/2 2 2 15m N
```

Thus, we can use `member1` as the cluster with existing resources, while `member2` as a bare cluster.
You will see the current `nginx-deploy` deployment in the member1 cluster,
with `ADOPTION=N` indicating it is not currently managed by Karmada.

## Tutorials

### Migrate all the resources to Karmada
### Migrate application to Karmada

To make the tutorial simpler and easier to understand,
here we assume the application example contains only a single simple deployment.

#### Step 1: Run the command

```shell
$ kubectl --context karmada-apiserver apply -f /tmp/deployments-and-services.yaml
$ kubectl --context karmada-apiserver apply -f /tmp/deployments.yaml
deployment.apps/nginx-deploy created
service/nginx-svc created
deployment.apps/hello-deploy created
service/hello-svc created

$ karmadactl --karmada-context karmada-apiserver get deploy --operation-scope=all
NAME CLUSTER READY UP-TO-DATE AVAILABLE AGE ADOPTION
nginx-deploy Karmada 0/2 0 0 7s -
nginx-deploy member1 2/2 2 2 16m N
```

> **Note：**
>
> Same Deployments and Services should be deployed to Karmada control plane as [ResourceTemplate](https://karmada.io/docs/core-concepts/concepts#resource-template).
You will see that the Deployment has been deployed to the Karmada control plane as a [ResourceTemplate](https://karmada.io/docs/core-concepts/concepts#resource-template),
but there is currently no matching PropagationPolicy to propagate it.

#### Step 2: Write the code

Create new file `/tmp/pp-for-migrating-deployments-and-services.yaml` and copy text below to it:
Create new file `/tmp/pp-for-migrating-deployments.yaml` and copy text below to it:

```yaml
apiVersion: policy.karmada.io/v1alpha1
Expand All @@ -138,98 +113,83 @@ spec:
clusterAffinity:
clusterNames:
- member1
priority: 0
resourceSelectors:
- apiVersion: apps/v1
kind: Deployment
- apiVersion: v1
kind: Service
schedulerName: default-scheduler
name: nginx-deploy
```

> **Note:**
>
> You should pay attention to two fields：
> You should pay attention to two fields：
>
> * `spec.conflictResolution: Overwrite`：the value must be [Overwrite](https://github.com/karmada-io/karmada/blob/master/docs/proposals/migration/design-of-seamless-cluster-migration-scheme.md#proposal).
> * `spec.resourceSelectors`：selecting resources to migrate, you can define your custom [ResourceSelector](https://karmada.io/docs/userguide/scheduling/override-policy/#resource-selector).

#### Step 3: Run the command
#### Step 3: Run the Command

Apply this PropagationPolicy to Karmada control plane.
Apply the above PropagationPolicy to the Karmada control plane:

```shell
$ kubectl --context karmada-apiserver apply -f /tmp/pp-for-migrating-deployments-and-services.yaml
$ kubectl --context karmada-apiserver apply -f /tmp/pp-for-migrating-deployments.yaml
propagationpolicy.policy.karmada.io/migrate-pp created
```

#### Step 4: Verification

```shell
$ kubectl --context karmada-apiserver get deploy
NAME READY UP-TO-DATE AVAILABLE AGE
nginx-deploy 2/2 2 2 38s
$ kubectl --context karmada-apiserver get rb
NAME SCHEDULED FULLYAPPLIED AGE
nginx-deploy-deployment True True 13s
nginx-svc-service True True 13s
$ karmadactl --karmada-context karmada-apiserver get deploy --operation-scope=all
NAME CLUSTER READY UP-TO-DATE AVAILABLE AGE ADOPTION
nginx-deploy Karmada 2/2 2 2 34s -
nginx-deploy member1 2/2 2 2 16m Y

$ karmadactl --karmada-context karmada-apiserver get pods --operation-scope=all
NAME CLUSTER READY STATUS RESTARTS AGE
nginx-deploy-54b9c68f67-4qjrz member1 1/1 Running 0 16m
nginx-deploy-54b9c68f67-f4qpm member1 1/1 Running 0 16m
```

You shall see the Deployments in Karmada are all ready and the `aggregatedStatus` of `ResourceBinding` is applied,
which means the existing resources in `member1` cluster has been taken over by Karmada.
You will see that all Deployments are ready,
which means `nginx-deploy` in the member1 cluster is successfully taken over by Karmada:

So far, you have finished the migration, isn't it so easy?
* `AGE=16m` indicates the resource is taken over rather than recreated, with corresponding pods no need to be restarted.
* `ADOPTION=Y` indicates that the resource is now managed by Karmada.

### Apply higher priority PropagationPolicy
You have now completed the migration.

#### Step 5: Write the code
### Rollback Migration

Create new file `/tmp/pp-for-nginx-app.yaml` and copy text below to it:
#### Step 5: Set preserveResourcesOnDeletion for PropagationPolicy

```yaml
apiVersion: policy.karmada.io/v1alpha1
kind: PropagationPolicy
metadata:
name: nginx-pp
spec:
conflictResolution: Overwrite
placement:
clusterAffinity:
clusterNames:
- member1
- member2 ## propagate to more clusters other than member1
priority: 10 ## priority greater than above PropagationPolicy (10 > 0)
preemption: Always ## preemption should equal to Always
resourceSelectors:
- apiVersion: apps/v1
kind: Deployment
name: nginx-deploy
- apiVersion: v1
kind: Service
name: nginx-svc
schedulerName: default-scheduler
Run the following command to modify `PropagationPolicy/migrate-pp`, setting `spec.preserveResourcesOnDeletion=true`:

```shell
$ kubectl --context karmada-apiserver patch pp migrate-pp --type='json' -p '[{"op": "replace", "path": "/spec/preserveResourcesOnDeletion", "value": true}]'
propagationpolicy.policy.karmada.io/nginx-pp patched
```

#### Step 6: Run the command
#### Step 6: Delete the Resource Template from the Control Plane

Apply this higher priority PropagationPolicy to Karmada control plane.
Run the following command:

```shell
$ kubectl --context karmada-apiserver apply -f /tmp/pp-for-nginx-app.yaml
propagationpolicy.policy.karmada.io/nginx-pp created
$ kubectl --context karmada-apiserver delete -f /tmp/deployments.yaml
deployment.apps "nginx-deploy" deleted
```

#### Step 7: Verification

Run the following command:

```shell
$ kubectl --context member2 get deploy -o wide
NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR
nginx-deploy 2/2 2 2 5m24s nginx nginx:latest app=nginx
$ kubectl --context member2 get svc -o wide
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
nginx-svc NodePort 10.13.161.255 <none> 80:30000/TCP 54s app=nginx
$ karmadactl --karmada-context karmada-apiserver get deploy --operation-scope=all
NAME CLUSTER READY UP-TO-DATE AVAILABLE AGE ADOPTION
nginx-deploy member1 2/2 2 2 17m N
...
```

As you see, you shall find `nginx` application related resource are all propagated to `member2` cluster,
which means the higher priority `PropagationPolicy` does work.
You will see that the resource template in the control plane has been deleted,
and the resources in the member clusters remain intact,
with `ADOPTION=N` indicating that the resource is not managed by Karmada.

You have now completed the migration rollback.
Loading