Skip to content

Commit

Permalink
en: add doc for advanced statefulset (#110) (#137)
Browse files Browse the repository at this point in the history
Signed-off-by: sre-bot <sre-bot@pingcap.com>
  • Loading branch information
sre-bot authored Mar 31, 2020
1 parent 65c39ca commit 6e74929
Show file tree
Hide file tree
Showing 3 changed files with 211 additions and 5 deletions.
1 change: 1 addition & 0 deletions en/TOC.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,5 +62,6 @@
- [TiDB Toolkit](tidb-toolkit.md)
+ Components
- [TiDB Scheduler](tidb-scheduler.md)
- [Advanced StatefulSet Controller](advanced-statefulset.md)
- [Troubleshoot](troubleshoot.md)
- [FAQs](faq.md)
205 changes: 205 additions & 0 deletions en/advanced-statefulset.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,205 @@
---
title: Advanced StatefulSet Controller
summary: Learn how to enable and use the advanced StatefulSet controller.
category: reference
---

# Advanced StatefulSet Controller

**Feature Stage**: Alpha

Kubernetes has a built-in [StatefulSet](https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/) that allocates consecutive serial numbers to Pods. For example, when there are three replicas, the Pods are named as pod-0, pod-1, and pod-2. When scaling out or scaling in, you must add a Pod at the end or delete the last pod. For example, when you scale out to four replicas, pod-3 is added. When you scale in to two replicas, pod-2 is deleted.

When you use local storage, Pods are associated with the Nodes storage resources and cannot be scheduled freely. If you want to delete one of the Pods in the middle to maintain its Node but no other Nodes can be migrated, or if you want to delete a Pod that fails and to create another Pod with a different serial number, you cannot implement such desired function by the built-in StatefulSet.

The [advanced StatefulSet controller](https://github.com/pingcap/advanced-statefulset) is implemented based on the built-in StatefulSet controller. It supports freely controlling the serial number of Pods. This document describes how to use the advanced StatefulSet controller in TiDB Operator.

## Enable

1. Load the Advanced StatefulSet CRD file:

* For Kubernetes versions < 1.16:

{{< copyable "shell-regular" >}}

```shell
kubectl apply -f https://raw.githubusercontent.com/pingcap/tidb-operator/master/manifests/advanced-statefulset-crd.v1beta1.yaml
```

* For Kubernetes versions >= 1.16:

{{< copyable "shell-regular" >}}

```
kubectl apply -f https://raw.githubusercontent.com/pingcap/tidb-operator/master/manifests/advanced-statefulset-crd.v1.yaml
```

2. Enable the `AdvancedStatefulSet` feature in `values.yaml` of the TiDB Operator chart:

{{< copyable "shell-regular" >}}

```yaml
features:
- AdvancedStatefulSet=true
advancedStatefulset:
create: true
```

Upgrade TiDB Operator. For details, refer to [Upgrade TiDB Operator](upgrade-tidb-operator.md).

> **Note:**
>
> If the `AdvancedStatefulSet` feature is enabled, TiDB Operator converts the current `StatefulSet` object into an `AdvancedStatefulSet` object. However, after the `AdvancedStatefulSet` feature is disabled, the `AdvancedStatefulSet` object cannot be automatically converted to the built-in `StatefulSet` object of Kubernetes.

## Usage

This section describes how to use the advanced StatefulSet controller.

### View the `AdvancedStatefulSet` Object by kubectl

The data format of `AdvancedStatefulSet` is the same as that of `StatefulSet`, but `AdvancedStatefulSet` is implemented based on CRD, with `asts` as the alias. You can view the `AdvancedStatefulSet` object in the namespace by running the following command:

{{< copyable "shell-regular" >}}

```shell
kubectl get -n <namespace> asts
```

### Specify the Pod to be scaled in

With the advanced StatefulSet controller, when scaling in TidbCluster, you can not only reduce the number of replicas, but also specify the scaling in of any Pod in the PD, TiDB, or TiKV components by configuring annotations.

For example:

{{< copyable "" >}}

```yaml
apiVersion: pingcap.com/v1alpha1
kind: TidbCluster
metadata:
name: asts
spec:
version: v3.0.12
timezone: UTC
pvReclaimPolicy: Delete
pd:
baseImage: pingcap/pd
replicas: 3
requests:
storage: "1Gi"
config: {}
tikv:
baseImage: pingcap/tikv
replicas: 4
requests:
storage: "1Gi"
config: {}
tidb:
baseImage: pingcap/tidb
replicas: 2
service:
type: ClusterIP
config: {}
```

The above configuration deploys 4 TiKV instances, namely `basic-tikv-0`, `basic-tikv-1`, ..., `basic-tikv-3`. If you want to delete `basic-tikv-1`, set `spec.tikv.replicas` to `3` and configure the following annotations:

{{< copyable "" >}}

```yaml
metadata:
annotations:
tikv.tidb.pingcap.com/delete-slots: '[1]'
```

> **Note:**
>
> When modifying `replicas` and `delete-slots annotation`, complete the modification in the same operation; otherwise, the controller operates the modification according to the general expectations.

The complete example is as follows:

{{< copyable "" >}}

```yaml
apiVersion: pingcap.com/v1alpha1
kind: TidbCluster
metadata:
annotations:
tikv.tidb.pingcap.com/delete-slots: '[1]'
name: asts
spec:
version: v3.0.12
timezone: UTC
pvReclaimPolicy: Delete
pd:
baseImage: pingcap/pd
replicas: 3
requests:
storage: "1Gi"
config: {}
tikv:
baseImage: pingcap/tikv
replicas: 3
requests:
storage: "1Gi"
config: {}
tidb:
baseImage: pingcap/tidb
replicas: 2
service:
type: ClusterIP
config: {}
```

The supported annotations are as follows:

- `pd.tidb.pingcap.com/delete-slots`: Specifies the serial numbers of the Pods to be deleted in the PD component.
- `tidb.tidb.pingcap.com/delete-slots`: Specifies the serial number of the Pods to be deleted in the TiDB component.
- `tikv.tidb.pingcap.com/delete-slots`: Specifies the serial number of the Pods to be deleted in the TiKV component.

The value of Annotation is an integer array of JSON, such as `[0]`, `[0,1]`, `[1,3]`.

### Specify the location to scale out

You can reverse the above operation of scaling in to restore `basic-tikv-1`.

> **Note:**
>
> The specified scaling out performed by the advanced StatefulSet controller is the same as the regular StatefulSet scaling, which does not delete the Persistent Volume Claims (PVCs) associated with the Pod. If you want to avoid using the previous data, delete the associated PVCs before scaling out at the original location.

For example:

{{< copyable "" >}}

```yaml
apiVersion: pingcap.com/v1alpha1
kind: TidbCluster
metadata:
annotations:
tikv.tidb.pingcap.com/delete-slots: '[]'
name: asts
spec:
version: v3.0.12
timezone: UTC
pvReclaimPolicy: Delete
pd:
baseImage: pingcap/pd
replicas: 3
requests:
storage: "1Gi"
config: {}
tikv:
baseImage: pingcap/tikv
replicas: 4
requests:
storage: "1Gi"
config: {}
tidb:
baseImage: pingcap/tidb
replicas: 2
service:
type: ClusterIP
config: {}
```

The delete-slots annotations can be left empty or deleted completely.
10 changes: 5 additions & 5 deletions zh/advanced-statefulset.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ category: reference

# 增强型 StatefulSet 控制器

> **特性状态**: Alpha
**特性状态**: Alpha

Kubernetes 内置 [StatefulSet](https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/) 为 Pods 分配连续的序号。比如 3 个副本时,Pods 分别为 pod-0, pod-1, pod-2。扩缩容时,必须在尾部增加或删除 Pods。比如扩容到 4 个副本时,会新增 pod-3。缩容到 2 副本时,会删除 pod-2。

Expand All @@ -26,7 +26,7 @@ Kubernetes 内置 [StatefulSet](https://kubernetes.io/docs/concepts/workloads/co
kubectl apply -f https://raw.githubusercontent.com/pingcap/tidb-operator/master/manifests/advanced-statefulset-crd.v1beta1.yaml
```

* Kubernetes 1.16 之后:
* Kubernetes 1.16 及之后版本:

{{< copyable "shell-regular" >}}

Expand All @@ -45,17 +45,17 @@ Kubernetes 内置 [StatefulSet](https://kubernetes.io/docs/concepts/workloads/co
create: true
```

然后升级 TiDB Operator,具体可参考[升级 TiDB Operator 文档](upgrade-TiDB Operator.md)。
然后升级 TiDB Operator,具体可参考[升级 TiDB Operator 文档](upgrade-tidb-operator.md)。

> **注意:**
>
> TiDB Operator 通过开启 `AdvancedStatefulSet` 特性,会将当前 `StatefulSet` 对象转换成 `AdvancedStatefulSet 对象。但是,TiDB Operator 不支持在关闭 `AdvancedStatefulSet` 特性后,自动从 `AdvancedStatefulSet` 转换为 Kubernetes 内置的 `StatefulSet` 对象。
> TiDB Operator 通过开启 `AdvancedStatefulSet` 特性,会将当前 `StatefulSet` 对象转换成 `AdvancedStatefulSet` 对象。但是,TiDB Operator 不支持在关闭 `AdvancedStatefulSet` 特性后,自动从 `AdvancedStatefulSet` 转换为 Kubernetes 内置的 `StatefulSet` 对象。

## 使用

### 通过 kubectl 查看 AdvancedStatefulSet 对象

`AdvancedStatefulSet` 数据格式与 `StatefulSet` 完全一致,但以 CRD 方式实现,别名为 `asts`可通过以下方法查看命令空间下的对象
`AdvancedStatefulSet` 数据格式与 `StatefulSet` 完全一致,但以 CRD 方式实现,别名为 `asts`可通过以下方法查看命名空间下的对象

{{< copyable "shell-regular" >}}

Expand Down

0 comments on commit 6e74929

Please sign in to comment.