Skip to content

Commit

Permalink
cherry pick pingcap#2694 to release-4.0
Browse files Browse the repository at this point in the history
Signed-off-by: sre-bot <sre-bot@pingcap.com>
  • Loading branch information
toutdesuite authored and sre-bot committed Jun 8, 2020
1 parent 8f84e19 commit 8a569dd
Showing 1 changed file with 23 additions and 4 deletions.
27 changes: 23 additions & 4 deletions ticdc/ticdc-overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,10 @@ aliases: ['/docs/stable/reference/tools/ticdc/overview/']

# TiCDC Overview

> **Note:**
>
> TiCDC is experimental. It is **not recommended** to use this feature in the production environment.
[TiCDC](https://github.com/pingcap/ticdc) is a tool for replicating the incremental data of TiDB. This tool is implemented by pulling TiKV change logs. It can restore data to a consistent state with any upstream TSO, and provides [TiCDC Open Protocol](/ticdc/ticdc-open-protocol.md) to support other systems to subscribe to data changes.

## TiCDC Architecture
Expand Down Expand Up @@ -41,13 +45,28 @@ Currently, the TiCDC sink component supports replicating data to the following d
- Databases compatible with MySQL protocol. The sink component provides the final consistency support.
- Kafka based on the TiCDC Open Protocol. The sink component ensures the row-level order, final consistency or strict transactional consistency.

### Black and white table lists

You can write blacklist and whitelist filtering rules to filter or only replicate all changed data in certain databases or tables. The filtering rules are similar to those of MySQL such as `replication-rules-db` or `replication-rules-table`.

## Restrictions

To replicate data to TiDB or MySQL, you must ensure that the following requirements are satisfied to guarantee data correctness:

- The table to be replicated has the primary key or a unique index.
- If the table to be replicated only has unique indexes, each column of at least one unique index is explicitly defined in the table schema as `NOT NULL`.

### Unsupported scenarios

Currently, The following scenarios are not supported:

- The TiKV cluster that uses RawKV alone.
- The [new framework for collations](/character-set-and-collation.md#new-framework-for-collations) in TiDB v4.0. Before you enable this feature, make sure that the downstream cluster is the TiDB cluster with the same collation as those in the upstream; otherwise, inconsistent collations lead to the issue that the data cannot be located.
- The [DDL operation `CREATE SEQUENCE`](/sql-statements/sql-statement-create-sequence.md) and the [SEQUENCE function](/sql-statements/sql-statement-create-sequence.md#sequence-function) in TiDB v4.0. When the upstream TiDB uses `SEQUENCE`, TiCDC ignores `SEQUENCE` DDL operations/functions performed upstream. However, DML operations using `SEQUENCE` functions can be correctly replicated.
- The [TiKV Hibernate Region](https://github.com/tikv/tikv/blob/master/docs/reference/configuration/raftstore-config.md#hibernate-region). TiCDC prevents the Region from entering the hibernated state.
- The scheduling of existing replication tables to new TiCDC nodes, after the capacity of the TiCDC cluster is scaled out.
- The replication of Black and white table lists.

## Manage TiCDC Cluster and Replication Tasks

For details, see [Manage TiCDC Cluster and Replication Tasks](/ticdc/manage-ticdc.md).

## TiCDC Open Protocol

TiCDC Open Protocol is a row-level data change notification protocol that provides data sources for monitoring, caching, full-text indexing, analysis engines, and master-slave replication between different databases. TiCDC complies with TiCDC Open Protocol and replicates data changes of TiDB to third-party data medium such as MQ (Message Queue). For more information, see [TiCDC Open Protocol](/ticdc/ticdc-open-protocol.md).

0 comments on commit 8a569dd

Please sign in to comment.