Skip to content

Commit

Permalink
cherry pick pingcap#2771 to release-2.1
Browse files Browse the repository at this point in the history
Signed-off-by: sre-bot <sre-bot@pingcap.com>
  • Loading branch information
ran-huang authored and sre-bot committed Jun 11, 2020
1 parent 8a2bbd8 commit cef5885
Show file tree
Hide file tree
Showing 3 changed files with 91 additions and 1 deletion.
26 changes: 26 additions & 0 deletions TOC.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@
+ Introduction
- [TiDB Introduction](/overview.md)
+ Benchmarks
<<<<<<< HEAD
- [How to Test TiDB Using Sysbench](/benchmark/benchmark-tidb-using-sysbench.md)
- [Sysbench Performance Test - v2.1 vs. v2.0](/benchmark/v2.1-performance-benchmarking-with-sysbench.md)
- [TPC-H 50G Performance Test - v2.1 vs. v2.0](/benchmark/v2.1-performance-benchmarking-with-tpch.md)
Expand Down Expand Up @@ -80,6 +81,31 @@
- [Use Mydumper and TiDB Lightning](/backup-and-restore.md)
- [Use Dumpling for Export or Backup](/export-or-backup-using-dumpling.md)
- [Identify Slow Queries](/identify-slow-queries.md)
=======
+ [Test TiDB Using Sysbench](/benchmark/benchmark-tidb-using-sysbench.md)
+ [Run TPC-C Test on TiDB](/benchmark/benchmark-tidb-using-tpcc.md)
+ [v4.0 Sysbench Performance Test](/benchmark/benchmark-sysbench-v4-vs-v3.md)
+ [v4.0 TPC-H Performance Test](/benchmark/v4.0-performance-benchmarking-with-tpch.md)
+ [v4.0 TPC-C Performance Test](/benchmark/v4.0-performance-benchmarking-with-tpcc.md)
+ [v3.0 Sysbench Performance Test](/benchmark/v3.0-performance-benchmarking-with-sysbench.md)
+ [v3.0 TPC-C Performance Test](/benchmark/v3.0-performance-benchmarking-with-tpcc.md)
+ [Interaction Test on Online Workloads and `ADD INDEX`](/benchmark/online-workloads-and-add-index-operations.md)
+ Migrate
+ [Migration Tools Guide](/ecosystem-tool-user-guide.md)
+ Migrate from MySQL
+ [Migrate from Mydumper Files](/migrate-from-mysql-mydumper-files.md)
+ [Migrate from Amazon Aurora MySQL Using DM](/migrate-from-aurora-mysql-database.md)
+ Migrate from CSV Files
+ [Use TiDB Lightning](/tidb-lightning/migrate-from-csv-using-tidb-lightning.md)
+ [Use `LOAD DATA` Statement](/sql-statements/sql-statement-load-data.md)
+ [Migrate from SQL Files](/migrate-from-mysql-mydumper-files.md)
+ Maintain
+ Upgrade
+ [Use TiUP (Recommended)](/upgrade-tidb-using-tiup.md)
+ [Use TiUP Offline (Recommended)](/upgrade-tidb-using-tiup-offline.md)
+ [Use TiDB Operator](https://pingcap.com/docs/tidb-in-kubernetes/stable/upgrade-a-tidb-cluster/)
+ [Use TiDB Ansible](/upgrade-tidb-using-ansible.md)
>>>>>>> ab764f1... migrate: migrate from mysql files (#2771)
+ Scale
- [Scale using Ansible](/scale-tidb-using-ansible.md)
- [Scale a TiDB Cluster](/horizontal-scale.md)
Expand Down
64 changes: 64 additions & 0 deletions migrate-from-mysql-mydumper-files.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
---
title: Migrate Data from MySQL SQL Files
summary: Learn how to migrate data from MySQL SQL files to TiDB using TiDB Lightning.
category: how-to
---

# Migrate Data from MySQL SQL Files

This document describes how to migrate data from MySQL SQL files to TiDB using TiDB Lightning. For details on how to generate MySQL SQL files, refer to [Mydumper](/mydumper-overview.md) or [Dumpling](/export-or-backup-using-dumpling.md).

The data migration process described in this document uses TiDB Lightning. The steps are as follows.

## Step 1: Deploy TiDB Lightning

Before you start the migration, [deploy TiDB Lightning](/tidb-lightning/deploy-tidb-lightning.md).

> **Note:**
>
> - If you choose the Importer-backend to import data, you need to deploy `tikv-importer` along with TiDB Lightning. During the import process, the TiDB cluster cannot provide services. This mode imports data quickly, which is suitable for importing a large amount of data (above the TB level).
> - If you choose the TiDB-backend, deploy TiDB Lightning only. The cluster can provide services normally during the import.
> - For detailed differences between the two backend mode, see [TiDB Lightning Backend](/tidb-lightning/tidb-lightning-tidb-backend.md).
## Step 2: Configure data source of TiDB Lightning

This document takes the TiDB-backend as an example. Create the `tidb-lightning.toml` configuration file and add the following major configurations in the file:

1. Set the `data-source-dir` under `[mydumper]` to the path of the MySQL SQL file.

```
[mydumper]
# Data source directory
data-source-dir = "/data/export"
```
> **Note:**
>
> If a corresponding schema already exists in the downstream, set `no-schema=true` to skip the creation of the schema.
2. Add the configuration of the target TiDB cluster.
```
[tidb]
# The target cluster information. Fill in one address of tidb-server.
host = "172.16.31.1"
port = 4000
user = "root"
password = ""
```
For other configurations, see [TiDB Lightning Configuration](/tidb-lightning/tidb-lightning-configuration.md).
## Step 3: Run TiDB Lightning to import data
Run TiDB Lightning to start the import operation. If you start TiDB Lightning by using `nohup` directly in the command line, the program might exit because of the `SIGHUP` signal. Therefore, it is recommended to write `nohup` in a script. For example:
```bash
# !/bin/bash
nohup ./tidb-lightning -config tidb-lightning.toml > nohup.out &
```

When the import operation is started, view the progress by the following two ways:

- `grep` the keyword `progress` in logs, which is updated every 5 minutes by default.
- Access the monitoring dashboard. See [Monitor TiDB Lightning](/tidb-lightning/monitor-tidb-lightning.md) for details.
2 changes: 1 addition & 1 deletion mydumper-overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ aliases: ['/docs/v2.1/reference/tools/mydumper/']

## What is Mydumper?

[Mydumper](https://github.com/pingcap/mydumper) is a fork project optimized for TiDB. You can use this tool for logical backups of TiDB.
[Mydumper](https://github.com/pingcap/mydumper) is a fork project optimized for TiDB. You can use this tool for logical backups of **MySQL** or **TiDB**.

It can be [downloaded](/download-ecosystem-tools.md) as part of the Enterprise Tools package.

Expand Down

0 comments on commit cef5885

Please sign in to comment.