Add 'pkg/lightning/' from commit '1f120218b385d40702d7061830803cb11c1…

…a56b7'

git-subtree-dir: pkg/lightning
git-subtree-mainline: cb828b8
git-subtree-split: 1f12021

pkg/lightning/.gitattributes

@@ -0,0 +1 @@
+*_generated.go linguist-generated=true

pkg/lightning/.github/ISSUE_TEMPLATE/bug-report.md

@@ -0,0 +1,59 @@
+---
+name: "\U0001F41B Bug Report"
+labels: "bug"
+about: Something isn't working as expected
+
+---
+
+## Bug Report
+
+Please answer these questions before submitting your issue. Thanks!
+
+1. What did you do? If possible, provide a recipe for reproducing the error.
+
+2. What did you expect to see?
+
+3. What did you see instead?
+
+4. Versions of the cluster
+
+    - TiDB-Lightning version (run `tidb-lightning -V`):
+
+        ```
+        (paste TiDB-Lightning version here)
+        ```
+        
+    - TiKV-Importer version (run `tikv-importer -V`)
+
+        ```
+        (paste TiKV-Importer version here)
+        ```
+        
+    - TiKV version (run `tikv-server -V`):
+
+        ```
+        (paste TiKV version here)
+        ```
+
+    - TiDB cluster version (execute `SELECT tidb_version();` in a MySQL client):
+
+        ```
+        (paste TiDB cluster version here)
+        ```
+
+    - Other interesting information (system version, hardware config, etc):
+
+        >
+        >
+
+5. Operation logs
+   - Please upload `tidb-lightning.log` for TiDB-Lightning if possible
+   - Please upload `tikv-importer.log` from TiKV-Importer if possible
+   - Other interesting logs
+   
+6. Configuration of the cluster and the task
+   - `tidb-lightning.toml` for TiDB-Lightning if possible
+   - `tikv-importer.toml` for TiKV-Importer if possible
+   - `inventory.ini` if deployed by Ansible
+
+7. Screenshot/exported-PDF of Grafana dashboard or metrics' graph in Prometheus for TiDB-Lightning if possible

pkg/lightning/.github/ISSUE_TEMPLATE/feature-request.md

@@ -0,0 +1,20 @@
+---
+name: "\U0001F680 Feature Request"
+labels: "feature-request"
+about: I have a suggestion
+
+---
+
+## Feature Request
+
+**Is your feature request related to a problem? Please describe:**
+<!-- A clear and concise description of what the problem is. Ex. I'm always frustrated when [...] -->
+
+**Describe the feature you'd like:**
+<!-- A clear and concise description of what you want to happen. -->
+
+**Describe alternatives you've considered:**
+<!-- A clear and concise description of any alternative solutions or features you've considered. -->
+
+**Teachability, Documentation, Adoption, Optimization:**
+<!-- If you can, explain some scenarios how users might use this, situations it would be helpful in. Any API designs, mockups, or diagrams are also helpful. -->

pkg/lightning/.github/ISSUE_TEMPLATE/question.md

@@ -0,0 +1,17 @@
+---
+name: "\U0001F914 Question"
+labels: "question"
+about: Usage question that isn't answered in docs or discussion
+
+---
+
+## Question
+
+Before asking a question, make sure you have:
+
+- Searched existing Stack Overflow questions.
+- Googled your question.
+- Searched open and closed [GitHub issues](https://github.com/pingcap/tidb-lightning/issues?q=is%3Aissue)
+- Read the documentation:
+    * [Detailed documentation](https://pingcap.com/docs/tools/lightning/overview-architecture/)
+    * [简体中文文档](https://pingcap.com/docs-cn/tools/lightning/overview-architecture/)

pkg/lightning/.github/challenge-bot.yml

@@ -0,0 +1 @@
+defaultSigLabel: sig/migrate

pkg/lightning/.github/pull_request_template.md

@@ -0,0 +1,31 @@
+<!--
+Thank you for contributing to TiDB-Lightning! Please read the [CONTRIBUTING](https://github.com/pingcap/tidb-lightning/blob/master/CONTRIBUTING.md) document **BEFORE** filing this PR.
+-->
+
+### What problem does this PR solve? <!--add issue link with summary if exists-->
+
+
+### What is changed and how it works?
+
+
+### Check List <!--REMOVE the items that are not applicable-->
+
+Tests <!-- At least one of them must be included. -->
+
+ - Unit test
+ - Integration test
+ - Manual test (add detailed scripts or steps below)
+ - No code
+
+Side effects
+
+ - Possible performance regression
+ - Increased code complexity
+ - Breaking backward compatibility
+
+Related changes
+
+ - Need to cherry-pick to the release branch
+ - Need to update the documentation
+ - Need to update the `tidb-ansible` repository
+ - Need to be included in the release note

pkg/lightning/.gitignore

@@ -0,0 +1,17 @@
+.DS_Store
+bin/
+test_*/
+*.log
+*.tar.gz
+.idea
+*.local
+*.pyc
+*.ezdraw
+vendor/
+tools/bin/
+go.mod
+go.sum
+
+# for the web interface
+web/node_modules/
+web/dist/

pkg/lightning/CONTRIBUTING.md

@@ -0,0 +1,118 @@
+# How to contribute
+
+This document outlines some of the conventions on development workflow, commit
+message formatting, contact points and other resources to make it easier to get
+your contribution accepted.
+
+## Getting started
+
+- Fork the repository on GitHub.
+- Read the README.md for build instructions.
+- Play with the project, submit bugs, submit patches!
+
+## Building TiDB-Lightning
+
+Developing TiDB-Lightning requires:
+
+* [Go 1.13.5+](http://golang.org/doc/code.html)
+* An internet connection to download the dependencies
+
+Simply run `make` to build the program.
+
+```sh
+make
+```
+
+### Running tests
+
+This project contains unit tests and integration tests with coverage collection.
+See [tests/README.md](./tests/README.md) for how to execute and add tests.
+
+### Updating generated code
+
+TiDB-Lightning contains some generated source code for parsing. To modify them,
+you also need to install:
+
+* [Ragel 6.10+](https://www.colm.net/open-source/ragel/)
+* [protoc 3.6+](https://github.com/protocolbuffers/protobuf/releases)
+* [protoc-gen-gogofaster 1.2+](https://github.com/gogo/protobuf#more-speed-and-more-generated-code)
+
+Run `make data_parsers` to regenerate these source code.
+
+### Updating dependencies
+
+TiDB-Lightning manages dependencies using [Go 1.11 module](https://github.com/golang/go/wiki/Modules).
+To add or update a dependency, either
+
+* Use the `go mod edit` command to change the dependency, or
+* Edit `go.mod` and then run `make update` to update the checksum.
+
+## Contribution flow
+
+This is a rough outline of what a contributor's workflow looks like:
+
+- Create a topic branch from where you want to base your work. This is usually `master`.
+- Make commits of logical units and add test case if the change fixes a bug or adds new functionality.
+- Run tests and make sure all the tests are passed.
+- Make sure your commit messages are in the proper format (see below).
+- Push your changes to a topic branch in your fork of the repository.
+- Submit a pull request.
+- Your PR must receive LGTMs from two maintainers.
+
+Thanks for your contributions!
+
+### Code style
+
+The coding style suggested by the Golang community is used in TiDB-Lightning.
+See the [style doc](https://github.com/golang/go/wiki/CodeReviewComments) for details.
+
+Please follow this style to make TiDB-Lightning easy to review, maintain and develop.
+
+### Format of the Commit Message
+
+We follow a rough convention for commit messages that is designed to answer two
+questions: what changed and why. The subject line should feature the what and
+the body of the commit should describe the why.
+
+```
+restore: add comment for variable declaration
+
+Improve documentation.
+```
+
+The format can be described more formally as follows:
+
+```
+<subsystem>: <what changed>
+<BLANK LINE>
+<why this change was made>
+<BLANK LINE>
+<footer>(optional)
+```
+
+The first line is the subject and should be no longer than 70 characters, the
+second line is always blank, and other lines should be wrapped at 80 characters.
+This allows the message to be easier to read on GitHub as well as in various
+git tools.
+
+If the change affects more than one subsystem, you can use comma to separate them like `restore,mydump:`.
+
+If the change affects many subsystems, you can use ```*``` instead, like ```*:```.
+
+For the why part, if no specific reason for the change,
+you can use one of some generic reasons like "Improve documentation.",
+"Improve performance.", "Improve robustness.", "Improve test coverage."
+
+## Related projects
+
+This repository is one of the many components forming the whole TiDB-Lightning
+Toolset.
+
+The source code of `tikv-importer` can be found in the TiKV project:
+<https://github.com/tikv/tikv/tree/master/src/import/>
+
+The SQL→KV encoder is part of the TiDB project:
+<https://github.com/pingcap/tidb>
+
+The gRPC interface between `tidb-lightning` and `tikv-importer` is found in the
+`kvproto` repository: <https://github.com/pingcap/kvproto>