tidb-lightning tidb backend import tableA but report Error 1146: Table 'tableB' doesn't exist

[haiying-gao]

Bug Report

during tidb-lightning import another table dropped by someone else.

1. Minimal reproduce step (Required)

2. What did you expect to see? (Required)

import successful !

3. What did you see instead (Required)

tidb-lighting report another table not exist.

4. What is your TiDB version? (Required)

tidb-lightning v6.5.2

[haiying-gao]

i think the error was introduced by FetchRemoteTableModels funcation.

[dsdashun]

The error occurs when the user utilizes the 'tidb' backend to import data. Specifically, at the start of the import process, when Lightning is creating tables, the user drops an irrelevant table in the same database as the data source.

The specific reason for the error message is as follows:
At the beginning of the Lightning task, a list of all the table information that needs to be imported in the data source will be obtained based on the filter and file pattern. This list only contains the table filtered out by the filter.

Next, Lightning will iterate over all the databases involved in the data to be imported before and after executing the statement to restore the corresponding table structure. Then, each database will obtain all the existing table structures to form a map. The purpose of constructing this map is to:

1. Before creating the table, iterate through the list of tables to be imported, search the map for whether the table name to be imported already exists in the existing database. If it already exists, then there is no need to create the table.
2. After creating the table, iterate through the list of tables to be imported, search the corresponding table structure information in the map based on the table name to be imported, for subsequent use.

Therefore, the full table structure acquisition strategy is used when constructing this map, and then further filtered based on the list of tables to be imported.

The user's error occurred during the stage of restoring the table structure. At that time, Lightning was constructing this map, and an error occurred when executing SHOW TABLE <table_name> NEXT_ROW_ID to obtain the row ID information of the table, which had been deleted.

Lightning can consider optimizing this process. For example, when encountering this type of error, it can continue to construct the remaining map elements. Because the information in this map, minus one irrelevant table, will not affect the import.