planner,executor: fix 'select ...(join on partition table) for update' panic

[tiancaiamao]

What problem does this PR solve?

Issue Number: close #20028

Problem Summary:

'select ... for update' need the table ID to construct the lock key.
When this SQL works on a partitioned table, the table ID should be partition ID.

So how to get that partition ID?
In http://github.com/pingcap/tidb/pull/14921 I try to avoid column pruning of the partition columns,
so we can get the partition column data, and then use it to calculate the partition to get the ID.

That fix works on a single table, but not the joining of tables.

What is changed and how it works?

What's Changed:

In the logical plan phase, if buildSelectLock find itself works on partitioned tables,
change the schema add an extra partition ID column to the DataSource.

Some tiny changes in the column pruning step to get the correct schema.

In the TableReader executor, fill the extra partition ID column in the chunk row.

In the SelectLockExec, use the partition ID from the chunk row, to construct the correct lock key.

How it Works:

Let the table reader return an extra column with partition ID, so the
SelectLockExec can use that partition ID to construct the lock key.

Related changes

• Need to cherry-pick to the release branch

Check List

Tests

• Unit test

Release note

• Fix panic when 'select ... for update' works on a join operation and the join uses partition table

[tiancaiamao]

PTAL @SunRunAway @eurekaka

[tiancaiamao]

Adjusted according to the advice from @eurekaka

• add ExtraPidCol in buildDataSource and change LogicalLock to a logicalSchemaProducer
• add ExtraPidCol into LogicalLock's schema, and let column pruning handle the rest of the thing

Some extra information is still necessary for SelectLock to know the mapping from table ID to the partition column index.

[tiancaiamao]

/run-check_dev_2

[tiancaiamao]

I have reverted the code to the previous implementation.
Add extra partition ID columns in table reader and depends on column pruning to prune them does not always work.

For example, in the physical optimizing phase, if there is only one KV entry in the range, DataSource will be converted to PointGet, but the schema is not correct, the extra PID column is not pruned.
And another case is Delete, delete may not handle the projection of the used columns, so column pruning will not prune the extra partition ID column for Delete, and this lead to a panic in the binlog encoding (the schema mismatch with the chunk data)

The current implementation only adds the partition ID column when there is a SelectLock on the partitioned table, so the change is minimal and no other parts are affected.

PTAL @eurekaka

[eurekaka]

LGTM

[ti-srebot]

@eurekaka, Thanks for your review. The bot only counts LGTMs from Reviewers and higher roles, but you're still welcome to leave your comments. See the corresponding SIG page for more information. Related SIG: execution(slack).

[XuHuaiyu]

Check the column.ID == model.ExtraPidColID is better

[tiancaiamao]

Check the column to be ExtraPidColID can get the column index in the schema, but can't build the mapping of

table ID => partition column index in the schema

Here we take each partition column (v.ExtraPIDInfo.Columns[i]) and find its index in the schema (offset), to build the
mapping tblID(v.ExtraPIDInfo.TblIDs[i]) => offset

[ti-chi-bot]

This pull request has been accepted and is ready to merge.

Commit hash: 98872f4

[ti-srebot]

cherry pick to release-4.0 in PR #25501

[ti-srebot]

cherry pick to release-5.1 in PR #25502

[ti-srebot]

cherry pick to release-5.0 in PR #25845

[tiancaiamao]

This PR is terrible... it's a nightmare...
Just see how many bugs it has introduced!