Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Improve the performance of adding index #35983

Closed
20 tasks done
Benjamin2037 opened this issue Jul 6, 2022 · 1 comment
Closed
20 tasks done

Improve the performance of adding index #35983

Benjamin2037 opened this issue Jul 6, 2022 · 1 comment
Assignees
@tangenta
Labels
sig/sql-infra SIG: SQL Infra type/feature-request Categorizes issue or PR as related to a new feature. type/new-feature

Comments

@Benjamin2037
Copy link
Collaborator

Benjamin2037 commented Jul 6, 2022 •
edited by tangenta
Loading

Feature Request

Is your feature request related to a problem? Please describe:

Adding indexes is a high-frequency operation in the proof-of-concept scenarios. However, for the large tables, it is a time-consuming process that may affect the user experience. There is still room for improvement in TiDB. For the similar requests, see #12963 and #14454.

Describe the feature you'd like:

We would like to achieve a better performance for adding index in TiDB, and reduce the impact on DML transactions for foreground traffic.

Stage 1: support the ingest method (3X)

Feature branch: #36767

Design:

Refactoring:

Config:

Framework:

Tests:

Stage 2: optimize add index reorganization (10X)

Experimental branch: #38110
@Benjamin2037 Benjamin2037 added the type/feature-request Categorizes issue or PR as related to a new feature. label Jul 6, 2022
@Benjamin2037 Benjamin2037 self-assigned this Jul 6, 2022
@Benjamin2037 Benjamin2037 mentioned this issue Jul 6, 2022
Closed
12 tasks
@Benjamin2037 Benjamin2037 changed the title Add index Acceleration DDL:Add index Acceleration Jul 6, 2022
This was referenced Jul 11, 2022
Merged
Merged
@Benjamin2037
Copy link
Collaborator Author

Track issue：
#36165 dead lock problem

@Benjamin2037 Benjamin2037 mentioned this issue Jul 20, 2022
Merged
Benjamin2037 added a commit to Benjamin2037/tidb that referenced this issue Jul 20, 2022
@Benjamin2037
Design doc for add index acceleration(pingcap#35983)
6c7128a
Benjamin2037 added a commit to Benjamin2037/tidb that referenced this issue Jul 25, 2022
@Benjamin2037
Preparation for add index acceleration, handle 'import cycle'(pingcap…
705793d 
…#35983)
Benjamin2037 added a commit to Benjamin2037/tidb that referenced this issue Jul 25, 2022
@Benjamin2037
expend interface of lightning for add index acceleration. (pingcap#35983
40f7233 
)
Benjamin2037 added a commit to Benjamin2037/tidb that referenced this issue Jul 25, 2022
@Benjamin2037
Add index acceleration:
8673970 
Add sysvars and config parameter.
Change the system variables name(pingcap#35983)
tidb_ddl_enable_fast_reorg
tidb_ddl_disk_quota
fast-reorg-local-path.
This was referenced Sep 20, 2022
Closed
Closed
@tangenta tangenta reopened this Oct 26, 2022
@tangenta tangenta assigned tangenta and unassigned Benjamin2037 Oct 26, 2022
This was referenced Oct 26, 2022
Merged
Merged
ti-chi-bot pushed a commit that referenced this issue Oct 28, 2022
@tangenta
*: show backfill type in admin show ddl jobs (#38733)
6d6e9c4 
ref #35983
ti-chi-bot pushed a commit that referenced this issue Nov 10, 2022
@tangenta
ddl: improve the reorg task scheduling (#38646)
cfbe3c9 
ref #35983
@tangenta tangenta mentioned this issue Nov 16, 2022
Merged
12 tasks
ti-chi-bot pushed a commit that referenced this issue Nov 18, 2022
@tangenta
ddl: support read records with copr for adding index (#39191)
e9253f7 
ref #35983
@tangenta tangenta mentioned this issue Nov 18, 2022
Merged
12 tasks
@Benjamin2037 Benjamin2037 mentioned this issue Nov 21, 2022
Merged
12 tasks
ti-chi-bot pushed a commit that referenced this issue Nov 23, 2022
@tangenta
ddl: make read and write async during adding index (#39249)
c9531d4 
ref #35983
ti-chi-bot pushed a commit that referenced this issue Nov 23, 2022
@Benjamin2037
ddl: Add index fast path to be a default add index path (#39267)
9784615 
ref #35983
@tangenta tangenta mentioned this issue Nov 23, 2022
Merged
12 tasks
ti-chi-bot pushed a commit that referenced this issue Nov 24, 2022
@tangenta
ddl: support read generated columns with copr for adding index (#39345)
7611a03 
ref #35983
@lance6716 lance6716 mentioned this issue Apr 7, 2023
Merged
12 tasks
ti-chi-bot pushed a commit that referenced this issue Apr 7, 2023
@lance6716
ddl: set lightning cron config to real behaviour (#42855)
f0427c3 
ref #35983
@Ivy-YinSu Ivy-YinSu mentioned this issue Aug 5, 2024
Closed
11 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@tangenta tangenta

Labels
sig/sql-infra SIG: SQL Infra type/feature-request Categorizes issue or PR as related to a new feature. type/new-feature
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@Benjamin2037 @tangenta