Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

executor: reuse chunk for GetMatchedRowsAndPtrs calls (#48090) #48134

Conversation

lcwangchao
Copy link
Collaborator

What problem does this PR solve?

cherry-pick of #48090

Issue Number: close #48082
Problem Summary:

What is changed and how it works?

Reuse chunk for GetMatchedRowsAndPtrs.
Add a new benchmark case.

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
    BenchmarkHashJoinExec
Old:
 go test -benchmem -run=^$ -bench ^BenchmarkHashJoinExec$ github.com/pingcap/tidb/pkg/executor
goos: linux
goarch: amd64
pkg: github.com/pingcap/tidb/pkg/executor
cpu: 13th Gen Intel(R) Core(TM) i9-13900KF
BenchmarkHashJoinExec/(rows:100000,_cols:[bigint(20)_var_string(5)],_concurency:4,_joinKeyIdx:_[0_1],_disk:false)-32                   2         625585277 ns/op        2428636920 B/op   512892 allocs/op
BenchmarkHashJoinExec/(rows:100000,_cols:[bigint(20)_var_string(5)],_concurency:4,_joinKeyIdx:_[0],_disk:false)-32                    19          64306968 ns/op        278193261 B/op    112877 allocs/op
BenchmarkHashJoinExec/(rows:100000,_cols:[bigint(20)_var_string(5)],_concurency:4,_joinKeyIdx:_[0],_disk:true)-32                      2         510499472 ns/op        3491117756 B/op  2213412 allocs/op
BenchmarkHashJoinExec/(rows:5,_cols:[bigint(20)_double],_concurency:4,_joinKeyIdx:_[0],_disk:false)-32                             26020             48058 ns/op           87853 B/op        934 allocs/op
BenchmarkHashJoinExec/(rows:100000,_cols:[bigint(20)_double],_concurency:4,_joinKeyIdx:_[0_1],_disk:false)-32                         79          16758415 ns/op        15902526 B/op     512736 allocs/op
BenchmarkHashJoinExec/(rows:100000,_cols:[bigint(20)_double],_concurency:4,_joinKeyIdx:_[0],_disk:false)-32                           92          15112195 ns/op        12524965 B/op     112666 allocs/op
BenchmarkHashJoinExec/(rows:100000,_cols:[bigint(20)],_concurency:4,_joinKeyIdx:_[0],_disk:true)-32                                    9         123780841 ns/op        894195201 B/op   1714292 allocs/op


New:
go test -benchmem -run=^$ -bench ^BenchmarkHashJoinExec$ github.com/pingcap/tidb/pkg/executor                      
goos: linux
goarch: amd64
pkg: github.com/pingcap/tidb/pkg/executor
cpu: 13th Gen Intel(R) Core(TM) i9-13900KF
BenchmarkHashJoinExec/(rows:100000,_cols:[bigint(20)_var_string(5)],_concurency:4,_joinKeyIdx:_[0_1],_disk:false)-32                   2         627044000 ns/op        2428678160 B/op   512971 allocs/op
BenchmarkHashJoinExec/(rows:100000,_cols:[bigint(20)_var_string(5)],_concurency:4,_joinKeyIdx:_[0],_disk:false)-32                    18          63483060 ns/op        278192079 B/op    112888 allocs/op
BenchmarkHashJoinExec/(rows:100000,_cols:[bigint(20)_var_string(5)],_concurency:4,_joinKeyIdx:_[0],_disk:true)-32                      2         502814122 ns/op        3639970448 B/op  1216829 allocs/op
BenchmarkHashJoinExec/(rows:5,_cols:[bigint(20)_double],_concurency:4,_joinKeyIdx:_[0],_disk:false)-32                             28047             48197 ns/op           87847 B/op        934 allocs/op
BenchmarkHashJoinExec/(rows:100000,_cols:[bigint(20)_double],_concurency:4,_joinKeyIdx:_[0_1],_disk:false)-32                         70          16853512 ns/op        15902144 B/op     512739 allocs/op
BenchmarkHashJoinExec/(rows:100000,_cols:[bigint(20)_double],_concurency:4,_joinKeyIdx:_[0],_disk:false)-32                           82          14872368 ns/op        12524746 B/op     112660 allocs/op
BenchmarkHashJoinExec/(rows:100000,_cols:[bigint(20)],_concurency:4,_joinKeyIdx:_[0],_disk:true)-32                                   22          53258487 ns/op        40282174 B/op    1113179 allocs/op

Profile for the new case: left is the master, right is this pr.
image

  • No need to test
    • I checked and no code files have been changed.

Side effects

  • Performance regression: Consumes more CPU
  • Performance regression: Consumes more Memory
  • Breaking backward compatibility

Documentation

  • Affects user behaviors
  • Contains syntax changes
  • Contains variable changes
  • Contains experimental features
  • Changes MySQL compatibility

Release note

Please refer to Release Notes Language Style Guide to write a quality release note.

None

@ti-chi-bot ti-chi-bot bot added release-note-none Denotes a PR that doesn't merit a release note. size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Oct 31, 2023
@ti-chi-bot ti-chi-bot bot added needs-1-more-lgtm Indicates a PR needs 1 more LGTM. approved labels Oct 31, 2023
Copy link
Member

@YangKeao YangKeao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ti-chi-bot
Copy link

ti-chi-bot bot commented Oct 31, 2023

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: wshwsh12, YangKeao

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ti-chi-bot ti-chi-bot bot added lgtm and removed needs-1-more-lgtm Indicates a PR needs 1 more LGTM. labels Oct 31, 2023
@ti-chi-bot
Copy link

ti-chi-bot bot commented Oct 31, 2023

[LGTM Timeline notifier]

Timeline:

  • 2023-10-31 08:22:19.407833173 +0000 UTC m=+2941336.994943326: ☑️ agreed by wshwsh12.
  • 2023-10-31 08:39:01.750856713 +0000 UTC m=+2942339.337966845: ☑️ agreed by YangKeao.

@codecov
Copy link

codecov bot commented Oct 31, 2023

We're currently processing your upload. This comment will be updated when the results are available.

@ti-chi-bot ti-chi-bot bot merged commit 795585b into pingcap:release-6.5-20230706-v6.5.3 Oct 31, 2023
8 checks passed
@lcwangchao lcwangchao deleted the release-6.5-20230706-v6.5.3-cherry-pick-40890 branch October 31, 2023 08:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved lgtm release-note-none Denotes a PR that doesn't merit a release note. size/S Denotes a PR that changes 10-29 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants