Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Interpreter: Print BlockInputStream #4911

Merged
merged 55 commits into from
May 25, 2022
Merged

Interpreter: Print BlockInputStream #4911

merged 55 commits into from
May 25, 2022

Conversation

ywqzzy
Copy link
Contributor

@ywqzzy ywqzzy commented May 17, 2022 •
edited
Loading

What problem does this PR solve?

Issue Number: ref #4609

Problem Summary:

What is changed and how it works?

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
  • No code

Side effects

  • Performance regression: Consumes more CPU
  • Performance regression: Consumes more Memory
  • Breaking backward compatibility

Documentation

  • Affects user behaviors
  • Contains syntax changes
  • Contains variable changes
  • Contains experimental features
  • Changes MySQL compatibility

Release note

None

@ywqzzy
init.
8e17622
@ywqzzy
fix.
c8fd369
@ywqzzy
Merge branch 'master' of https://github.com/pingcap/tiflash into run_…
4a29a27 
…interpreter_test
@ywqzzy
fix restoreConcurrency.
491dafd
@ywqzzy
some tests
fbd66bb
@ywqzzy
format.
b5e5614
@ywqzzy
fix build fail.
99830e3
@ywqzzy
format, more tests.
347bf8a
@ywqzzy
clean
a7d4602
@ywqzzy
refactor
9a7b988
@ywqzzy
getColumnsFromTableScan utils and clean code.
781489e
@ywqzzy
refactor gencolumn related functions.
283639e
@ywqzzy
remove useless cout.
7dbd5c5
@ywqzzy
all user choose stream size.
120a983
@ywqzzy
address comments.
0ce3668
@ywqzzy
rename.
02f3509
@ywqzzy
rename.
3dad372
@ywqzzy
address comments.
4910d32
@ywqzzy
clean
cc81990
@ywqzzy
address comments.
fc46cd5
@ywqzzy
address comments
b23f514
@ywqzzy
Merge branch 'master' of https://github.com/pingcap/tiflash into run_…
0216daa 
…interpreter_test
@ywqzzy
address comments.
927234c
@ywqzzy
format.
2639b7f
@ywqzzy
address comment.
eca8746
@ywqzzy
Merge branch 'master' of https://github.com/pingcap/tiflash into run_…
eb50f1e 
…interpreter_test
@ywqzzy
fix.
589263d
@ywqzzy
init.
caf6b73
@ywqzzy
update.
0874ad5
@ywqzzy
update.
dc82bcb
SeaRise
SeaRise reviewed May 25, 2022
View reviewed changes
Copy link
Contributor

@SeaRise SeaRise left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

other LGTM

dbms/src/DataStreams/FilterBlockInputStream.cpp Outdated Show resolved Hide resolved
dbms/src/Flash/Coprocessor/DAGQueryBlockInterpreter.h Outdated Show resolved Hide resolved
dbms/src/Flash/Coprocessor/DAGQueryBlockInterpreter.h Outdated Show resolved Hide resolved
dbms/src/Flash/Coprocessor/DAGQueryBlockInterpreter.h Outdated Show resolved Hide resolved
dbms/src/Flash/Coprocessor/DAGQueryBlockInterpreter.cpp Outdated Show resolved Hide resolved
dbms/src/Flash/Coprocessor/DAGQueryBlockInterpreter.cpp Outdated Show resolved Hide resolved
@ti-chi-bot ti-chi-bot added the status/LGT1 Indicates that a PR has LGTM 1. label May 25, 2022
@ywqzzy ywqzzy requested review from fuzhe1989 and yibin87 May 25, 2022 03:49
yibin87
yibin87 approved these changes May 25, 2022
View reviewed changes
Copy link
Contributor

@yibin87 yibin87 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ti-chi-bot ti-chi-bot added status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. labels May 25, 2022
@ywqzzy
Copy link
Contributor Author

ywqzzy commented May 25, 2022

/run-all-tests

1 similar comment
@ywqzzy
Copy link
Contributor Author

ywqzzy commented May 25, 2022

/run-all-tests

@sre-bot
Copy link
Collaborator

sre-bot commented May 25, 2022

Coverage for changed files

Filename                                                Regions    Missed Regions     Cover   Functions  Missed Functions  Executed       Lines      Missed Lines     Cover    Branches   Missed Branches     Cover
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
DataStreams/FilterBlockInputStream.cpp                       75                67    10.67%           4                 2    50.00%         122               105    13.93%          56                53     5.36%
DataStreams/HashJoinBuildBlockInputStream.cpp                 8                 5    37.50%           2                 1    50.00%          24                 8    66.67%           4                 3    25.00%
DataStreams/HashJoinBuildBlockInputStream.h                   3                 0   100.00%           3                 0   100.00%           6                 0   100.00%           0                 0         -
DataStreams/IBlockInputStream.cpp                            27                11    59.26%           5                 2    60.00%          64                17    73.44%          22                 8    63.64%
DataStreams/IBlockInputStream.h                              30                20    33.33%          17                 9    47.06%          50                38    24.00%          16                13    18.75%
DataStreams/LimitBlockInputStream.cpp                        25                23     8.00%           3                 1    66.67%          42                36    14.29%          16                16     0.00%
DataStreams/LimitBlockInputStream.h                           2                 0   100.00%           2                 0   100.00%           2                 0   100.00%           0                 0         -
DataStreams/MergeSortingBlockInputStream.cpp                114                45    60.53%          10                 0   100.00%         171                57    66.67%          76                36    52.63%
DataStreams/MergeSortingBlockInputStream.h                   11                 7    36.36%          11                 7    36.36%          11                 7    36.36%           0                 0         -
DataStreams/ParallelAggregatingBlockInputStream.cpp         113               101    10.62%          13                 9    30.77%         164               147    10.37%          72                67     6.94%
DataStreams/ParallelAggregatingBlockInputStream.h             6                 4    33.33%           6                 4    33.33%          14                12    14.29%           0                 0         -
DataStreams/PartialSortingBlockInputStream.cpp                2                 0   100.00%           2                 0   100.00%           8                 0   100.00%           0                 0         -
DataStreams/PartialSortingBlockInputStream.h                  6                 3    50.00%           6                 3    50.00%           8                 3    62.50%           0                 0         -
DataStreams/TiRemoteBlockInputStream.h                      117               117     0.00%          17                17     0.00%         164               164     0.00%          60                60     0.00%
Flash/Coprocessor/DAGContext.h                               37                12    67.57%          28                10    64.29%          83                30    63.86%          12                 6    50.00%
Flash/Coprocessor/DAGQueryBlockInterpreter.cpp              229                80    65.07%          38                 4    89.47%         574               127    77.87%         138                55    60.14%
Flash/Coprocessor/DAGQueryBlockInterpreter.h                  2                 0   100.00%           2                 0   100.00%           2                 0   100.00%           0                 0         -
Flash/Coprocessor/DAGStorageInterpreter.cpp                 424               424     0.00%          32                32     0.00%         870               870     0.00%         282               282     0.00%
Flash/Coprocessor/InterpreterDAG.cpp                         23                 8    65.22%           5                 0   100.00%          60                10    83.33%          16                 8    50.00%
Flash/Coprocessor/InterpreterUtils.cpp                       32                 8    75.00%           4                 0   100.00%          55                17    69.09%          24                10    58.33%
Flash/tests/gtest_interpreter.cpp                            82                38    53.66%           3                 0   100.00%         339                 0   100.00%           8                 8     0.00%
Interpreters/ExpressionActions.cpp                          439               260    40.77%          34                12    64.71%         787               441    43.96%         392               253    35.46%
Interpreters/ExpressionActions.h                             19                 2    89.47%          10                 0   100.00%          34                 2    94.12%          10                 2    80.00%
Interpreters/executeQuery.cpp                               199               133    33.17%          15                 8    46.67%         409               213    47.92%         136               112    17.65%
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL                                                      2025              1368    32.44%         272               121    55.51%        4063              2304    43.29%        1340               992    25.97%

Coverage summary

Functions  MissedFunctions  Executed  Lines   MissedLines  Cover
18262      9775             46.47%    204342  97942        52.07%

full coverage report (for internal network access only)

@ywqzzy
Copy link
Contributor Author

ywqzzy commented May 25, 2022

/run-all-tests

@ywqzzy
Copy link
Contributor Author

ywqzzy commented May 25, 2022

/run-unit-test

@sre-bot
Copy link
Collaborator

sre-bot commented May 25, 2022

Coverage for changed files

Filename                                                Regions    Missed Regions     Cover   Functions  Missed Functions  Executed       Lines      Missed Lines     Cover    Branches   Missed Branches     Cover
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
DataStreams/FilterBlockInputStream.cpp                       75                67    10.67%           4                 2    50.00%         122               105    13.93%          56                53     5.36%
DataStreams/HashJoinBuildBlockInputStream.cpp                 8                 5    37.50%           2                 1    50.00%          28                 8    71.43%           4                 3    25.00%
DataStreams/HashJoinBuildBlockInputStream.h                   3                 0   100.00%           3                 0   100.00%           6                 0   100.00%           0                 0         -
DataStreams/IBlockInputStream.cpp                            27                11    59.26%           5                 2    60.00%          64                17    73.44%          22                 8    63.64%
DataStreams/IBlockInputStream.h                              30                20    33.33%          17                 9    47.06%          50                38    24.00%          16                13    18.75%
DataStreams/LimitBlockInputStream.cpp                        25                23     8.00%           3                 1    66.67%          42                36    14.29%          16                16     0.00%
DataStreams/LimitBlockInputStream.h                           2                 0   100.00%           2                 0   100.00%           2                 0   100.00%           0                 0         -
DataStreams/MergeSortingBlockInputStream.cpp                114                45    60.53%          10                 0   100.00%         171                57    66.67%          76                36    52.63%
DataStreams/MergeSortingBlockInputStream.h                   11                 7    36.36%          11                 7    36.36%          11                 7    36.36%           0                 0         -
DataStreams/ParallelAggregatingBlockInputStream.cpp         113               101    10.62%          13                 9    30.77%         164               147    10.37%          72                67     6.94%
DataStreams/ParallelAggregatingBlockInputStream.h             6                 4    33.33%           6                 4    33.33%          14                12    14.29%           0                 0         -
DataStreams/PartialSortingBlockInputStream.cpp                2                 0   100.00%           2                 0   100.00%           8                 0   100.00%           0                 0         -
DataStreams/PartialSortingBlockInputStream.h                  6                 3    50.00%           6                 3    50.00%           8                 3    62.50%           0                 0         -
DataStreams/TiRemoteBlockInputStream.h                      117               117     0.00%          17                17     0.00%         164               164     0.00%          60                60     0.00%
Flash/Coprocessor/DAGContext.h                               37                11    70.27%          28                 9    67.86%          83                29    65.06%          12                 6    50.00%
Flash/Coprocessor/DAGQueryBlockInterpreter.cpp              229                80    65.07%          38                 4    89.47%         574               127    77.87%         138                55    60.14%
Flash/Coprocessor/DAGQueryBlockInterpreter.h                  2                 0   100.00%           2                 0   100.00%           2                 0   100.00%           0                 0         -
Flash/Coprocessor/DAGStorageInterpreter.cpp                 424               424     0.00%          32                32     0.00%         870               870     0.00%         282               282     0.00%
Flash/Coprocessor/InterpreterDAG.cpp                         25                 4    84.00%           5                 0   100.00%          60                 9    85.00%          18                 7    61.11%
Flash/Coprocessor/InterpreterUtils.cpp                       32                 8    75.00%           4                 0   100.00%          55                17    69.09%          24                10    58.33%
Flash/tests/gtest_interpreter.cpp                            82                38    53.66%           3                 0   100.00%         339                 0   100.00%           8                 8     0.00%
Interpreters/ExpressionActions.cpp                          439               260    40.77%          34                12    64.71%         787               441    43.96%         392               253    35.46%
Interpreters/ExpressionActions.h                             19                 2    89.47%          10                 0   100.00%          34                 2    94.12%          10                 2    80.00%
Interpreters/executeQuery.cpp                               199               133    33.17%          15                 8    46.67%         409               213    47.92%         136               112    17.65%
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL                                                      2027              1363    32.76%         272               120    55.88%        4067              2302    43.40%        1342               991    26.15%

Coverage summary

Functions  MissedFunctions  Executed  Lines   MissedLines  Cover
18265      9778             46.47%    204517  97932        52.12%

full coverage report (for internal network access only)

@ywqzzy
Copy link
Contributor Author

ywqzzy commented May 25, 2022

/merge

@ti-chi-bot
Copy link
Member

@ywqzzy: It seems you want to merge this PR, I will help you trigger all the tests:

/run-all-tests

You only need to trigger /merge once, and if the CI test fails, you just re-trigger the test that failed and the bot will merge the PR for you after the CI passes.

If you have any questions about the PR merge process, please refer to pr process.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

@ti-chi-bot
Copy link
Member

This pull request has been accepted and is ready to merge.

Commit hash: e8fda6f

@ti-chi-bot ti-chi-bot added the status/can-merge Indicates a PR has been approved by a committer. label May 25, 2022
@ywqzzy
Copy link
Contributor Author

ywqzzy commented May 25, 2022

/hold

@ti-chi-bot ti-chi-bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label May 25, 2022
@ywqzzy
Copy link
Contributor Author

ywqzzy commented May 25, 2022 •
edited
Loading

/run-unit-test

@sre-bot
Copy link
Collaborator

sre-bot commented May 25, 2022

Coverage for changed files

Filename                                                Regions    Missed Regions     Cover   Functions  Missed Functions  Executed       Lines      Missed Lines     Cover    Branches   Missed Branches     Cover
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
DataStreams/FilterBlockInputStream.cpp                       75                67    10.67%           4                 2    50.00%         122               105    13.93%          56                53     5.36%
DataStreams/HashJoinBuildBlockInputStream.cpp                 8                 5    37.50%           2                 1    50.00%          28                 8    71.43%           4                 3    25.00%
DataStreams/HashJoinBuildBlockInputStream.h                   3                 0   100.00%           3                 0   100.00%           6                 0   100.00%           0                 0         -
DataStreams/IBlockInputStream.cpp                            27                11    59.26%           5                 2    60.00%          64                17    73.44%          22                 8    63.64%
DataStreams/IBlockInputStream.h                              30                20    33.33%          17                 9    47.06%          50                38    24.00%          16                13    18.75%
DataStreams/LimitBlockInputStream.cpp                        25                23     8.00%           3                 1    66.67%          42                36    14.29%          16                16     0.00%
DataStreams/LimitBlockInputStream.h                           2                 0   100.00%           2                 0   100.00%           2                 0   100.00%           0                 0         -
DataStreams/MergeSortingBlockInputStream.cpp                114                45    60.53%          10                 0   100.00%         171                57    66.67%          76                36    52.63%
DataStreams/MergeSortingBlockInputStream.h                   11                 7    36.36%          11                 7    36.36%          11                 7    36.36%           0                 0         -
DataStreams/ParallelAggregatingBlockInputStream.cpp         113               101    10.62%          13                 9    30.77%         164               147    10.37%          72                67     6.94%
DataStreams/ParallelAggregatingBlockInputStream.h             6                 4    33.33%           6                 4    33.33%          14                12    14.29%           0                 0         -
DataStreams/PartialSortingBlockInputStream.cpp                2                 0   100.00%           2                 0   100.00%           8                 0   100.00%           0                 0         -
DataStreams/PartialSortingBlockInputStream.h                  6                 3    50.00%           6                 3    50.00%           8                 3    62.50%           0                 0         -
DataStreams/TiRemoteBlockInputStream.h                      117               117     0.00%          17                17     0.00%         164               164     0.00%          60                60     0.00%
Flash/Coprocessor/DAGContext.h                               37                11    70.27%          28                 9    67.86%          83                29    65.06%          12                 6    50.00%
Flash/Coprocessor/DAGQueryBlockInterpreter.cpp              229                80    65.07%          38                 4    89.47%         574               127    77.87%         138                55    60.14%
Flash/Coprocessor/DAGQueryBlockInterpreter.h                  2                 0   100.00%           2                 0   100.00%           2                 0   100.00%           0                 0         -
Flash/Coprocessor/DAGStorageInterpreter.cpp                 424               424     0.00%          32                32     0.00%         870               870     0.00%         282               282     0.00%
Flash/Coprocessor/InterpreterDAG.cpp                         25                 4    84.00%           5                 0   100.00%          60                 9    85.00%          18                 7    61.11%
Flash/Coprocessor/InterpreterUtils.cpp                       32                 8    75.00%           4                 0   100.00%          55                17    69.09%          24                10    58.33%
Flash/tests/gtest_interpreter.cpp                            82                38    53.66%           3                 0   100.00%         339                 0   100.00%           8                 8     0.00%
Interpreters/ExpressionActions.cpp                          439               260    40.77%          34                12    64.71%         787               441    43.96%         392               253    35.46%
Interpreters/ExpressionActions.h                             19                 2    89.47%          10                 0   100.00%          34                 2    94.12%          10                 2    80.00%
Interpreters/executeQuery.cpp                               199               133    33.17%          15                 8    46.67%         409               213    47.92%         136               112    17.65%
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL                                                      2027              1363    32.76%         272               120    55.88%        4067              2302    43.40%        1342               991    26.15%

Coverage summary

Functions  MissedFunctions  Executed  Lines   MissedLines  Cover
18265      9778             46.47%    204517  97943        52.11%

full coverage report (for internal network access only)

@ywqzzy
Copy link
Contributor Author

ywqzzy commented May 25, 2022

/unhold

@ti-chi-bot ti-chi-bot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label May 25, 2022
@ti-chi-bot ti-chi-bot merged commit 35c8ea1 into pingcap:master May 25, 2022
@ywqzzy ywqzzy mentioned this pull request May 27, 2022
Closed
54 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@yibin87 yibin87 yibin87 approved these changes

@SeaRise SeaRise SeaRise approved these changes

@fuzhe1989 fuzhe1989 Awaiting requested review from fuzhe1989

Assignees
No one assigned
Labels
release-note-none Denotes a PR that doesn't merit a release note. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. status/can-merge Indicates a PR has been approved by a committer. status/LGT2 Indicates that a PR has LGTM 2.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@ywqzzy @ti-chi-bot @sre-bot @fuzhe1989 @yibin87 @SeaRise