Test: Mock Input columns for operator tests #5041

ywqzzy · 2022-06-01T07:01:40Z

What problem does this PR solve?

Issue Number: ref #4609

Problem Summary:

What is changed and how it works?

Check List

Tests

Unit test
Integration test
Manual test (add detailed scripts or steps below)
No code

Side effects

Performance regression: Consumes more CPU
Performance regression: Consumes more Memory
Breaking backward compatibility

Documentation

Release note

None

ti-chi-bot · 2022-06-01T07:01:41Z

[REVIEW NOTIFICATION]

This pull request has been approved by:

SeaRise
mengxin9014

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

ywqzzy · 2022-06-01T07:25:47Z

/cc @xzhangxian1008

ti-chi-bot · 2022-06-01T07:25:48Z

@ywqzzy: GitHub didn't allow me to request PR reviews from the following users: xzhangxian1008.

Note that only pingcap members and repo collaborators can review this PR, and authors cannot review their own PRs.

In response to this:

/cc @xzhangxian1008

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

dbms/src/Flash/Coprocessor/DAGContext.h

dbms/src/Flash/Coprocessor/DAGQueryBlockInterpreter.cpp

…_stream_inputblock

dbms/src/DataStreams/CreatingSetsBlockInputStream.cpp

dbms/src/Flash/Coprocessor/DAGContext.h

dbms/src/TestUtils/InterpreterTestUtils.cpp

…_stream_inputblock

sre-bot · 2022-06-07T07:07:41Z

Coverage for changed files

Filename                                            Regions    Missed Regions     Cover   Functions  Missed Functions  Executed       Lines      Missed Lines     Cover    Branches   Missed Branches     Cover
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
DataStreams/MockExchangeReceiverInputStream.cpp          17                 0   100.00%           4                 0   100.00%          40                 1    97.50%          12                 1    91.67%
DataStreams/MockExchangeReceiverInputStream.h             2                 0   100.00%           2                 0   100.00%           4                 0   100.00%           0                 0         -
Flash/Coprocessor/DAGContext.cpp                        104                60    42.31%          28                13    53.57%         235               120    48.94%          74                54    27.03%
Flash/Coprocessor/DAGContext.h                           39                11    71.79%          30                 9    70.00%          85                29    65.88%          12                 6    50.00%
Flash/Coprocessor/DAGQueryBlockInterpreter.cpp          261                87    66.67%          39                 4    89.74%         620               127    79.52%         164                60    63.41%
Flash/Coprocessor/DAGQueryBlockInterpreter.h              2                 0   100.00%           2                 0   100.00%           2                 0   100.00%           0                 0         -
Flash/Coprocessor/InterpreterDAG.cpp                     27                 4    85.19%           5                 0   100.00%          60                 8    86.67%          20                 7    65.00%
Flash/tests/gtest_executor.cpp                          103                76    26.21%           7                 0   100.00%         189                 0   100.00%          16                16     0.00%
Flash/tests/gtest_interpreter.cpp                        82                38    53.66%           3                 0   100.00%         339                 0   100.00%           8                 8     0.00%
TestUtils/InterpreterTestUtils.cpp                       67                 9    86.57%          10                 1    90.00%          84                 6    92.86%          26                 7    73.08%
TestUtils/InterpreterTestUtils.h                          2                 0   100.00%           2                 0   100.00%           5                 0   100.00%           0                 0         -
TestUtils/mockExecutor.cpp                               70                14    80.00%          43                12    72.09%         266                57    78.57%          32                 2    93.75%
TestUtils/mockExecutor.h                                  6                 1    83.33%           6                 1    83.33%          15                 3    80.00%           0                 0         -
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL                                                   782               300    61.64%         181                40    77.90%        1944               351    81.94%         364               161    55.77%

Coverage summary

Functions  MissedFunctions  Executed  Lines   MissedLines  Cover
18308      9642             47.33%    205331  96604        52.95%

full coverage report (for internal network access only)

dbms/src/Flash/Coprocessor/InterpreterDAG.cpp

dbms/src/Flash/Coprocessor/MockSourceStream.h

dbms/src/TestUtils/InterpreterTestUtils.h

dbms/src/TestUtils/mockExecutor.h

SeaRise

excellent job!
LGTM

dbms/src/Flash/tests/gtest_executor.cpp

dbms/src/TestUtils/ExecutorTestUtils.cpp

dbms/src/TestUtils/mockExecutor.cpp

mengxin9014 · 2022-06-07T12:06:02Z

dbms/src/TestUtils/ExecutorTestUtils.cpp

+    try
+    {
+        DB::registerFunctions();
+        DB::registerAggregateFunctions();


Add registerWindowFunctions?

The code here may have potential bug in unit test if we run function related test and MockExecutor test simultaneously(#5081), I wonder if I can fix it in another pr.

dbms/src/TestUtils/mockExecutor.cpp

dbms/src/TestUtils/ExecutorTestUtils.cpp

ywqzzy · 2022-06-08T02:31:46Z

/merge

ti-chi-bot · 2022-06-08T02:31:47Z

@ywqzzy: It seems you want to merge this PR, I will help you trigger all the tests:

/run-all-tests

You only need to trigger /merge once, and if the CI test fails, you just re-trigger the test that failed and the bot will merge the PR for you after the CI passes.

If you have any questions about the PR merge process, please refer to pr process.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

ti-chi-bot · 2022-06-08T02:31:48Z

This pull request has been accepted and is ready to merge.

Commit hash: 38e6a7a

sre-bot · 2022-06-08T02:56:53Z

Coverage for changed files

Filename                                            Regions    Missed Regions     Cover   Functions  Missed Functions  Executed       Lines      Missed Lines     Cover    Branches   Missed Branches     Cover
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
DataStreams/MockExchangeReceiverInputStream.cpp          17                 0   100.00%           4                 0   100.00%          40                 1    97.50%          12                 1    91.67%
DataStreams/MockExchangeReceiverInputStream.h             2                 0   100.00%           2                 0   100.00%           4                 0   100.00%           0                 0         -
Flash/Coprocessor/DAGContext.cpp                        104                60    42.31%          28                13    53.57%         235               120    48.94%          74                54    27.03%
Flash/Coprocessor/DAGContext.h                           39                11    71.79%          30                 9    70.00%          85                29    65.88%          12                 6    50.00%
Flash/Coprocessor/DAGQueryBlockInterpreter.cpp          239                80    66.53%          38                 4    89.47%         592               127    78.55%         150                57    62.00%
Flash/Coprocessor/InterpreterDAG.cpp                     28                 7    75.00%           5                 0   100.00%          62                11    82.26%          20                 8    60.00%
Flash/Coprocessor/MockSourceStream.h                     33                14    57.58%           1                 0   100.00%          34                 0   100.00%          18                 6    66.67%
Flash/tests/gtest_executor.cpp                          101                76    24.75%           5                 0   100.00%         182                 0   100.00%          16                16     0.00%
Flash/tests/gtest_interpreter.cpp                        82                38    53.66%           3                 0   100.00%         339                 0   100.00%           8                 8     0.00%
TestUtils/ExecutorTestUtils.cpp                          67                 9    86.57%          10                 1    90.00%          83                 6    92.77%          26                 7    73.08%
TestUtils/ExecutorTestUtils.h                             5                 0   100.00%           5                 0   100.00%          14                 0   100.00%           0                 0         -
TestUtils/mockExecutor.cpp                               70                14    80.00%          43                12    72.09%         266                57    78.57%          32                 2    93.75%
TestUtils/mockExecutor.h                                  6                 1    83.33%           6                 1    83.33%          15                 3    80.00%           0                 0         -
TestUtils/tests/gtest_mock_executors.cpp                238               171    28.15%          10                 0   100.00%         204                 0   100.00%          36                36     0.00%
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL                                                  1031               481    53.35%         190                40    78.95%        2155               354    83.57%         404               201    50.25%

Coverage summary

Functions  MissedFunctions  Executed  Lines   MissedLines  Cover
18311      9641             47.35%    205470  96552        53.01%

full coverage report (for internal network access only)

init.

8e3205a

SeaRise self-requested a review June 1, 2022 07:11

ywqzzy mentioned this pull request Jun 1, 2022

Test: Mock window function, refactor window function tests #5021

Merged

12 tasks

more apis

a54e510

ti-chi-bot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Jun 1, 2022

ywqzzy changed the title ~~[WIP]Test: Mock Input columns for operator tests(poc)~~ Test: Mock Input columns for operator tests Jun 1, 2022

ti-chi-bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jun 1, 2022

mock exchange receiver data.

27f98d2

ywqzzy mentioned this pull request Jun 1, 2022

Interpreter & Executor & MPP unit test framework #4609

Closed

54 tasks

SeaRise reviewed Jun 1, 2022

View reviewed changes

dbms/src/Flash/Coprocessor/DAGContext.h Outdated Show resolved Hide resolved

dbms/src/Flash/Coprocessor/DAGQueryBlockInterpreter.cpp Outdated Show resolved Hide resolved

ywqzzy changed the title ~~Test: Mock Input columns for operator tests~~ [WIP] Test: Mock Input columns for operator tests Jun 1, 2022

ti-chi-bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jun 1, 2022

ywqzzy added 5 commits June 2, 2022 09:55

support multiple source, bug not fixed.

b2bf618

join done.

75820cf

remove comments.

803f831

format.

8072210

Merge branch 'master' of https://github.com/pingcap/tiflash into mock…

200d7b5

…_stream_inputblock

SeaRise reviewed Jun 2, 2022

View reviewed changes

dbms/src/DataStreams/CreatingSetsBlockInputStream.cpp Outdated Show resolved Hide resolved

dbms/src/Flash/Coprocessor/DAGContext.h Outdated Show resolved Hide resolved

ywqzzy added 2 commits June 2, 2022 14:49

update.

071fcb0

little refine.

43db3e9

SeaRise reviewed Jun 2, 2022

View reviewed changes

dbms/src/Flash/Coprocessor/DAGContext.h Outdated Show resolved Hide resolved

dbms/src/TestUtils/InterpreterTestUtils.cpp Outdated Show resolved Hide resolved

Merge branch 'master' of https://github.com/pingcap/tiflash into mock…

f6a1ec5

…_stream_inputblock

address comments.

0dc63ae

ywqzzy requested a review from windtalker June 7, 2022 08:04

fix build fail.

b6ede66

SeaRise reviewed Jun 7, 2022

View reviewed changes

ywqzzy added 2 commits June 7, 2022 17:14

Address comments.

8d472f0

address comments

61ba6b1

SeaRise self-requested a review June 7, 2022 09:57

SeaRise approved these changes Jun 7, 2022

View reviewed changes

dbms/src/Flash/tests/gtest_executor.cpp Outdated Show resolved Hide resolved

dbms/src/TestUtils/ExecutorTestUtils.cpp Outdated Show resolved Hide resolved

dbms/src/TestUtils/ExecutorTestUtils.cpp Show resolved Hide resolved

ti-chi-bot added the status/LGT1 Indicates that a PR has LGTM 1. label Jun 7, 2022

address comments.

cbef9bb

SeaRise mentioned this pull request Jun 7, 2022

Feature Branch: merge master to planner_refactory branch #5034

Merged

12 tasks

ywqzzy requested review from mengxin9014 and guo-shaoge June 7, 2022 10:42

mengxin9014 reviewed Jun 7, 2022

View reviewed changes

dbms/src/TestUtils/mockExecutor.cpp Show resolved Hide resolved

mengxin9014 reviewed Jun 7, 2022

View reviewed changes

dbms/src/TestUtils/mockExecutor.cpp Show resolved Hide resolved

mengxin9014 reviewed Jun 7, 2022

View reviewed changes

address comments

6a4b8c9

ywqzzy requested a review from mengxin9014 June 8, 2022 02:21

add more comments

c64220b

mengxin9014 approved these changes Jun 8, 2022

View reviewed changes

ti-chi-bot added status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. labels Jun 8, 2022

Merge branch 'master' into mock_stream_inputblock

38e6a7a

ti-chi-bot added the status/can-merge Indicates a PR has been approved by a committer. label Jun 8, 2022

ti-chi-bot merged commit fdab3f5 into pingcap:master Jun 8, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Test: Mock Input columns for operator tests #5041

Test: Mock Input columns for operator tests #5041

ywqzzy commented Jun 1, 2022 •

edited

Loading

ti-chi-bot commented Jun 1, 2022 •

edited

Loading

ywqzzy commented Jun 1, 2022

ti-chi-bot commented Jun 1, 2022

sre-bot commented Jun 7, 2022

SeaRise left a comment

mengxin9014 Jun 7, 2022

ywqzzy Jun 8, 2022

ywqzzy commented Jun 8, 2022

ti-chi-bot commented Jun 8, 2022

ti-chi-bot commented Jun 8, 2022

sre-bot commented Jun 8, 2022

Test: Mock Input columns for operator tests #5041

Test: Mock Input columns for operator tests #5041

Conversation

ywqzzy commented Jun 1, 2022 • edited Loading

What problem does this PR solve?

What is changed and how it works?

Check List

Release note

ti-chi-bot commented Jun 1, 2022 • edited Loading

ywqzzy commented Jun 1, 2022

ti-chi-bot commented Jun 1, 2022

sre-bot commented Jun 7, 2022

Coverage for changed files

Coverage summary

SeaRise left a comment

Choose a reason for hiding this comment

mengxin9014 Jun 7, 2022

Choose a reason for hiding this comment

ywqzzy Jun 8, 2022

Choose a reason for hiding this comment

ywqzzy commented Jun 8, 2022

ti-chi-bot commented Jun 8, 2022

ti-chi-bot commented Jun 8, 2022

sre-bot commented Jun 8, 2022

Coverage for changed files

Coverage summary

ywqzzy commented Jun 1, 2022 •

edited

Loading

ti-chi-bot commented Jun 1, 2022 •

edited

Loading