Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Fix](nereids) fix bug when insert into select empty table with agg #35291

Merged
merged 1 commit into from
May 24, 2024
Merged

[Fix](nereids) fix bug when insert into select empty table with agg #35291

merged 1 commit into from
May 24, 2024

Conversation

feiniaofeiafei
Copy link
Contributor

@feiniaofeiafei feiniaofeiafei commented May 23, 2024 •
edited
Loading

This bug introduced by #34418.
Some operators can produce data even if the input is an empty table, such as AGGREGATE. Therefore, it is necessary to check whether sink's child is in an empty relation. Only when the child is in an empty relation can it return directly without performing actual data insertion.

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@morrySnow
Copy link
Contributor

add a description explaining the cause of the problem and the PR number that introduced it

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label May 23, 2024
@github-actions GitHub Actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions GitHub Actions
Copy link
Contributor

PR approved by anyone and no changes requested.

@feiniaofeiafei
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 39945 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit ddcd1ffa289dc494ee768a2f375397538b2a9654, data reload: false

------ Round 1 ----------------------------------
q1	17818	4608	4317	4317
q2	2027	196	192	192
q3	10572	1171	1222	1171
q4	10187	828	759	759
q5	7486	2642	2652	2642
q6	216	134	135	134
q7	950	589	598	589
q8	9234	2074	2122	2074
q9	9037	6552	6529	6529
q10	8991	3764	3635	3635
q11	467	237	236	236
q12	445	216	216	216
q13	17764	2966	3003	2966
q14	260	232	228	228
q15	512	469	475	469
q16	478	380	379	379
q17	960	686	741	686
q18	8165	7509	7500	7500
q19	6906	1551	1525	1525
q20	649	295	300	295
q21	4926	3842	3134	3134
q22	341	271	269	269
Total cold run time: 118391 ms
Total hot run time: 39945 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4294	4225	4205	4205
q2	368	280	266	266
q3	3019	2746	2724	2724
q4	1866	1562	1575	1562
q5	5289	5283	5243	5243
q6	217	127	130	127
q7	2134	1773	1742	1742
q8	3151	3309	3323	3309
q9	8375	8329	8497	8329
q10	3879	3646	3680	3646
q11	592	469	483	469
q12	774	603	585	585
q13	16441	2998	3000	2998
q14	298	259	272	259
q15	515	468	473	468
q16	473	412	418	412
q17	1787	1507	1466	1466
q18	7826	7738	7465	7465
q19	2023	1557	1507	1507
q20	1990	1788	1790	1788
q21	4943	4758	4759	4758
q22	573	506	474	474
Total cold run time: 70827 ms
Total hot run time: 53802 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 172231 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit ddcd1ffa289dc494ee768a2f375397538b2a9654, data reload: false

query1	905	390	385	385
query2	6444	2504	2452	2452
query3	6655	206	209	206
query4	22051	17426	17195	17195
query5	4150	426	420	420
query6	246	154	176	154
query7	4590	302	306	302
query8	239	195	191	191
query9	8530	2392	2337	2337
query10	448	276	260	260
query11	10556	10089	10068	10068
query12	137	85	88	85
query13	1633	357	368	357
query14	10124	6833	6840	6833
query15	220	169	177	169
query16	7830	270	261	261
query17	1790	520	507	507
query18	1965	265	272	265
query19	197	157	159	157
query20	102	91	85	85
query21	199	130	133	130
query22	4244	3865	4133	3865
query23	33752	32878	33118	32878
query24	12086	2878	2823	2823
query25	662	360	376	360
query26	1780	165	164	164
query27	2954	328	326	326
query28	7524	2028	2024	2024
query29	1106	640	593	593
query30	276	170	176	170
query31	966	761	741	741
query32	91	51	56	51
query33	765	268	264	264
query34	1032	475	492	475
query35	722	586	590	586
query36	1082	891	920	891
query37	189	69	72	69
query38	2942	2773	2767	2767
query39	862	795	812	795
query40	276	149	124	124
query41	46	43	45	43
query42	103	97	100	97
query43	596	568	577	568
query44	1230	718	733	718
query45	178	161	157	157
query46	1070	726	720	720
query47	1875	1793	1777	1777
query48	369	302	293	293
query49	1215	384	389	384
query50	771	399	386	386
query51	6810	6831	6751	6751
query52	100	96	92	92
query53	352	283	291	283
query54	993	422	424	422
query55	75	72	76	72
query56	260	243	249	243
query57	1131	1040	1032	1032
query58	278	216	205	205
query59	3565	3268	3180	3180
query60	311	264	253	253
query61	88	84	87	84
query62	646	466	455	455
query63	309	287	287	287
query64	9793	2320	1760	1760
query65	3186	3133	3098	3098
query66	1375	344	315	315
query67	15479	15083	14845	14845
query68	4533	537	547	537
query69	438	265	263	263
query70	1163	1124	1072	1072
query71	400	267	275	267
query72	7969	5739	5442	5442
query73	732	322	328	322
query74	5966	5745	5625	5625
query75	3315	2666	2673	2666
query76	2498	984	964	964
query77	425	263	265	263
query78	10214	9873	9810	9810
query79	2040	533	536	533
query80	1170	452	445	445
query81	529	242	240	240
query82	642	100	102	100
query83	253	172	173	172
query84	232	87	84	84
query85	1844	315	265	265
query86	485	302	300	300
query87	3314	3116	3136	3116
query88	4190	2340	2339	2339
query89	479	401	401	401
query90	1980	193	190	190
query91	129	97	99	97
query92	64	50	50	50
query93	2394	524	505	505
query94	1231	190	190	190
query95	425	316	312	312
query96	598	264	265	264
query97	3244	3029	3078	3029
query98	232	222	222	222
query99	1141	841	857	841
Total cold run time: 278795 ms
Total hot run time: 172231 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.56 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit ddcd1ffa289dc494ee768a2f375397538b2a9654, data reload: false

query1	0.04	0.03	0.03
query2	0.08	0.04	0.05
query3	0.24	0.05	0.06
query4	1.67	0.08	0.08
query5	0.50	0.47	0.49
query6	1.13	0.71	0.72
query7	0.01	0.01	0.01
query8	0.04	0.04	0.04
query9	0.55	0.49	0.50
query10	0.53	0.55	0.55
query11	0.16	0.11	0.11
query12	0.15	0.12	0.12
query13	0.59	0.58	0.58
query14	0.76	0.78	0.78
query15	0.82	0.80	0.80
query16	0.36	0.35	0.35
query17	1.00	0.96	0.99
query18	0.21	0.26	0.23
query19	1.74	1.72	1.73
query20	0.01	0.01	0.01
query21	16.03	0.69	0.66
query22	4.55	6.62	2.00
query23	18.26	1.37	1.27
query24	1.76	0.27	0.22
query25	0.14	0.08	0.08
query26	0.25	0.16	0.17
query27	0.08	0.09	0.08
query28	13.35	1.03	0.99
query29	13.15	3.24	3.24
query30	0.24	0.06	0.06
query31	2.86	0.39	0.38
query32	3.29	0.47	0.46
query33	2.85	2.90	2.92
query34	16.98	4.42	4.41
query35	4.51	4.48	4.48
query36	0.65	0.46	0.46
query37	0.17	0.16	0.15
query38	0.15	0.15	0.14
query39	0.04	0.03	0.03
query40	0.18	0.14	0.14
query41	0.08	0.04	0.05
query42	0.06	0.04	0.05
query43	0.04	0.04	0.04
Total cold run time: 110.26 s
Total hot run time: 30.56 s

Jibing-Li
Jibing-Li approved these changes May 24, 2024
View reviewed changes
Copy link
Contributor

@Jibing-Li Jibing-Li left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@morrySnow morrySnow merged commit a62fdcd into apache:master May 24, 2024
28 of 31 checks passed
@feiniaofeiafei feiniaofeiafei mentioned this pull request May 24, 2024
Merged
dataroaring pushed a commit that referenced this pull request May 26, 2024
@feiniaofeiafei @dataroaring
[Fix](nereids) fix when insert into select empty table (#35291)
ed842ed 
This bug introduced by #34418.
Some operators can produce data even if the input is an empty table, such as AGGREGATE. Therefore, it is necessary to check whether sink's child is in an empty relation. Only when the child is in an empty relation can it return directly without performing actual data insertion.
seawinde pushed a commit to seawinde/doris that referenced this pull request May 27, 2024
@feiniaofeiafei @seawinde
[Fix](nereids) fix when insert into select empty table (apache#35291)
fb22540 
This bug introduced by apache#34418.
Some operators can produce data even if the input is an empty table, such as AGGREGATE. Therefore, it is necessary to check whether sink's child is in an empty relation. Only when the child is in an empty relation can it return directly without performing actual data insertion.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Jibing-Li Jibing-Li Jibing-Li approved these changes

@morrySnow morrySnow morrySnow approved these changes

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by one committer. dev/2.1.4-merged dev/3.0.0-merged p0_b reviewed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@feiniaofeiafei @doris-robot @morrySnow @Jibing-Li @yiguolei @dataroaring