Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[fix](nereids)adjust aggregate function's nullable property #37215

Closed
wants to merge 4 commits into from

Conversation

starocean999
Copy link
Contributor

aggregate functions can be divided into 3 types by their nullable properties:

  • AlwaysNullable
  • AlwaysNotNullable
  • NullableAggregateFunction
    this pr correct the wrong nullable properties

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@starocean999
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 39868 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit fb47841b098436ed88e9ca5f017db6d12379e5ba, data reload: false

------ Round 1 ----------------------------------
q1	17987	4612	4382	4382
q2	2545	215	189	189
q3	11337	1291	1036	1036
q4	10780	825	767	767
q5	8108	2725	2678	2678
q6	225	139	144	139
q7	970	602	624	602
q8	9380	2049	2077	2049
q9	8960	6448	6488	6448
q10	8855	3741	3728	3728
q11	470	240	242	240
q12	408	242	232	232
q13	17782	2994	3020	2994
q14	271	227	222	222
q15	527	484	497	484
q16	505	386	377	377
q17	971	643	705	643
q18	8139	7450	7396	7396
q19	2492	1623	1422	1422
q20	659	325	328	325
q21	4893	3217	3173	3173
q22	396	342	346	342
Total cold run time: 116660 ms
Total hot run time: 39868 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4372	4224	4277	4224
q2	379	263	271	263
q3	3005	2722	2715	2715
q4	1889	1647	1596	1596
q5	5282	5278	5301	5278
q6	220	131	134	131
q7	2134	1698	1772	1698
q8	3188	3313	3313	3313
q9	8390	8387	8280	8280
q10	3853	3683	3661	3661
q11	581	481	506	481
q12	792	604	618	604
q13	17391	2997	2978	2978
q14	299	257	256	256
q15	506	490	494	490
q16	461	423	430	423
q17	1768	1482	1464	1464
q18	7779	7426	7319	7319
q19	1689	1551	1548	1548
q20	1970	1777	1756	1756
q21	5002	4792	4791	4791
q22	626	549	552	549
Total cold run time: 71576 ms
Total hot run time: 53818 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 172964 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit fb47841b098436ed88e9ca5f017db6d12379e5ba, data reload: false

query1	918	387	381	381
query2	6457	2383	2441	2383
query3	6654	206	213	206
query4	19694	17401	17370	17370
query5	4200	475	465	465
query6	261	161	166	161
query7	4600	303	293	293
query8	302	291	283	283
query9	8500	2364	2338	2338
query10	638	299	299	299
query11	10398	10009	9893	9893
query12	129	81	80	80
query13	1636	367	366	366
query14	9430	7402	7488	7402
query15	253	184	188	184
query16	7812	267	255	255
query17	1899	532	522	522
query18	1783	287	264	264
query19	197	144	150	144
query20	93	80	85	80
query21	215	132	129	129
query22	4359	4030	3987	3987
query23	33607	32891	33060	32891
query24	11202	2773	2739	2739
query25	620	361	358	358
query26	1639	157	186	157
query27	2931	317	317	317
query28	7606	2053	2047	2047
query29	993	636	603	603
query30	283	151	153	151
query31	967	726	767	726
query32	91	52	55	52
query33	760	289	281	281
query34	947	458	477	458
query35	728	621	609	609
query36	1125	934	938	934
query37	143	72	77	72
query38	2850	2706	2732	2706
query39	870	811	813	811
query40	283	128	129	128
query41	56	51	54	51
query42	118	102	111	102
query43	605	529	511	511
query44	1240	752	753	752
query45	199	170	164	164
query46	1084	724	719	719
query47	1861	1752	1782	1752
query48	384	297	292	292
query49	1172	409	412	409
query50	783	386	391	386
query51	6919	6792	6708	6708
query52	98	93	92	92
query53	353	298	294	294
query54	867	446	466	446
query55	74	75	73	73
query56	288	265	261	261
query57	1139	1035	1100	1035
query58	254	246	240	240
query59	3590	3158	3168	3158
query60	324	277	269	269
query61	125	89	88	88
query62	654	438	457	438
query63	327	291	300	291
query64	9861	2217	1706	1706
query65	3150	3088	3068	3068
query66	1066	344	342	342
query67	15317	14928	15163	14928
query68	4638	544	557	544
query69	467	313	308	308
query70	1118	1064	1075	1064
query71	398	286	286	286
query72	7169	5680	5705	5680
query73	742	327	328	327
query74	5906	5483	5501	5483
query75	3398	2626	2658	2626
query76	2692	926	927	926
query77	464	285	287	285
query78	10239	9757	9687	9687
query79	2637	513	520	513
query80	1095	495	472	472
query81	572	223	214	214
query82	757	111	105	105
query83	229	167	168	167
query84	246	82	84	82
query85	1848	280	269	269
query86	506	300	305	300
query87	3253	3133	3080	3080
query88	4140	2470	2460	2460
query89	536	372	374	372
query90	1861	187	185	185
query91	127	99	98	98
query92	65	50	51	50
query93	1951	517	522	517
query94	1241	195	186	186
query95	416	321	320	320
query96	588	268	266	266
query97	3183	3010	2987	2987
query98	213	199	197	197
query99	1303	836	808	808
Total cold run time: 273979 ms
Total hot run time: 172964 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.88 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit fb47841b098436ed88e9ca5f017db6d12379e5ba, data reload: false

query1	0.04	0.04	0.03
query2	0.08	0.03	0.03
query3	0.22	0.05	0.05
query4	1.68	0.07	0.07
query5	0.52	0.48	0.48
query6	1.13	0.73	0.72
query7	0.02	0.02	0.01
query8	0.05	0.04	0.05
query9	0.55	0.49	0.49
query10	0.55	0.54	0.53
query11	0.15	0.11	0.12
query12	0.14	0.12	0.12
query13	0.59	0.58	0.57
query14	0.76	0.77	0.78
query15	0.86	0.81	0.83
query16	0.37	0.35	0.36
query17	1.04	1.04	0.97
query18	0.23	0.25	0.26
query19	1.88	1.74	1.72
query20	0.02	0.01	0.01
query21	15.43	0.76	0.66
query22	4.39	6.91	2.24
query23	18.23	1.34	1.22
query24	2.23	0.22	0.23
query25	0.15	0.09	0.08
query26	0.26	0.17	0.18
query27	0.09	0.08	0.08
query28	13.19	1.01	1.00
query29	12.61	3.40	3.32
query30	0.26	0.05	0.06
query31	2.86	0.39	0.39
query32	3.28	0.48	0.48
query33	2.84	2.94	2.90
query34	17.05	4.45	4.46
query35	4.50	4.45	4.50
query36	0.66	0.46	0.50
query37	0.19	0.15	0.14
query38	0.16	0.15	0.15
query39	0.04	0.04	0.04
query40	0.17	0.15	0.13
query41	0.10	0.05	0.05
query42	0.05	0.04	0.05
query43	0.04	0.03	0.04
Total cold run time: 109.66 s
Total hot run time: 30.88 s

zhangstar333 added a commit that referenced this pull request Jul 15, 2024
## Proposed changes

in this PR have change some age function nullable property.
aggregate functions can be divided into 3 types by their nullable
properties:
- AlwaysNullable
- AlwaysNotNullable
- NullableAggregateFunction

this PR code of FE part from #37215,
the author is @starocean999
---------
Co-authored-by: starocean999 <12095047@qq.com>
seawinde pushed a commit to seawinde/doris that referenced this pull request Jul 17, 2024
…37330)

## Proposed changes

in this PR have change some age function nullable property.
aggregate functions can be divided into 3 types by their nullable
properties:
- AlwaysNullable
- AlwaysNotNullable
- NullableAggregateFunction

this PR code of FE part from apache#37215,
the author is @starocean999
---------
Co-authored-by: starocean999 <12095047@qq.com>
dataroaring pushed a commit that referenced this pull request Jul 17, 2024
## Proposed changes

in this PR have change some age function nullable property.
aggregate functions can be divided into 3 types by their nullable
properties:
- AlwaysNullable
- AlwaysNotNullable
- NullableAggregateFunction

this PR code of FE part from #37215,
the author is @starocean999
---------
Co-authored-by: starocean999 <12095047@qq.com>
@zhiqiang-hhhh
Copy link
Contributor

why this pr is closed but #37330 is merged?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants