[opt](inverted index) performance optimization for need_read_data in compound #35346

zzzxl1993 · 2024-05-24T07:27:32Z

…the compound process

Proposed changes

performance is relatively low when there are too many query conditions

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

doris-robot · 2024-05-24T07:27:37Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

zzzxl1993 · 2024-05-24T07:27:41Z

run buildall

github-actions · 2024-05-24T07:33:25Z

clang-tidy review says "All clean, LGTM! 👍"

doris-robot · 2024-05-24T08:05:04Z

TPC-H: Total hot run time: 42024 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 97ebb293398b0bcf4455e60735128a777340eeb8, data reload: false

------ Round 1 ----------------------------------
q1	17641	4360	4277	4277
q2	2013	197	189	189
q3	10458	1198	1209	1198
q4	10212	816	780	780
q5	7470	2712	2620	2620
q6	218	132	133	132
q7	977	604	609	604
q8	9251	2118	2117	2117
q9	9408	6761	6741	6741
q10	9182	3958	3905	3905
q11	447	243	246	243
q12	464	221	231	221
q13	17743	3293	3278	3278
q14	255	222	219	219
q15	513	471	471	471
q16	505	423	406	406
q17	999	664	775	664
q18	8550	7875	7785	7785
q19	6285	1565	1562	1562
q20	646	319	330	319
q21	5186	4132	4012	4012
q22	346	281	286	281
Total cold run time: 118769 ms
Total hot run time: 42024 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4665	4618	4516	4516
q2	376	272	269	269
q3	3142	2967	2937	2937
q4	1966	1669	1697	1669
q5	5609	5707	5654	5654
q6	219	127	132	127
q7	2254	1929	1966	1929
q8	3355	3477	3428	3428
q9	8817	8834	8820	8820
q10	4133	3785	3776	3776
q11	593	485	508	485
q12	793	626	624	624
q13	16832	3186	3200	3186
q14	310	279	265	265
q15	533	491	505	491
q16	496	440	440	440
q17	1832	1488	1481	1481
q18	7675	7674	7477	7477
q19	2684	1587	1544	1544
q20	2025	1814	1802	1802
q21	12469	4797	4681	4681
q22	578	482	504	482
Total cold run time: 81356 ms
Total hot run time: 56083 ms

doris-robot · 2024-05-24T08:07:13Z

TeamCity be ut coverage result:
Function Coverage: 35.64% (9017/25297)
Line Coverage: 27.30% (74564/273090)
Region Coverage: 26.54% (38599/145444)
Branch Coverage: 23.40% (19689/84140)
Coverage Report: http://coverage.selectdb-in.cc/coverage/97ebb293398b0bcf4455e60735128a777340eeb8_97ebb293398b0bcf4455e60735128a777340eeb8/report/index.html

doris-robot · 2024-05-24T08:15:57Z

TPC-DS: Total hot run time: 172355 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 97ebb293398b0bcf4455e60735128a777340eeb8, data reload: false

query1	918	391	373	373
query2	7040	2308	2325	2308
query3	6642	201	207	201
query4	20094	17221	17442	17221
query5	4072	424	422	422
query6	239	160	152	152
query7	4576	302	292	292
query8	239	188	185	185
query9	8423	2430	2401	2401
query10	452	287	267	267
query11	10778	10144	10018	10018
query12	143	89	90	89
query13	1668	367	362	362
query14	10161	7503	7502	7502
query15	216	170	179	170
query16	7861	259	270	259
query17	1711	525	527	525
query18	2028	272	273	272
query19	212	153	156	153
query20	97	85	84	84
query21	189	128	131	128
query22	4277	3991	3874	3874
query23	33485	32924	32985	32924
query24	10883	2903	2892	2892
query25	605	355	366	355
query26	1323	162	164	162
query27	2927	328	320	320
query28	7179	2055	2092	2055
query29	864	637	620	620
query30	254	172	174	172
query31	970	757	742	742
query32	92	50	52	50
query33	750	276	269	269
query34	945	480	478	478
query35	753	606	596	596
query36	1092	925	900	900
query37	136	71	72	71
query38	2897	2777	2790	2777
query39	870	791	795	791
query40	204	134	133	133
query41	48	47	49	47
query42	107	105	103	103
query43	568	570	537	537
query44	1265	753	767	753
query45	183	163	167	163
query46	1065	729	724	724
query47	1845	1822	1798	1798
query48	370	299	296	296
query49	865	381	380	380
query50	764	393	387	387
query51	6891	6863	6645	6645
query52	108	92	97	92
query53	355	284	286	284
query54	870	435	423	423
query55	73	74	74	74
query56	264	249	249	249
query57	1106	1039	1043	1039
query58	238	230	212	212
query59	3207	3006	3053	3006
query60	312	257	252	252
query61	90	85	85	85
query62	635	455	450	450
query63	316	294	292	292
query64	8824	2215	1749	1749
query65	3147	3087	3096	3087
query66	875	326	319	319
query67	15110	14752	14923	14752
query68	4643	529	540	529
query69	441	271	267	267
query70	1147	1149	1113	1113
query71	422	268	277	268
query72	7590	5802	5451	5451
query73	714	325	321	321
query74	5973	5628	5550	5550
query75	3391	2610	2622	2610
query76	2739	999	987	987
query77	395	278	281	278
query78	10192	9739	9646	9646
query79	2463	512	530	512
query80	1159	437	441	437
query81	537	253	241	241
query82	784	98	99	98
query83	239	175	176	175
query84	252	88	89	88
query85	1433	279	261	261
query86	469	324	313	313
query87	3346	3089	3099	3089
query88	4347	2337	2351	2337
query89	478	394	389	389
query90	1980	190	189	189
query91	127	97	97	97
query92	62	50	48	48
query93	1724	564	501	501
query94	1223	192	185	185
query95	403	314	307	307
query96	576	267	261	261
query97	3210	2976	2975	2975
query98	251	217	206	206
query99	1107	863	864	863
Total cold run time: 271583 ms
Total hot run time: 172355 ms

doris-robot · 2024-05-24T08:21:29Z

ClickBench: Total hot run time: 30.3 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 97ebb293398b0bcf4455e60735128a777340eeb8, data reload: false

query1	0.04	0.04	0.04
query2	0.09	0.04	0.04
query3	0.23	0.06	0.05
query4	1.68	0.08	0.08
query5	0.49	0.48	0.50
query6	1.12	0.72	0.71
query7	0.01	0.01	0.02
query8	0.06	0.04	0.04
query9	0.54	0.48	0.49
query10	0.54	0.53	0.53
query11	0.15	0.12	0.11
query12	0.14	0.12	0.12
query13	0.59	0.60	0.60
query14	0.80	0.79	0.79
query15	0.83	0.80	0.80
query16	0.37	0.36	0.36
query17	0.96	0.93	0.96
query18	0.21	0.24	0.22
query19	1.77	1.69	1.67
query20	0.02	0.01	0.01
query21	15.73	0.65	0.64
query22	4.82	6.92	1.82
query23	18.29	1.39	1.20
query24	1.55	0.33	0.22
query25	0.15	0.07	0.08
query26	0.27	0.16	0.17
query27	0.08	0.08	0.07
query28	13.31	1.02	0.99
query29	13.84	3.27	3.26
query30	0.24	0.06	0.06
query31	2.86	0.39	0.39
query32	3.27	0.47	0.47
query33	2.93	2.93	2.97
query34	17.25	4.41	4.41
query35	4.47	4.51	4.66
query36	0.67	0.47	0.46
query37	0.17	0.16	0.15
query38	0.16	0.14	0.14
query39	0.04	0.03	0.04
query40	0.16	0.14	0.14
query41	0.09	0.05	0.04
query42	0.06	0.04	0.05
query43	0.04	0.03	0.04
Total cold run time: 111.09 s
Total hot run time: 30.3 s

xiaokang

LGTM

github-actions · 2024-05-26T07:31:21Z

PR approved by at least one committer and no changes requested.

github-actions · 2024-05-26T07:31:24Z

PR approved by anyone and no changes requested.

zzzxl1993 · 2024-05-28T08:29:42Z

run buildall

zzzxl1993 · 2024-05-28T08:29:56Z

run buildall

github-actions · 2024-05-28T08:34:45Z

clang-tidy review says "All clean, LGTM! 👍"

doris-robot · 2024-05-28T09:26:56Z

TPC-H: Total hot run time: 42168 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 4f46ccb4f54051717a98c22d599c48c3057c3891, data reload: false

------ Round 1 ----------------------------------
q1	17628	4447	4352	4352
q2	2031	205	199	199
q3	10456	1264	1293	1264
q4	10191	886	799	799
q5	7512	2744	2743	2743
q6	228	146	131	131
q7	971	606	617	606
q8	9213	2163	2132	2132
q9	9558	6718	6697	6697
q10	9220	3898	3979	3898
q11	481	232	241	232
q12	433	225	227	225
q13	17392	3265	3162	3162
q14	275	222	237	222
q15	508	475	479	475
q16	550	405	396	396
q17	1003	675	744	675
q18	8282	7841	7754	7754
q19	7520	1596	1544	1544
q20	1020	327	316	316
q21	5137	4059	4154	4059
q22	366	291	287	287
Total cold run time: 119975 ms
Total hot run time: 42168 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4552	4407	4452	4407
q2	528	277	269	269
q3	3174	2961	2980	2961
q4	2002	1754	1673	1673
q5	5310	5491	5543	5491
q6	220	129	128	128
q7	2191	1825	1867	1825
q8	3256	3459	3463	3459
q9	8635	8643	8697	8643
q10	4101	3856	3713	3713
q11	593	490	530	490
q12	773	630	633	630
q13	16060	3201	3108	3108
q14	315	269	276	269
q15	541	478	487	478
q16	490	429	431	429
q17	1868	1514	1544	1514
q18	7991	7718	7476	7476
q19	2683	1603	1578	1578
q20	2009	1768	1777	1768
q21	9206	4786	4738	4738
q22	565	481	477	477
Total cold run time: 77063 ms
Total hot run time: 55524 ms

doris-robot · 2024-05-28T09:37:33Z

TPC-DS: Total hot run time: 168807 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 4f46ccb4f54051717a98c22d599c48c3057c3891, data reload: false

query1	927	393	373	373
query2	6428	2350	2197	2197
query3	6659	211	205	205
query4	19230	17356	17220	17220
query5	4099	418	430	418
query6	250	158	149	149
query7	4576	304	301	301
query8	238	180	179	179
query9	8721	2469	2466	2466
query10	444	298	265	265
query11	10386	9942	9960	9942
query12	145	90	89	89
query13	1626	387	381	381
query14	8508	6978	6271	6271
query15	202	166	175	166
query16	7420	270	259	259
query17	1305	514	539	514
query18	1926	265	267	265
query19	218	148	154	148
query20	98	85	85	85
query21	198	136	130	130
query22	4213	4024	4049	4024
query23	33626	33048	32982	32982
query24	7278	2949	2902	2902
query25	578	358	356	356
query26	709	159	157	157
query27	2160	313	328	313
query28	4640	2097	2083	2083
query29	853	604	594	594
query30	220	151	153	151
query31	948	746	737	737
query32	101	60	57	57
query33	529	291	294	291
query34	865	478	479	478
query35	726	601	606	601
query36	1089	901	923	901
query37	104	66	66	66
query38	2901	2749	2719	2719
query39	834	800	786	786
query40	198	123	125	123
query41	47	43	43	43
query42	107	94	98	94
query43	575	504	518	504
query44	1060	721	752	721
query45	179	159	161	159
query46	1070	709	738	709
query47	1863	1782	1776	1776
query48	381	303	299	299
query49	843	388	449	388
query50	769	387	398	387
query51	6910	6703	6739	6703
query52	102	96	93	93
query53	358	281	283	281
query54	549	449	433	433
query55	75	74	70	70
query56	262	250	254	250
query57	1096	1091	1044	1044
query58	229	209	209	209
query59	3398	3356	3020	3020
query60	275	253	259	253
query61	95	103	127	103
query62	602	454	457	454
query63	314	282	280	280
query64	8448	2244	1746	1746
query65	3161	3100	3102	3100
query66	856	346	338	338
query67	15257	14658	14831	14658
query68	4548	539	545	539
query69	483	297	280	280
query70	1154	1134	1075	1075
query71	393	273	267	267
query72	7703	4740	2693	2693
query73	726	332	329	329
query74	5988	5583	5613	5583
query75	3287	2657	2640	2640
query76	2311	1026	1001	1001
query77	417	288	278	278
query78	10270	9862	9947	9862
query79	2562	523	519	519
query80	1137	477	469	469
query81	551	223	226	223
query82	740	95	94	94
query83	254	179	180	179
query84	249	94	91	91
query85	1359	326	309	309
query86	471	284	328	284
query87	3328	3102	3078	3078
query88	4234	2448	2454	2448
query89	486	398	387	387
query90	2078	185	191	185
query91	133	117	107	107
query92	63	52	59	52
query93	2132	536	512	512
query94	1191	198	194	194
query95	416	325	321	321
query96	595	268	275	268
query97	3144	3029	3039	3029
query98	238	224	214	214
query99	1173	858	838	838
Total cold run time: 258871 ms
Total hot run time: 168807 ms

github-actions · 2024-05-28T09:39:05Z

clang-tidy review says "All clean, LGTM! 👍"

doris-robot · 2024-05-28T09:41:40Z

TeamCity be ut coverage result:
Function Coverage: 35.77% (9004/25173)
Line Coverage: 27.39% (74554/272168)
Region Coverage: 26.62% (38576/144928)
Branch Coverage: 23.48% (19670/83776)
Coverage Report: http://coverage.selectdb-in.cc/coverage/4f46ccb4f54051717a98c22d599c48c3057c3891_4f46ccb4f54051717a98c22d599c48c3057c3891/report/index.html

doris-robot · 2024-05-28T09:43:03Z

ClickBench: Total hot run time: 30.5 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 4f46ccb4f54051717a98c22d599c48c3057c3891, data reload: false

query1	0.03	0.03	0.02
query2	0.08	0.05	0.04
query3	0.22	0.05	0.06
query4	1.67	0.09	0.08
query5	0.50	0.48	0.54
query6	1.13	0.72	0.72
query7	0.02	0.01	0.02
query8	0.05	0.04	0.04
query9	0.54	0.50	0.49
query10	0.55	0.54	0.56
query11	0.16	0.12	0.11
query12	0.15	0.12	0.12
query13	0.59	0.59	0.60
query14	0.76	0.80	0.76
query15	0.83	0.82	0.81
query16	0.37	0.36	0.37
query17	0.96	1.01	0.95
query18	0.21	0.28	0.24
query19	1.78	1.67	1.68
query20	0.01	0.01	0.01
query21	15.44	0.70	0.68
query22	4.91	6.60	1.90
query23	18.34	1.33	1.26
query24	1.41	0.32	0.27
query25	0.17	0.08	0.08
query26	0.26	0.16	0.16
query27	0.08	0.07	0.07
query28	13.37	1.02	1.00
query29	12.92	3.28	3.24
query30	0.24	0.06	0.05
query31	2.86	0.37	0.38
query32	3.30	0.47	0.47
query33	2.87	2.90	2.95
query34	16.83	4.43	4.45
query35	4.52	4.49	4.58
query36	0.69	0.48	0.46
query37	0.17	0.15	0.16
query38	0.15	0.14	0.14
query39	0.04	0.04	0.03
query40	0.16	0.14	0.14
query41	0.08	0.04	0.05
query42	0.06	0.05	0.04
query43	0.04	0.04	0.04
Total cold run time: 109.52 s
Total hot run time: 30.5 s

xiaokang

LGTM

github-actions · 2024-05-30T09:41:18Z

PR approved by at least one committer and no changes requested.

doris-robot · 2024-05-31T09:37:51Z

TPC-H: Total hot run time: 40131 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 4923d25b28b214d339b56defecfa5ac4365996f9, data reload: false

------ Round 1 ----------------------------------
q1	17608	4320	4278	4278
q2	2022	192	186	186
q3	10564	1233	1149	1149
q4	10562	792	816	792
q5	7802	2699	2649	2649
q6	222	141	144	141
q7	982	619	619	619
q8	9606	2086	2105	2086
q9	9341	6576	6604	6576
q10	9110	3781	3726	3726
q11	472	249	234	234
q12	447	221	217	217
q13	18766	2954	3000	2954
q14	263	214	215	214
q15	510	478	465	465
q16	494	374	387	374
q17	949	701	736	701
q18	8083	7446	7489	7446
q19	3771	1559	1531	1531
q20	658	314	305	305
q21	4884	3228	3163	3163
q22	369	325	329	325
Total cold run time: 117485 ms
Total hot run time: 40131 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4295	4216	4210	4210
q2	381	266	267	266
q3	2977	2744	2720	2720
q4	1878	1599	1647	1599
q5	5271	5260	5266	5260
q6	215	129	134	129
q7	2153	1789	1775	1775
q8	3243	3275	3277	3275
q9	8326	8333	8309	8309
q10	3865	3705	3688	3688
q11	614	496	501	496
q12	767	615	603	603
q13	17561	2991	2993	2991
q14	299	250	268	250
q15	521	470	481	470
q16	464	414	426	414
q17	1758	1504	1451	1451
q18	7611	7623	7488	7488
q19	3623	1574	1578	1574
q20	1978	1769	1766	1766
q21	4900	4717	4736	4717
q22	601	543	557	543
Total cold run time: 73301 ms
Total hot run time: 53994 ms

doris-robot · 2024-05-31T09:48:41Z

TPC-DS: Total hot run time: 169162 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 4923d25b28b214d339b56defecfa5ac4365996f9, data reload: false

query1	915	370	377	370
query2	6438	2339	2281	2281
query3	6656	203	208	203
query4	19395	17429	17300	17300
query5	4151	414	413	413
query6	245	155	152	152
query7	4585	293	301	293
query8	320	275	284	275
query9	8476	2406	2379	2379
query10	451	271	281	271
query11	10501	9959	9984	9959
query12	130	93	96	93
query13	1651	359	362	359
query14	8362	7786	7270	7270
query15	233	194	192	192
query16	7829	272	263	263
query17	1833	540	525	525
query18	1963	279	293	279
query19	202	161	161	161
query20	96	87	91	87
query21	198	131	137	131
query22	4292	3943	3911	3911
query23	33691	32959	33075	32959
query24	11540	2889	2832	2832
query25	667	353	357	353
query26	1775	153	153	153
query27	3038	320	346	320
query28	7677	2036	2041	2036
query29	1080	613	593	593
query30	283	146	150	146
query31	973	774	746	746
query32	91	51	51	51
query33	765	268	257	257
query34	1037	488	479	479
query35	773	603	597	597
query36	1108	894	943	894
query37	144	66	71	66
query38	2895	2782	2754	2754
query39	900	777	776	776
query40	282	125	124	124
query41	52	50	52	50
query42	104	96	94	94
query43	566	532	541	532
query44	1282	723	734	723
query45	181	167	165	165
query46	1067	728	718	718
query47	1859	1734	1779	1734
query48	359	297	293	293
query49	1174	375	379	375
query50	771	375	390	375
query51	6732	6607	6738	6607
query52	103	87	98	87
query53	352	290	293	290
query54	968	430	423	423
query55	73	73	70	70
query56	267	242	249	242
query57	1129	1075	1058	1058
query58	230	263	204	204
query59	3285	3212	3155	3155
query60	270	258	252	252
query61	91	87	86	86
query62	658	455	446	446
query63	318	285	293	285
query64	9824	2212	1733	1733
query65	3208	3116	3132	3116
query66	1385	326	318	318
query67	15004	14973	14699	14699
query68	4614	520	521	520
query69	445	266	273	266
query70	1127	1159	1091	1091
query71	412	273	268	268
query72	7259	2749	2579	2579
query73	709	321	322	321
query74	5891	5579	5610	5579
query75	3600	2628	2578	2578
query76	3042	1017	1003	1003
query77	636	263	267	263
query78	10271	9808	9738	9738
query79	2292	508	506	506
query80	860	431	433	431
query81	510	223	216	216
query82	645	88	94	88
query83	258	171	168	168
query84	241	82	85	82
query85	1749	349	264	264
query86	483	320	304	304
query87	3325	3077	3112	3077
query88	4244	2366	2348	2348
query89	494	401	377	377
query90	2001	189	195	189
query91	126	98	96	96
query92	59	47	51	47
query93	2247	502	492	492
query94	1188	187	184	184
query95	396	312	309	309
query96	586	268	270	268
query97	3212	3033	3012	3012
query98	250	223	218	218
query99	1104	847	861	847
Total cold run time: 273229 ms
Total hot run time: 169162 ms

doris-robot · 2024-05-31T09:54:10Z

ClickBench: Total hot run time: 30.55 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 4923d25b28b214d339b56defecfa5ac4365996f9, data reload: false

query1	0.03	0.03	0.04
query2	0.09	0.04	0.04
query3	0.23	0.05	0.04
query4	1.68	0.08	0.08
query5	0.50	0.49	0.48
query6	1.12	0.73	0.73
query7	0.02	0.02	0.01
query8	0.04	0.04	0.04
query9	0.54	0.50	0.49
query10	0.53	0.55	0.54
query11	0.15	0.11	0.12
query12	0.15	0.13	0.13
query13	0.59	0.58	0.59
query14	0.81	0.78	0.78
query15	0.83	0.83	0.81
query16	0.37	0.36	0.35
query17	0.95	1.01	0.94
query18	0.21	0.24	0.25
query19	1.82	1.73	1.67
query20	0.02	0.01	0.01
query21	15.43	0.72	0.69
query22	4.86	6.66	2.02
query23	18.39	1.37	1.22
query24	1.41	0.39	0.24
query25	0.15	0.09	0.08
query26	0.26	0.17	0.17
query27	0.08	0.08	0.07
query28	13.27	1.02	1.00
query29	12.74	3.36	3.31
query30	0.25	0.06	0.05
query31	2.87	0.39	0.37
query32	3.29	0.46	0.47
query33	2.88	2.83	2.91
query34	17.12	4.42	4.41
query35	4.47	4.46	4.46
query36	0.65	0.48	0.46
query37	0.18	0.16	0.16
query38	0.16	0.15	0.15
query39	0.05	0.04	0.03
query40	0.17	0.15	0.14
query41	0.09	0.05	0.06
query42	0.06	0.05	0.05
query43	0.04	0.03	0.04
Total cold run time: 109.55 s
Total hot run time: 30.55 s

xiaokang

LGTM

github-actions · 2024-06-01T00:40:30Z

PR approved by at least one committer and no changes requested.

xiaokang

LGTM

airborne12

LGTM

…compound #35346 (#36192)

…compound (#35346) Performance is relatively low when there are too many query conditions

…dle non-pushdown conditions (#36292) …dle non-pushdown conditions 1. Fixed the previous optimization of #35346 not handling non-pushdown conditions.

…compound #35346 #36292 (#36404) pick from master #35346 #36292

…dle non-pushdown conditions (#36292) …dle non-pushdown conditions 1. Fixed the previous optimization of #35346 not handling non-pushdown conditions.

…compound apache#35346 (apache#35695)

xiaokang changed the title ~~[opt](inverted index) performance optimization for need_read_data in …~~ [opt](inverted index) performance optimization for need_read_data in compound May 26, 2024

xiaokang previously approved these changes May 26, 2024

View reviewed changes

xiaokang added usercase Important user case type label dev/2.0.x labels May 26, 2024

github-actions bot added the approved Indicates a PR has been approved by one committer. label May 26, 2024

github-actions bot added the reviewed label May 26, 2024

zzzxl1993 dismissed xiaokang’s stale review via 4f46ccb May 28, 2024 08:29

zzzxl1993 force-pushed the compound_opt_1 branch from 97ebb29 to 4f46ccb Compare May 28, 2024 08:29

github-actions bot removed the approved Indicates a PR has been approved by one committer. label May 28, 2024

zzzxl1993 force-pushed the compound_opt_1 branch from 4f46ccb to 2501333 Compare May 28, 2024 09:33

xiaokang previously approved these changes May 30, 2024

View reviewed changes

github-actions bot added the approved Indicates a PR has been approved by one committer. label May 30, 2024

xiaokang approved these changes Jun 1, 2024

View reviewed changes

github-actions bot added the approved Indicates a PR has been approved by one committer. label Jun 1, 2024

xiaokang added dev/2.0.11-merged and removed dev/2.0.x labels Jun 2, 2024

xiaokang approved these changes Jun 2, 2024

View reviewed changes

airborne12 approved these changes Jun 11, 2024

View reviewed changes

qidaye approved these changes Jun 11, 2024

View reviewed changes

qidaye merged commit d9b490a into apache:master Jun 11, 2024
30 of 34 checks passed

zzzxl1993 mentioned this pull request Jun 12, 2024

[opt](inverted index) performance optimization for need_read_data in … #36192

Merged

xiaokang pushed a commit that referenced this pull request Jun 12, 2024

[opt](inverted index) performance optimization for need_read_data in …

f1e83f5

…compound #35346 (#36192)

xiaokang added the dev/2.1.4-merged label Jun 12, 2024

dataroaring pushed a commit that referenced this pull request Jun 13, 2024

[opt](inverted index) performance optimization for need_read_data in …

3919c02

…compound (#35346) Performance is relatively low when there are too many query conditions

zzzxl1993 mentioned this pull request Jun 14, 2024

[fix](inverted index) fix the issue where need_read_data does not handle non-pushdown conditions #36292

Merged

zzzxl1993 mentioned this pull request Jun 18, 2024

[opt](inverted index) performance optimization for need_read_data in compound #36404

Merged

xiaokang pushed a commit that referenced this pull request Jun 20, 2024

[opt](inverted index) performance optimization for need_read_data in …

dabd27e

…compound #35346 #36292 (#36404) pick from master #35346 #36292

xiaokang added dev/2.1.x doing and removed dev/2.1.4-merged labels Jul 15, 2024

yiguolei added dev/2.0.14-merged and removed dev/2.1.x labels Jul 16, 2024

xiaokang mentioned this pull request Aug 3, 2024

Release Note 2.0.14 #38819

Open

dataroaring added the dev/3.0.0-merged label Aug 5, 2024

xiaokang removed the doing label Aug 14, 2024

mongo360 pushed a commit to mongo360/doris that referenced this pull request Aug 16, 2024

[opt](inverted index) performance optimization for need_read_data in …

688f01e

…compound apache#35346 (apache#35695)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[opt](inverted index) performance optimization for need_read_data in compound #35346

[opt](inverted index) performance optimization for need_read_data in compound #35346

zzzxl1993 commented May 24, 2024

doris-robot commented May 24, 2024

zzzxl1993 commented May 24, 2024

github-actions bot commented May 24, 2024

doris-robot commented May 24, 2024

doris-robot commented May 24, 2024

doris-robot commented May 24, 2024

doris-robot commented May 24, 2024

xiaokang left a comment

github-actions bot commented May 26, 2024

github-actions bot commented May 26, 2024

zzzxl1993 commented May 28, 2024

zzzxl1993 commented May 28, 2024

github-actions bot commented May 28, 2024

doris-robot commented May 28, 2024

doris-robot commented May 28, 2024

github-actions bot commented May 28, 2024

doris-robot commented May 28, 2024

doris-robot commented May 28, 2024

xiaokang left a comment

github-actions bot commented May 30, 2024

doris-robot commented May 31, 2024

doris-robot commented May 31, 2024

doris-robot commented May 31, 2024

xiaokang left a comment

github-actions bot commented Jun 1, 2024

xiaokang left a comment

airborne12 left a comment

[opt](inverted index) performance optimization for need_read_data in compound #35346

[opt](inverted index) performance optimization for need_read_data in compound #35346

Conversation

zzzxl1993 commented May 24, 2024

Proposed changes

Further comments

doris-robot commented May 24, 2024

zzzxl1993 commented May 24, 2024

github-actions bot commented May 24, 2024

doris-robot commented May 24, 2024

doris-robot commented May 24, 2024

doris-robot commented May 24, 2024

doris-robot commented May 24, 2024

xiaokang left a comment

Choose a reason for hiding this comment

github-actions bot commented May 26, 2024

github-actions bot commented May 26, 2024

zzzxl1993 commented May 28, 2024

zzzxl1993 commented May 28, 2024

github-actions bot commented May 28, 2024

doris-robot commented May 28, 2024

doris-robot commented May 28, 2024

github-actions bot commented May 28, 2024

doris-robot commented May 28, 2024

doris-robot commented May 28, 2024

xiaokang left a comment

Choose a reason for hiding this comment

github-actions bot commented May 30, 2024

doris-robot commented May 31, 2024

doris-robot commented May 31, 2024

doris-robot commented May 31, 2024

xiaokang left a comment

Choose a reason for hiding this comment

github-actions bot commented Jun 1, 2024

xiaokang left a comment

Choose a reason for hiding this comment

airborne12 left a comment

Choose a reason for hiding this comment