Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[enhancement](schema-change) Cloud schema change do clean up when job failed #48426

Merged
merged 6 commits into from
Mar 11, 2025
Merged

[enhancement](schema-change) Cloud schema change do clean up when job failed #48426

merged 6 commits into from
Mar 11, 2025

Conversation

TangSiyang2001
Copy link
Collaborator

@TangSiyang2001 TangSiyang2001 commented Feb 27, 2025 •
edited
Loading

What problem does this PR solve?

Problem Summary:

If schema change failed, clear output rowset directly from file cache.

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@Thearas
Copy link
Contributor

Thearas commented Feb 27, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@TangSiyang2001
Copy link
Collaborator Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 32801 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit aeae75df14e8edac417d948f14b488a8552f817c, data reload: false

------ Round 1 ----------------------------------
q1	17573	5353	5244	5244
q2	2061	320	189	189
q3	10375	1359	798	798
q4	10222	1084	561	561
q5	7523	2532	2470	2470
q6	210	182	138	138
q7	958	807	644	644
q8	9324	1439	1221	1221
q9	4989	4730	4797	4730
q10	6901	2335	1910	1910
q11	504	293	285	285
q12	356	384	232	232
q13	17770	3847	3198	3198
q14	238	241	216	216
q15	547	471	477	471
q16	635	646	594	594
q17	591	927	348	348
q18	6754	6276	6258	6258
q19	1218	954	599	599
q20	330	341	203	203
q21	2979	2405	2180	2180
q22	366	341	312	312
Total cold run time: 102424 ms
Total hot run time: 32801 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5326	5312	5374	5312
q2	247	358	228	228
q3	2222	2774	2334	2334
q4	1530	1944	1440	1440
q5	4308	4198	4177	4177
q6	217	175	126	126
q7	2013	1990	1829	1829
q8	2843	2809	2754	2754
q9	7307	7242	7217	7217
q10	3125	3310	2856	2856
q11	621	538	505	505
q12	696	766	585	585
q13	3607	4100	3345	3345
q14	287	302	293	293
q15	532	498	464	464
q16	652	716	671	671
q17	1204	1793	1374	1374
q18	7613	7401	7286	7286
q19	864	899	1010	899
q20	2023	2026	1919	1919
q21	5722	5186	4829	4829
q22	701	617	552	552
Total cold run time: 53660 ms
Total hot run time: 50995 ms

@doris-robot
Copy link

BE UT Coverage Report

Increment line coverage 2.70% (2/74) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 44.79% (11941/26659)
Line Coverage 34.22% (99915/291953)
Region Coverage 33.38% (51130/153197)
Branch Coverage 28.92% (25667/88760)

@doris-robot
Copy link

TPC-DS: Total hot run time: 192643 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit aeae75df14e8edac417d948f14b488a8552f817c, data reload: false

query1	1317	969	951	951
query2	6207	1876	1886	1876
query3	11178	4744	4708	4708
query4	26346	23580	23697	23580
query5	5518	719	536	536
query6	325	189	191	189
query7	3990	525	311	311
query8	286	259	245	245
query9	8511	2527	2520	2520
query10	510	342	260	260
query11	15410	15078	14963	14963
query12	166	111	110	110
query13	1580	533	412	412
query14	9206	6869	7016	6869
query15	211	204	189	189
query16	7651	681	507	507
query17	1222	761	604	604
query18	2005	442	351	351
query19	205	213	175	175
query20	132	137	129	129
query21	216	127	112	112
query22	4438	4671	4346	4346
query23	34544	33724	33412	33412
query24	8248	2502	2476	2476
query25	488	501	413	413
query26	1191	273	160	160
query27	2199	535	342	342
query28	4419	2453	2430	2430
query29	593	566	457	457
query30	219	191	165	165
query31	929	910	854	854
query32	73	69	60	60
query33	565	381	310	310
query34	827	891	508	508
query35	838	853	762	762
query36	970	1022	912	912
query37	120	105	76	76
query38	4229	4434	4095	4095
query39	1508	1443	1406	1406
query40	227	122	116	116
query41	53	55	50	50
query42	127	111	110	110
query43	526	527	487	487
query44	1352	821	803	803
query45	191	171	165	165
query46	919	1103	687	687
query47	1808	1822	1781	1781
query48	409	439	310	310
query49	797	531	472	472
query50	743	781	458	458
query51	4396	4352	4222	4222
query52	105	103	100	100
query53	250	273	195	195
query54	498	526	428	428
query55	85	85	87	85
query56	275	283	254	254
query57	1152	1189	1115	1115
query58	262	269	238	238
query59	2790	2805	2650	2650
query60	321	290	284	284
query61	134	138	127	127
query62	801	702	690	690
query63	237	189	201	189
query64	3997	1066	746	746
query65	3388	3303	3307	3303
query66	1173	413	311	311
query67	16189	15572	15482	15482
query68	8579	900	506	506
query69	514	299	267	267
query70	1110	1167	1129	1129
query71	494	315	287	287
query72	5682	3666	3820	3666
query73	766	767	352	352
query74	9026	8981	9069	8981
query75	3920	3177	2719	2719
query76	3739	1190	781	781
query77	785	390	290	290
query78	10129	9998	9290	9290
query79	3346	916	586	586
query80	903	543	476	476
query81	517	290	259	259
query82	727	128	97	97
query83	207	177	164	164
query84	293	95	73	73
query85	752	362	318	318
query86	347	316	283	283
query87	4495	4522	4464	4464
query88	3676	2189	2170	2170
query89	438	335	297	297
query90	1908	194	193	193
query91	136	144	111	111
query92	84	62	59	59
query93	2413	1072	582	582
query94	642	426	308	308
query95	343	273	261	261
query96	480	611	267	267
query97	3363	3453	3301	3301
query98	233	206	204	204
query99	1436	1431	1263	1263
Total cold run time: 283278 ms
Total hot run time: 192643 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.27 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit aeae75df14e8edac417d948f14b488a8552f817c, data reload: false

query1	0.04	0.04	0.03
query2	0.07	0.04	0.03
query3	0.23	0.07	0.08
query4	1.61	0.10	0.10
query5	0.58	0.54	0.55
query6	1.19	0.72	0.70
query7	0.02	0.02	0.02
query8	0.04	0.03	0.04
query9	0.58	0.55	0.53
query10	0.57	0.58	0.56
query11	0.15	0.11	0.11
query12	0.14	0.11	0.13
query13	0.62	0.60	0.60
query14	2.68	2.70	2.86
query15	0.91	0.86	0.86
query16	0.39	0.39	0.38
query17	1.02	1.02	1.02
query18	0.21	0.20	0.20
query19	1.92	1.76	1.99
query20	0.02	0.01	0.01
query21	15.35	0.93	0.55
query22	0.77	1.32	0.88
query23	14.76	1.37	0.65
query24	6.66	1.66	0.95
query25	0.49	0.21	0.10
query26	0.66	0.17	0.16
query27	0.05	0.05	0.05
query28	9.95	0.88	0.44
query29	12.54	3.91	3.26
query30	0.25	0.09	0.06
query31	2.85	0.60	0.40
query32	3.22	0.56	0.48
query33	3.00	3.01	3.18
query34	15.79	5.18	4.54
query35	4.57	4.57	4.58
query36	0.68	0.51	0.50
query37	0.09	0.06	0.06
query38	0.06	0.04	0.04
query39	0.03	0.03	0.02
query40	0.18	0.13	0.12
query41	0.08	0.03	0.02
query42	0.03	0.02	0.02
query43	0.03	0.03	0.03
Total cold run time: 105.08 s
Total hot run time: 31.27 s

@TangSiyang2001 TangSiyang2001 force-pushed the garbage-collection-after-sc-failed branch from aeae75d to 7f8aa77 Compare March 6, 2025 07:49
@TangSiyang2001
Copy link
Collaborator Author

run buildall

@TangSiyang2001 TangSiyang2001 marked this pull request as ready for review March 6, 2025 08:30
@doris-robot
Copy link

TPC-H: Total hot run time: 32791 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 529f96438fdbd6d73a456b470eec5840da597541, data reload: false

------ Round 1 ----------------------------------
q1	17587	5275	5115	5115
q2	2057	301	171	171
q3	10425	1299	765	765
q4	10270	1037	531	531
q5	8214	2351	2359	2351
q6	189	165	132	132
q7	923	758	614	614
q8	9308	1292	1101	1101
q9	4877	4707	4830	4707
q10	6824	2325	1973	1973
q11	474	278	268	268
q12	343	347	223	223
q13	17772	3723	3077	3077
q14	228	239	217	217
q15	535	494	496	494
q16	622	612	583	583
q17	584	860	349	349
q18	6805	6521	6440	6440
q19	1768	954	553	553
q20	317	324	188	188
q21	2798	2215	1986	1986
q22	1113	994	953	953
Total cold run time: 104033 ms
Total hot run time: 32791 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5222	5152	5123	5123
q2	238	327	241	241
q3	2178	2680	2287	2287
q4	1413	1822	1394	1394
q5	4268	4131	4175	4131
q6	210	162	123	123
q7	1972	1961	1781	1781
q8	2603	2558	2537	2537
q9	7318	7243	7235	7235
q10	3061	3205	2799	2799
q11	572	515	499	499
q12	679	772	593	593
q13	3483	3931	3279	3279
q14	292	304	276	276
q15	551	493	500	493
q16	643	691	684	684
q17	1148	1586	1372	1372
q18	7781	7662	7586	7586
q19	831	839	844	839
q20	2002	2103	1893	1893
q21	5567	4830	4707	4707
q22	1119	1061	1035	1035
Total cold run time: 53151 ms
Total hot run time: 50907 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 192777 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 529f96438fdbd6d73a456b470eec5840da597541, data reload: false

query1	1439	1063	1032	1032
query2	6138	1878	1830	1830
query3	10979	4696	4388	4388
query4	54299	25530	22959	22959
query5	4961	607	530	530
query6	340	215	207	207
query7	4870	513	299	299
query8	321	258	240	240
query9	5536	2581	2571	2571
query10	437	319	283	283
query11	15432	15213	15233	15213
query12	162	115	110	110
query13	1058	539	403	403
query14	10455	6832	7221	6832
query15	220	203	176	176
query16	7104	696	481	481
query17	1089	759	586	586
query18	1552	413	326	326
query19	211	206	182	182
query20	139	137	131	131
query21	252	126	103	103
query22	4535	4433	4545	4433
query23	33890	33443	33476	33443
query24	5788	2473	2471	2471
query25	474	447	397	397
query26	708	278	173	173
query27	1881	503	352	352
query28	2757	2461	2435	2435
query29	598	574	441	441
query30	279	232	202	202
query31	902	871	780	780
query32	78	67	63	63
query33	478	351	310	310
query34	794	907	523	523
query35	851	847	792	792
query36	972	1013	915	915
query37	125	107	77	77
query38	4377	4290	4235	4235
query39	1502	1442	1444	1442
query40	208	119	102	102
query41	54	55	51	51
query42	131	107	110	107
query43	502	531	479	479
query44	1352	848	841	841
query45	185	171	168	168
query46	860	1053	671	671
query47	1814	1847	1755	1755
query48	392	412	307	307
query49	692	513	433	433
query50	705	753	438	438
query51	4360	4306	4269	4269
query52	111	109	97	97
query53	241	274	195	195
query54	492	506	439	439
query55	83	81	85	81
query56	266	276	257	257
query57	1168	1199	1086	1086
query58	252	252	239	239
query59	2710	2735	2692	2692
query60	292	284	258	258
query61	124	147	115	115
query62	752	736	682	682
query63	245	196	202	196
query64	1984	1051	689	689
query65	4522	4446	4499	4446
query66	755	385	317	317
query67	15697	15824	15251	15251
query68	6909	887	526	526
query69	557	302	275	275
query70	1214	1124	1106	1106
query71	480	303	264	264
query72	5875	3599	3866	3599
query73	1496	769	358	358
query74	8908	9165	8942	8942
query75	3962	3161	2748	2748
query76	4193	1196	797	797
query77	659	386	290	290
query78	10166	10123	9274	9274
query79	2502	828	604	604
query80	638	533	456	456
query81	496	251	223	223
query82	622	127	100	100
query83	172	168	158	158
query84	294	93	75	75
query85	754	355	306	306
query86	358	300	274	274
query87	4517	4534	4306	4306
query88	3821	2288	2265	2265
query89	415	313	286	286
query90	1804	223	216	216
query91	145	143	112	112
query92	78	66	59	59
query93	1831	1064	580	580
query94	653	406	301	301
query95	352	268	264	264
query96	491	566	266	266
query97	3406	3425	3258	3258
query98	226	204	217	204
query99	1685	1374	1258	1258
Total cold run time: 298611 ms
Total hot run time: 192777 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.11 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 529f96438fdbd6d73a456b470eec5840da597541, data reload: false

query1	0.03	0.03	0.03
query2	0.07	0.03	0.03
query3	0.24	0.06	0.07
query4	1.60	0.10	0.10
query5	0.57	0.55	0.57
query6	1.21	0.72	0.73
query7	0.02	0.02	0.02
query8	0.04	0.04	0.03
query9	0.62	0.52	0.52
query10	0.58	0.58	0.57
query11	0.16	0.11	0.11
query12	0.14	0.12	0.12
query13	0.62	0.60	0.61
query14	2.81	2.68	2.69
query15	0.93	0.86	0.85
query16	0.39	0.39	0.39
query17	1.06	1.05	1.06
query18	0.22	0.20	0.20
query19	1.92	1.80	1.98
query20	0.01	0.01	0.01
query21	15.35	0.90	0.53
query22	0.76	1.27	0.68
query23	14.80	1.35	0.62
query24	6.60	1.53	0.86
query25	0.49	0.25	0.21
query26	0.53	0.16	0.14
query27	0.06	0.06	0.06
query28	9.53	0.91	0.44
query29	12.55	4.07	3.37
query30	0.25	0.09	0.06
query31	2.81	0.59	0.38
query32	3.23	0.56	0.48
query33	3.04	3.05	3.02
query34	15.79	5.08	4.52
query35	4.59	4.57	4.51
query36	0.66	0.50	0.47
query37	0.10	0.07	0.06
query38	0.06	0.04	0.04
query39	0.03	0.03	0.02
query40	0.16	0.13	0.14
query41	0.08	0.02	0.02
query42	0.04	0.02	0.02
query43	0.04	0.04	0.03
Total cold run time: 104.79 s
Total hot run time: 31.11 s

@TangSiyang2001 TangSiyang2001 force-pushed the garbage-collection-after-sc-failed branch from 529f964 to 60e0782 Compare March 6, 2025 09:55
@TangSiyang2001
Copy link
Collaborator Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 32555 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 6edd84f09867014ad548a2e05b891d959941bcd1, data reload: false

------ Round 1 ----------------------------------
q1	17634	5165	5099	5099
q2	2051	302	190	190
q3	10379	1349	726	726
q4	10227	1023	530	530
q5	7499	2443	2360	2360
q6	190	169	137	137
q7	904	756	598	598
q8	9297	1314	1105	1105
q9	5055	4661	4695	4661
q10	6827	2324	1894	1894
q11	478	277	272	272
q12	347	350	225	225
q13	17790	3747	3131	3131
q14	231	226	208	208
q15	549	505	501	501
q16	631	616	599	599
q17	588	878	354	354
q18	7438	6424	6299	6299
q19	1672	966	554	554
q20	321	331	196	196
q21	2808	2193	1953	1953
q22	1051	1028	963	963
Total cold run time: 103967 ms
Total hot run time: 32555 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5215	5127	5161	5127
q2	241	352	235	235
q3	2170	2703	2297	2297
q4	1463	1892	1383	1383
q5	4265	4205	4178	4178
q6	207	168	124	124
q7	1941	1959	1780	1780
q8	2667	2795	2802	2795
q9	7177	7152	7127	7127
q10	3002	3207	2783	2783
q11	592	517	495	495
q12	694	756	607	607
q13	3564	3942	3315	3315
q14	280	321	272	272
q15	543	498	517	498
q16	657	684	649	649
q17	1165	1640	1314	1314
q18	7752	7538	7449	7449
q19	846	885	1069	885
q20	1977	2017	1868	1868
q21	5395	5024	4855	4855
q22	1161	1039	1009	1009
Total cold run time: 52974 ms
Total hot run time: 51045 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 185474 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 6edd84f09867014ad548a2e05b891d959941bcd1, data reload: false

query1	979	409	406	406
query2	6521	1974	1935	1935
query3	6795	209	208	208
query4	26770	23729	22903	22903
query5	4324	670	501	501
query6	321	207	211	207
query7	4615	500	297	297
query8	292	249	252	249
query9	8617	2563	2578	2563
query10	494	331	258	258
query11	15710	15206	14900	14900
query12	156	117	104	104
query13	1679	527	415	415
query14	8604	6212	6212	6212
query15	214	200	180	180
query16	7141	640	480	480
query17	1210	721	581	581
query18	1957	408	311	311
query19	198	191	170	170
query20	125	122	118	118
query21	208	128	106	106
query22	4098	4343	4114	4114
query23	34105	33205	32970	32970
query24	8464	2364	2400	2364
query25	532	482	386	386
query26	1226	274	156	156
query27	2713	511	328	328
query28	4275	2427	2417	2417
query29	735	542	423	423
query30	286	221	192	192
query31	967	840	775	775
query32	77	64	65	64
query33	567	350	308	308
query34	766	874	506	506
query35	803	796	732	732
query36	1016	973	885	885
query37	115	102	79	79
query38	4101	4056	4128	4056
query39	1456	1426	1380	1380
query40	222	114	106	106
query41	54	52	50	50
query42	126	103	112	103
query43	530	522	489	489
query44	1370	802	802	802
query45	177	172	163	163
query46	853	1023	647	647
query47	1743	1793	1701	1701
query48	381	417	302	302
query49	788	497	410	410
query50	690	745	395	395
query51	4192	4204	4168	4168
query52	111	112	97	97
query53	238	255	186	186
query54	490	489	416	416
query55	80	86	83	83
query56	265	284	250	250
query57	1133	1150	1073	1073
query58	246	236	256	236
query59	2728	2933	2766	2766
query60	285	277	285	277
query61	123	122	113	113
query62	785	724	681	681
query63	240	186	196	186
query64	4268	985	661	661
query65	4408	4370	4366	4366
query66	1125	413	317	317
query67	15588	15611	15470	15470
query68	7820	881	529	529
query69	481	304	270	270
query70	1218	1129	1124	1124
query71	484	295	261	261
query72	5660	3611	3794	3611
query73	777	758	363	363
query74	9259	9161	8782	8782
query75	3853	3182	2732	2732
query76	3639	1223	771	771
query77	788	386	295	295
query78	10245	10062	9264	9264
query79	2688	837	595	595
query80	630	524	444	444
query81	502	259	221	221
query82	498	125	97	97
query83	179	181	157	157
query84	245	98	77	77
query85	870	362	300	300
query86	384	302	280	280
query87	4365	4554	4358	4358
query88	3970	2299	2300	2299
query89	390	319	347	319
query90	1866	229	222	222
query91	141	132	109	109
query92	81	59	60	59
query93	2283	1018	583	583
query94	652	415	308	308
query95	357	266	260	260
query96	483	554	278	278
query97	3300	3426	3288	3288
query98	254	200	193	193
query99	1361	1380	1272	1272
Total cold run time: 276194 ms
Total hot run time: 185474 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.14 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 6edd84f09867014ad548a2e05b891d959941bcd1, data reload: false

query1	0.04	0.04	0.03
query2	0.06	0.03	0.03
query3	0.23	0.07	0.06
query4	1.61	0.10	0.10
query5	0.55	0.57	0.55
query6	1.22	0.72	0.73
query7	0.02	0.02	0.01
query8	0.03	0.03	0.03
query9	0.60	0.55	0.52
query10	0.56	0.58	0.56
query11	0.15	0.10	0.11
query12	0.14	0.11	0.11
query13	0.63	0.60	0.60
query14	2.66	2.84	2.80
query15	0.94	0.87	0.86
query16	0.39	0.39	0.38
query17	1.02	1.00	1.02
query18	0.22	0.20	0.20
query19	1.90	1.82	1.97
query20	0.02	0.01	0.01
query21	15.35	0.87	0.55
query22	0.76	1.19	0.64
query23	14.95	1.36	0.59
query24	6.69	2.32	0.88
query25	0.53	0.25	0.19
query26	0.54	0.16	0.14
query27	0.05	0.05	0.05
query28	10.71	0.87	0.44
query29	12.63	3.95	3.33
query30	0.25	0.09	0.07
query31	2.83	0.59	0.39
query32	3.22	0.54	0.46
query33	3.03	3.07	3.11
query34	15.66	5.12	4.56
query35	4.48	4.48	4.50
query36	0.66	0.51	0.48
query37	0.09	0.07	0.06
query38	0.05	0.04	0.04
query39	0.03	0.03	0.02
query40	0.17	0.14	0.14
query41	0.08	0.02	0.02
query42	0.03	0.03	0.02
query43	0.03	0.03	0.03
Total cold run time: 105.81 s
Total hot run time: 31.14 s

@hello-stephen
Copy link
Contributor

BE UT Coverage Report

Increment line coverage 1.54% (1/65) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 46.82% (12522/26747)
Line Coverage 36.41% (106701/293059)
Region Coverage 35.45% (54500/153718)
Branch Coverage 30.81% (27400/88930)

@TangSiyang2001
Copy link
Collaborator Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 32986 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 6f38897d3dd66c2979f053911b3a9145c5c1c44e, data reload: false

------ Round 1 ----------------------------------
q1	17601	5200	5098	5098
q2	2050	291	164	164
q3	10423	1303	789	789
q4	10193	1033	529	529
q5	7509	2364	2418	2364
q6	178	175	133	133
q7	914	753	627	627
q8	9300	1318	1149	1149
q9	4933	4865	4747	4747
q10	6842	2327	1887	1887
q11	501	276	264	264
q12	349	353	225	225
q13	17770	3722	3178	3178
q14	238	238	211	211
q15	522	486	477	477
q16	658	634	597	597
q17	574	859	365	365
q18	6762	6567	6413	6413
q19	1985	966	604	604
q20	326	333	195	195
q21	3024	2141	1983	1983
q22	1043	1004	987	987
Total cold run time: 103695 ms
Total hot run time: 32986 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5247	5191	5134	5134
q2	241	326	230	230
q3	2199	2694	2345	2345
q4	1446	1842	1390	1390
q5	4263	4169	4245	4169
q6	203	171	125	125
q7	1953	1938	1781	1781
q8	2661	2666	2663	2663
q9	7298	7222	7182	7182
q10	3064	3227	2756	2756
q11	575	517	501	501
q12	642	752	590	590
q13	3536	3969	3288	3288
q14	280	303	277	277
q15	537	511	501	501
q16	670	692	655	655
q17	1145	1629	1321	1321
q18	7796	7504	7552	7504
q19	870	865	1044	865
q20	1979	2016	1869	1869
q21	5531	5057	4652	4652
q22	1052	1021	1034	1021
Total cold run time: 53188 ms
Total hot run time: 50819 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 191522 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 6f38897d3dd66c2979f053911b3a9145c5c1c44e, data reload: false

query1	1425	1028	998	998
query2	6196	1987	1946	1946
query3	11025	4420	4635	4420
query4	53524	25732	22955	22955
query5	5130	518	473	473
query6	328	190	184	184
query7	4937	516	283	283
query8	324	233	225	225
query9	5801	2520	2522	2520
query10	417	325	253	253
query11	15094	15087	14996	14996
query12	154	109	105	105
query13	1068	512	397	397
query14	10500	6786	6678	6678
query15	194	198	181	181
query16	7070	639	465	465
query17	1098	760	578	578
query18	1590	420	334	334
query19	207	199	175	175
query20	129	131	135	131
query21	214	128	110	110
query22	4438	4516	4484	4484
query23	33822	33308	33189	33189
query24	5645	2473	2428	2428
query25	461	483	400	400
query26	684	282	164	164
query27	1728	502	338	338
query28	2855	2461	2420	2420
query29	552	561	433	433
query30	269	215	186	186
query31	886	846	759	759
query32	69	61	61	61
query33	447	366	304	304
query34	828	848	535	535
query35	823	907	780	780
query36	986	1003	918	918
query37	119	99	79	79
query38	4389	4278	4152	4152
query39	1509	1452	1430	1430
query40	208	117	107	107
query41	53	52	49	49
query42	119	111	112	111
query43	501	518	485	485
query44	1362	795	804	795
query45	182	172	167	167
query46	847	1034	647	647
query47	1874	1923	1793	1793
query48	373	414	296	296
query49	700	515	391	391
query50	698	750	421	421
query51	4300	4334	4212	4212
query52	112	102	96	96
query53	230	272	188	188
query54	484	518	434	434
query55	84	88	87	87
query56	290	270	287	270
query57	1179	1172	1095	1095
query58	241	241	245	241
query59	2714	2906	2776	2776
query60	313	286	287	286
query61	147	139	143	139
query62	749	745	701	701
query63	236	201	199	199
query64	1640	1080	689	689
query65	4543	4432	4365	4365
query66	721	398	293	293
query67	15819	15766	15217	15217
query68	7124	874	498	498
query69	541	370	266	266
query70	1216	1145	1107	1107
query71	514	310	264	264
query72	5752	3627	3824	3627
query73	1420	748	345	345
query74	8958	9154	8796	8796
query75	3751	3162	2718	2718
query76	4210	1185	775	775
query77	596	393	276	276
query78	10088	10008	9307	9307
query79	2647	827	589	589
query80	640	589	458	458
query81	478	260	216	216
query82	672	127	92	92
query83	299	170	153	153
query84	290	99	71	71
query85	804	364	319	319
query86	371	294	284	284
query87	4528	4411	4259	4259
query88	3471	2243	2237	2237
query89	408	314	284	284
query90	1954	209	216	209
query91	146	234	113	113
query92	79	58	60	58
query93	1497	1059	569	569
query94	678	410	291	291
query95	345	269	264	264
query96	493	564	280	280
query97	3326	3357	3285	3285
query98	223	204	200	200
query99	1448	1374	1295	1295
Total cold run time: 296621 ms
Total hot run time: 191522 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.22 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 6f38897d3dd66c2979f053911b3a9145c5c1c44e, data reload: false

query1	0.03	0.03	0.03
query2	0.07	0.03	0.04
query3	0.24	0.06	0.06
query4	1.63	0.11	0.10
query5	0.55	0.55	0.55
query6	1.21	0.71	0.72
query7	0.03	0.02	0.02
query8	0.03	0.03	0.03
query9	0.60	0.52	0.50
query10	0.57	0.60	0.58
query11	0.15	0.11	0.10
query12	0.14	0.11	0.11
query13	0.62	0.61	0.60
query14	2.79	2.79	2.72
query15	0.93	0.85	0.85
query16	0.38	0.39	0.38
query17	1.08	1.03	1.02
query18	0.21	0.20	0.20
query19	1.92	1.80	1.98
query20	0.01	0.01	0.01
query21	15.37	0.88	0.55
query22	0.76	1.10	0.70
query23	15.00	1.38	0.63
query24	6.63	1.09	1.97
query25	0.52	0.26	0.12
query26	0.53	0.17	0.13
query27	0.05	0.06	0.05
query28	10.04	0.82	0.44
query29	12.59	3.99	3.34
query30	0.25	0.09	0.06
query31	2.86	0.58	0.39
query32	3.23	0.55	0.46
query33	3.07	3.14	3.02
query34	15.80	5.08	4.51
query35	4.55	4.58	4.53
query36	0.66	0.49	0.48
query37	0.09	0.06	0.07
query38	0.06	0.04	0.04
query39	0.03	0.03	0.02
query40	0.18	0.13	0.13
query41	0.08	0.02	0.02
query42	0.03	0.02	0.02
query43	0.03	0.04	0.03
Total cold run time: 105.6 s
Total hot run time: 31.22 s

xinyiZzz
xinyiZzz approved these changes Mar 10, 2025
View reviewed changes
Copy link
Contributor

@xinyiZzz xinyiZzz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Mar 10, 2025
@github-actions GitHub Actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions GitHub Actions
Copy link
Contributor

PR approved by anyone and no changes requested.

dataroaring
dataroaring approved these changes Mar 10, 2025
View reviewed changes
Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@TangSiyang2001
Copy link
Collaborator Author

run p0

@dataroaring dataroaring merged commit 08549b7 into apache:master Mar 11, 2025
25 of 27 checks passed
github-actions bot pushed a commit that referenced this pull request Mar 11, 2025
@TangSiyang2001
[enhancement](schema-change) Cloud schema change do clean up when job…
fb563ea 
… failed (#48426)

If schema change failed, clear output rowset directly from file cache.
@github-actions github-actions bot mentioned this pull request Mar 11, 2025
Merged
dataroaring pushed a commit that referenced this pull request Mar 11, 2025
@github-actions @TangSiyang2001
branch-3.0: [enhancement](schema-change) Cloud schema change do clean…
51ccdfa 
… up when job failed #48426 (#48897)

Cherry-picked from #48426

Co-authored-by: Siyang Tang <tangsiyang2001@foxmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@xinyiZzz xinyiZzz xinyiZzz approved these changes

@dataroaring dataroaring dataroaring approved these changes

@gavinchou gavinchou Awaiting requested review from gavinchou gavinchou is a code owner

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by one committer. dev/3.0.5-merged reviewed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@TangSiyang2001 @Thearas @doris-robot @hello-stephen @xinyiZzz @dataroaring