[Feat](nereids)when dealing insert into stmt with empty table source, fe returns directly #34418

feiniaofeiafei · 2024-05-06T07:03:09Z

When a LogicalOlapScan has no partitions, transform it to a LogicalEmptyRelation.
When dealing insert into stmt with empty table source, fe returns directly.

doris-robot · 2024-05-06T07:03:15Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

gavinchou · 2024-05-06T07:14:45Z

regression-test/suites/load_p0/insert/insert_select_empty_table.groovy

+        """
+
+    sql """
+        create table insert_select_empty_table2(pk int, a int, b int) distributed by hash(pk) buckets 10


consider adding more cases, such as join, union and etc.

feiniaofeiafei · 2024-05-06T11:09:06Z

run buildall

doris-robot · 2024-05-06T11:45:30Z

TPC-H: Total hot run time: 41682 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit c1bc4a4c310ccc4065e31dd8094ea80225c83827, data reload: false

------ Round 1 ----------------------------------
q1	17872	4311	4210	4210
q2	2024	188	185	185
q3	10464	1249	1151	1151
q4	10204	809	858	809
q5	7484	2651	2712	2651
q6	226	133	140	133
q7	1055	616	612	612
q8	9234	2155	2108	2108
q9	9111	6750	6730	6730
q10	9886	3851	3974	3851
q11	444	246	241	241
q12	439	223	215	215
q13	17215	3132	3164	3132
q14	282	230	227	227
q15	529	476	475	475
q16	485	379	381	379
q17	952	661	724	661
q18	8289	7714	7731	7714
q19	2995	1533	1566	1533
q20	644	305	319	305
q21	5252	4079	4208	4079
q22	365	285	281	281
Total cold run time: 115451 ms
Total hot run time: 41682 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4533	4484	4396	4396
q2	396	267	276	267
q3	3148	2938	2870	2870
q4	2013	1677	1680	1677
q5	5363	5504	5479	5479
q6	218	125	129	125
q7	2338	1964	1967	1964
q8	3274	3406	3396	3396
q9	8762	8862	8821	8821
q10	4042	3759	3835	3759
q11	606	488	491	488
q12	766	637	632	632
q13	15965	3173	3147	3147
q14	298	287	266	266
q15	522	485	486	485
q16	475	458	441	441
q17	1820	1452	1445	1445
q18	7707	7521	7369	7369
q19	1976	1574	1559	1559
q20	1968	1744	1769	1744
q21	10294	4980	5021	4980
q22	574	475	489	475
Total cold run time: 77058 ms
Total hot run time: 55785 ms

doris-robot · 2024-05-06T11:57:16Z

TPC-DS: Total hot run time: 184747 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit c1bc4a4c310ccc4065e31dd8094ea80225c83827, data reload: false

query1	917	358	362	358
query2	6686	2430	2321	2321
query3	6643	211	212	211
query4	23696	21090	21204	21090
query5	4148	437	398	398
query6	252	190	168	168
query7	4595	282	286	282
query8	244	198	202	198
query9	8450	2361	2336	2336
query10	444	248	245	245
query11	14767	14114	14055	14055
query12	135	91	86	86
query13	1647	377	378	377
query14	9678	7527	6738	6738
query15	246	167	174	167
query16	8080	268	263	263
query17	1725	571	548	548
query18	2112	282	275	275
query19	205	156	153	153
query20	94	90	93	90
query21	194	131	128	128
query22	5034	4865	4803	4803
query23	33713	33227	33078	33078
query24	6730	2837	2908	2837
query25	549	381	387	381
query26	700	155	151	151
query27	2062	318	316	316
query28	3839	2040	2035	2035
query29	875	611	635	611
query30	236	167	156	156
query31	969	755	734	734
query32	100	55	55	55
query33	496	251	251	251
query34	876	495	488	488
query35	760	665	675	665
query36	1086	875	911	875
query37	109	69	66	66
query38	3128	2996	3026	2996
query39	1597	1530	1551	1530
query40	204	128	130	128
query41	44	41	44	41
query42	103	96	97	96
query43	551	547	548	547
query44	1063	732	727	727
query45	264	253	264	253
query46	1059	705	720	705
query47	1910	1876	1885	1876
query48	366	301	318	301
query49	786	464	389	389
query50	778	374	388	374
query51	6691	6695	6702	6695
query52	93	98	86	86
query53	351	278	275	275
query54	259	228	233	228
query55	74	70	74	70
query56	239	218	213	213
query57	1207	1143	1130	1130
query58	215	201	193	193
query59	3400	3439	3173	3173
query60	259	253	230	230
query61	100	87	88	87
query62	574	478	447	447
query63	302	274	283	274
query64	8241	7182	7207	7182
query65	3081	3052	3037	3037
query66	780	342	335	335
query67	15736	15302	15001	15001
query68	9590	537	541	537
query69	563	305	309	305
query70	1365	1056	1161	1056
query71	490	266	267	266
query72	8207	2532	2342	2342
query73	1618	325	337	325
query74	6609	6055	6055	6055
query75	4658	2638	2606	2606
query76	5164	937	1033	937
query77	689	263	261	261
query78	10875	10175	10153	10153
query79	11810	515	521	515
query80	1826	467	423	423
query81	503	219	219	219
query82	230	94	92	92
query83	202	163	164	163
query84	262	81	83	81
query85	950	265	262	262
query86	347	271	292	271
query87	3317	3052	3099	3052
query88	5470	2322	2313	2313
query89	514	376	369	369
query90	2377	180	180	180
query91	126	95	94	94
query92	64	46	47	46
query93	7019	514	493	493
query94	1591	221	176	176
query95	400	302	298	298
query96	606	261	261	261
query97	3136	2931	2915	2915
query98	232	217	220	217
query99	1113	856	872	856
Total cold run time: 298470 ms
Total hot run time: 184747 ms

...c/main/java/org/apache/doris/nereids/trees/plans/commands/insert/InsertIntoTableCommand.java

feiniaofeiafei · 2024-05-08T06:54:43Z

run buildall

feiniaofeiafei · 2024-05-08T10:14:09Z

run buildall

github-actions · 2024-05-08T13:42:10Z

`sh-checker report`

To get the full details, please check in the job output.

shellcheck errors


'shellcheck ' returned error 1 finding the following syntactical issues:

----------

In regression-test/suites/nereids_tpcds_shape_sf100_p0/code3.sh line 4:
if [ $# -eq 0 ]; then
   ^----------^ SC2292 (style): Prefer [[ ]] over [ ] for tests in Bash/Ksh.

Did you mean: 
if [[ $# -eq 0 ]]; then


In regression-test/suites/nereids_tpcds_shape_sf100_p0/code3.sh line 13:
if [ ! -f "$input_file" ]; then
   ^--------------------^ SC2292 (style): Prefer [[ ]] over [ ] for tests in Bash/Ksh.
           ^---------^ SC2250 (style): Prefer putting braces around variable references even when not strictly required.

Did you mean: 
if [[ ! -f "${input_file}" ]]; then


In regression-test/suites/nereids_tpcds_shape_sf100_p0/code3.sh line 14:
    echo "File '$input_file' does not exist."
                ^---------^ SC2250 (style): Prefer putting braces around variable references even when not strictly required.

Did you mean: 
    echo "File '${input_file}' does not exist."


In regression-test/suites/nereids_tpcds_shape_sf100_p0/code3.sh line 19:
if grep -q "sql \"set disable_nereids_rules=PRUNE_EMPTY_PARTITION\"" "$input_file"; then
                                                                      ^---------^ SC2250 (style): Prefer putting braces around variable references even when not strictly required.

Did you mean: 
if grep -q "sql \"set disable_nereids_rules=PRUNE_EMPTY_PARTITION\"" "${input_file}"; then


In regression-test/suites/nereids_tpcds_shape_sf100_p0/code3.sh line 23:
    awk '/sql '\''SET enable_pipeline_engine = true'\''/ { print; print "    sql \"set disable_nereids_rules=PRUNE_EMPTY_PARTITION\""; next } 1' "$input_file" > temp_file && mv temp_file "$input_file"
                                                                                                                                                  ^---------^ SC2250 (style): Prefer putting braces around variable references even when not strictly required.
                                                                                                                                                                                            ^---------^ SC2250 (style): Prefer putting braces around variable references even when not strictly required.

Did you mean: 
    awk '/sql '\''SET enable_pipeline_engine = true'\''/ { print; print "    sql \"set disable_nereids_rules=PRUNE_EMPTY_PARTITION\""; next } 1' "${input_file}" > temp_file && mv temp_file "${input_file}"


In regression-test/suites/nereids_tpcds_shape_sf100_p0/traverse_dir.sh line 4:
if [ $# -eq 0 ]; then
   ^----------^ SC2292 (style): Prefer [[ ]] over [ ] for tests in Bash/Ksh.

Did you mean: 
if [[ $# -eq 0 ]]; then


In regression-test/suites/nereids_tpcds_shape_sf100_p0/traverse_dir.sh line 13:
if [ ! -d "$directory" ]; then
   ^-------------------^ SC2292 (style): Prefer [[ ]] over [ ] for tests in Bash/Ksh.
           ^--------^ SC2250 (style): Prefer putting braces around variable references even when not strictly required.

Did you mean: 
if [[ ! -d "${directory}" ]]; then


In regression-test/suites/nereids_tpcds_shape_sf100_p0/traverse_dir.sh line 14:
    echo "Directory '$directory' does not exist."
                     ^--------^ SC2250 (style): Prefer putting braces around variable references even when not strictly required.

Did you mean: 
    echo "Directory '${directory}' does not exist."


In regression-test/suites/nereids_tpcds_shape_sf100_p0/traverse_dir.sh line 19:
for file in "$directory"/*; do
             ^--------^ SC2250 (style): Prefer putting braces around variable references even when not strictly required.

Did you mean: 
for file in "${directory}"/*; do


In regression-test/suites/nereids_tpcds_shape_sf100_p0/traverse_dir.sh line 21:
    if [ -f "$file" ]; then
       ^------------^ SC2292 (style): Prefer [[ ]] over [ ] for tests in Bash/Ksh.
             ^---^ SC2250 (style): Prefer putting braces around variable references even when not strictly required.

Did you mean: 
    if [[ -f "${file}" ]]; then


In regression-test/suites/nereids_tpcds_shape_sf100_p0/traverse_dir.sh line 23:
        echo "Processing file: $file"
                               ^---^ SC2250 (style): Prefer putting braces around variable references even when not strictly required.

Did you mean: 
        echo "Processing file: ${file}"


In regression-test/suites/nereids_tpcds_shape_sf100_p0/traverse_dir.sh line 25:
        bash code3.sh "$file"  # 调用之前的脚本对单个文件执行操作
                       ^---^ SC2250 (style): Prefer putting braces around variable references even when not strictly required.

Did you mean: 
        bash code3.sh "${file}"  # 调用之前的脚本对单个文件执行操作

For more information:
  https://www.shellcheck.net/wiki/SC2250 -- Prefer putting braces around vari...
  https://www.shellcheck.net/wiki/SC2292 -- Prefer [[ ]] over [ ] for tests i...
----------

You can address the above issues in one of three ways:
1. Manually correct the issue in the offending shell script;
2. Disable specific issues by adding the comment:
  # shellcheck disable=NNNN
above the line that contains the issue, where NNNN is the error code;
3. Add '-e NNNN' to the SHELLCHECK_OPTS setting in your .yml action file.

shfmt errors


'shfmt ' returned error 1 finding the following formatting issues:

----------
--- regression-test/suites/nereids_tpcds_shape_sf100_p0/code3.sh.orig
+++ regression-test/suites/nereids_tpcds_shape_sf100_p0/code3.sh
@@ -20,7 +20,6 @@
     echo "Line already exists. Skipping addition."
 else
     # 使用awk命令在文件中查找并添加行
-    awk '/sql '\''SET enable_pipeline_engine = true'\''/ { print; print "    sql \"set disable_nereids_rules=PRUNE_EMPTY_PARTITION\""; next } 1' "$input_file" > temp_file && mv temp_file "$input_file"
+    awk '/sql '\''SET enable_pipeline_engine = true'\''/ { print; print "    sql \"set disable_nereids_rules=PRUNE_EMPTY_PARTITION\""; next } 1' "$input_file" >temp_file && mv temp_file "$input_file"
     echo "Line added successfully."
 fi
-
--- regression-test/suites/nereids_tpcds_shape_sf100_p0/traverse_dir.sh.orig
+++ regression-test/suites/nereids_tpcds_shape_sf100_p0/traverse_dir.sh
@@ -22,7 +22,6 @@
         # 执行操作
         echo "Processing file: $file"
         # 在这里执行对单个文件的操作
-        bash code3.sh "$file"  # 调用之前的脚本对单个文件执行操作
+        bash code3.sh "$file" # 调用之前的脚本对单个文件执行操作
     fi
 done
-
----------

You can reformat the above files to meet shfmt's requirements by typing:

  shfmt  -w filename

feiniaofeiafei · 2024-05-08T13:45:50Z

run buildall

feiniaofeiafei · 2024-05-09T03:12:51Z

run buildall

feiniaofeiafei · 2024-05-09T03:21:47Z

run buildall

feiniaofeiafei · 2024-05-09T11:51:18Z

run buildall

feiniaofeiafei · 2024-05-09T12:25:41Z

run buildall

doris-robot · 2024-05-09T12:46:13Z

TPC-H: Total hot run time: 39602 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 409aaadc354680421a4eeb48adf8e273a39de6c4, data reload: false

------ Round 1 ----------------------------------
q1	17760	4377	4271	4271
q2	2021	187	192	187
q3	10478	1195	1200	1195
q4	10160	759	840	759
q5	7462	2674	2691	2674
q6	215	133	131	131
q7	996	608	576	576
q8	9214	2071	2052	2052
q9	8738	6519	6463	6463
q10	8945	3763	3678	3678
q11	465	243	242	242
q12	408	215	220	215
q13	17764	2918	2967	2918
q14	268	210	227	210
q15	498	478	475	475
q16	516	387	368	368
q17	959	565	677	565
q18	8074	7306	7435	7306
q19	6828	1540	1503	1503
q20	653	312	307	307
q21	4927	3221	3226	3221
q22	343	287	286	286
Total cold run time: 117692 ms
Total hot run time: 39602 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4392	4235	4200	4200
q2	384	267	262	262
q3	2945	2734	2746	2734
q4	1865	1594	1683	1594
q5	5256	5265	5259	5259
q6	209	127	127	127
q7	2223	1868	1855	1855
q8	3183	3324	3365	3324
q9	8327	8277	8265	8265
q10	3860	3705	3711	3705
q11	594	506	480	480
q12	763	587	582	582
q13	17349	2947	3004	2947
q14	291	283	263	263
q15	521	469	471	469
q16	467	413	419	413
q17	1761	1485	1471	1471
q18	7555	7587	7350	7350
q19	1650	1565	1524	1524
q20	1979	1765	1756	1756
q21	4963	4868	4838	4838
q22	569	503	504	503
Total cold run time: 71106 ms
Total hot run time: 53921 ms

doris-robot · 2024-05-09T12:58:08Z

TPC-DS: Total hot run time: 186386 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 409aaadc354680421a4eeb48adf8e273a39de6c4, data reload: false

query1	903	359	354	354
query2	6475	2349	2340	2340
query3	6664	204	205	204
query4	22525	21459	21207	21207
query5	4179	416	420	416
query6	259	173	172	172
query7	4588	293	302	293
query8	252	200	197	197
query9	8837	2416	2393	2393
query10	441	253	255	253
query11	14706	14277	14314	14277
query12	138	91	87	87
query13	1639	374	363	363
query14	10655	7432	6814	6814
query15	226	174	175	174
query16	7962	260	269	260
query17	1848	542	540	540
query18	2013	288	270	270
query19	204	148	143	143
query20	91	88	84	84
query21	195	130	129	129
query22	5077	4916	4839	4839
query23	34097	33697	33528	33528
query24	11730	2896	2974	2896
query25	680	384	380	380
query26	1803	164	155	155
query27	3146	328	331	328
query28	7771	2056	2051	2051
query29	1108	638	615	615
query30	296	157	162	157
query31	983	766	719	719
query32	95	53	54	53
query33	752	266	255	255
query34	1094	481	479	479
query35	808	691	683	683
query36	1042	945	929	929
query37	278	68	70	68
query38	2924	2798	2758	2758
query39	1676	1565	1575	1565
query40	279	126	129	126
query41	45	41	41	41
query42	106	95	96	95
query43	557	549	515	515
query44	1208	726	730	726
query45	280	256	248	248
query46	1071	717	691	691
query47	1988	1895	1891	1891
query48	375	308	305	305
query49	1211	414	418	414
query50	836	382	392	382
query51	6937	6733	6848	6733
query52	106	89	93	89
query53	346	281	276	276
query54	949	425	422	422
query55	72	71	69	69
query56	234	223	221	221
query57	1243	1142	1138	1138
query58	225	197	194	194
query59	3353	3127	3262	3127
query60	278	233	233	233
query61	89	86	87	86
query62	660	471	468	468
query63	302	278	280	278
query64	9700	7328	7405	7328
query65	3234	3138	3095	3095
query66	1372	344	334	334
query67	15659	14885	15219	14885
query68	8338	554	551	551
query69	536	309	312	309
query70	1177	1084	1090	1084
query71	504	266	267	266
query72	7380	2539	2338	2338
query73	1105	325	322	322
query74	6580	6073	6231	6073
query75	4355	2662	2597	2597
query76	4857	976	905	905
query77	660	259	259	259
query78	10665	10346	10273	10273
query79	7403	509	516	509
query80	1012	437	433	433
query81	492	222	222	222
query82	751	92	90	90
query83	214	165	172	165
query84	264	85	83	83
query85	1407	264	255	255
query86	402	309	290	290
query87	3289	3087	3137	3087
query88	4581	2414	2492	2414
query89	494	384	368	368
query90	2011	184	188	184
query91	120	98	98	98
query92	92	47	47	47
query93	5392	510	516	510
query94	1083	182	185	182
query95	392	308	303	303
query96	599	269	271	269
query97	3172	2954	2991	2954
query98	234	224	216	216
query99	1249	892	934	892
Total cold run time: 303685 ms
Total hot run time: 186386 ms

… the fe return directly

doris-robot · 2024-05-09T16:25:55Z

TPC-H: Total hot run time: 40244 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 4a4b6c04fd8814b47f6bcbbe18ba738abb27a236, data reload: false

------ Round 1 ----------------------------------
q1	17697	4513	4333	4333
q2	2024	186	201	186
q3	10542	1283	1261	1261
q4	10200	829	703	703
q5	7472	2705	2695	2695
q6	224	132	142	132
q7	1078	634	607	607
q8	9308	2169	2137	2137
q9	9409	6802	6658	6658
q10	9187	3849	3692	3692
q11	464	257	244	244
q12	472	221	215	215
q13	17764	2955	2958	2955
q14	247	224	202	202
q15	497	465	449	449
q16	510	393	367	367
q17	976	662	709	662
q18	8164	7401	7378	7378
q19	3291	1544	1514	1514
q20	641	305	311	305
q21	5019	3268	4024	3268
q22	346	281	282	281
Total cold run time: 115532 ms
Total hot run time: 40244 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4329	4275	4230	4230
q2	377	256	247	247
q3	2951	2729	2781	2729
q4	1894	1600	1560	1560
q5	5263	5293	5283	5283
q6	210	126	131	126
q7	2235	1895	1916	1895
q8	3181	3374	3371	3371
q9	8394	8393	8409	8393
q10	3832	3690	3652	3652
q11	558	481	474	474
q12	742	604	606	604
q13	16343	2931	2982	2931
q14	281	264	254	254
q15	520	465	468	465
q16	459	405	404	404
q17	1769	1495	1488	1488
q18	7549	7711	7579	7579
q19	2588	1569	1561	1561
q20	1983	1767	1743	1743
q21	7054	4764	4978	4764
q22	569	462	495	462
Total cold run time: 73081 ms
Total hot run time: 54215 ms

doris-robot · 2024-05-09T16:37:35Z

TPC-DS: Total hot run time: 188303 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 4a4b6c04fd8814b47f6bcbbe18ba738abb27a236, data reload: false

query1	909	373	358	358
query2	6445	2489	2482	2482
query3	6641	224	215	215
query4	23203	21533	21478	21478
query5	4161	427	427	427
query6	282	188	186	186
query7	5172	294	295	294
query8	260	195	185	185
query9	8518	2429	2420	2420
query10	462	261	263	261
query11	15103	14325	14142	14142
query12	153	89	86	86
query13	1641	370	372	370
query14	10676	8415	8141	8141
query15	251	185	169	169
query16	8180	263	278	263
query17	1824	553	543	543
query18	2109	276	265	265
query19	322	151	151	151
query20	90	84	83	83
query21	194	126	124	124
query22	5010	4922	4902	4902
query23	34222	33667	33466	33466
query24	10826	2924	2840	2840
query25	615	357	365	357
query26	1169	157	150	150
query27	2867	314	339	314
query28	7397	2038	2045	2038
query29	845	613	596	596
query30	296	153	150	150
query31	953	752	743	743
query32	95	52	54	52
query33	740	252	246	246
query34	1039	482	491	482
query35	820	707	689	689
query36	1090	887	866	866
query37	130	65	71	65
query38	2907	2769	2757	2757
query39	1606	1538	1534	1534
query40	195	126	123	123
query41	45	42	43	42
query42	106	102	96	96
query43	601	555	563	555
query44	1184	746	727	727
query45	264	251	241	241
query46	1076	726	740	726
query47	1973	1884	1910	1884
query48	371	299	288	288
query49	1099	408	395	395
query50	780	386	395	386
query51	6855	6711	6599	6599
query52	103	90	93	90
query53	354	288	278	278
query54	846	422	434	422
query55	78	72	72	72
query56	246	231	218	218
query57	1194	1155	1184	1155
query58	221	194	194	194
query59	3488	3253	3256	3253
query60	276	270	229	229
query61	86	89	85	85
query62	701	489	454	454
query63	307	283	276	276
query64	8807	7361	7300	7300
query65	3158	3128	3083	3083
query66	1103	335	337	335
query67	15384	15316	15326	15316
query68	4552	547	553	547
query69	491	308	313	308
query70	1166	1112	1116	1112
query71	400	271	267	267
query72	7763	2548	2322	2322
query73	717	325	327	325
query74	6616	6154	6150	6150
query75	3451	2642	2644	2642
query76	2926	1002	983	983
query77	427	266	321	266
query78	10655	10299	10088	10088
query79	2189	517	511	511
query80	967	459	431	431
query81	525	218	216	216
query82	657	99	89	89
query83	244	161	161	161
query84	234	85	86	85
query85	1925	269	260	260
query86	490	332	310	310
query87	3300	3123	3103	3103
query88	4447	2408	2404	2404
query89	478	391	397	391
query90	1983	195	193	193
query91	149	96	97	96
query92	63	50	49	49
query93	2314	519	495	495
query94	1262	182	186	182
query95	405	302	302	302
query96	589	270	265	265
query97	3147	2934	2968	2934
query98	246	224	218	218
query99	1132	927	886	886
Total cold run time: 286034 ms
Total hot run time: 188303 ms

feiniaofeiafei · 2024-05-10T02:40:29Z

run buildall

feiniaofeiafei · 2024-05-10T06:18:25Z

run feut

englefly · 2024-05-11T02:22:28Z

是不是可以直接扩展 EliminateEmptyRelation rule? 所有node直接裁剪了

morrySnow · 2024-05-11T08:12:25Z

fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/info/CreateMTMVInfo.java

@@ -88,7 +88,7 @@
 */
 public class CreateMTMVInfo {
    public static final Logger LOG = LogManager.getLogger(CreateMTMVInfo.class);
-    public static final String MTMV_PLANER_DISABLE_RULES = "OLAP_SCAN_PARTITION_PRUNE";
+    public static final String MTMV_PLANER_DISABLE_RULES = "OLAP_SCAN_PARTITION_PRUNE,PRUNE_EMPTY_PARTITION";


@seawinde disable this rule will lead to mv have more partition than query?

the mv plan used by query rewrite is generated by org.apache.doris.mtmv.MTMVCache#from which runs the same rules as query.
MTMV_PLANER_DISABLE_RULES is only used by mv meta check when create mtmv. it seems has no problems

github-actions · 2024-05-11T08:37:17Z

PR approved by anyone and no changes requested.

github-actions · 2024-05-14T02:56:28Z

PR approved by at least one committer and no changes requested.

…, fe returns directly (apache#34418) When a LogicalOlapScan has no partitions, transform it to a LogicalEmptyRelation. When dealing insert into stmt with empty table source, fe returns directly.

This bug introduced by #34418. Some operators can produce data even if the input is an empty table, such as AGGREGATE. Therefore, it is necessary to check whether sink's child is in an empty relation. Only when the child is in an empty relation can it return directly without performing actual data insertion.

…, fe returns directly (apache#34418) When a LogicalOlapScan has no partitions, transform it to a LogicalEmptyRelation. When dealing insert into stmt with empty table source, fe returns directly.

… fe returns directly (#35333) * [Feat](nereids) when dealing insert into stmt with empty table source, fe returns directly (#34418) When a LogicalOlapScan has no partitions, transform it to a LogicalEmptyRelation. When dealing insert into stmt with empty table source, fe returns directly. * [Fix](nereids) fix when insert into select empty table --------- Co-authored-by: feiniaofeiafei <moailing@selectdb.com>

This bug introduced by #34418. Some operators can produce data even if the input is an empty table, such as AGGREGATE. Therefore, it is necessary to check whether sink's child is in an empty relation. Only when the child is in an empty relation can it return directly without performing actual data insertion.

This bug introduced by apache#34418. Some operators can produce data even if the input is an empty table, such as AGGREGATE. Therefore, it is necessary to check whether sink's child is in an empty relation. Only when the child is in an empty relation can it return directly without performing actual data insertion.

gavinchou reviewed May 6, 2024

View reviewed changes

morrySnow reviewed May 6, 2024

View reviewed changes

...c/main/java/org/apache/doris/nereids/trees/plans/commands/insert/InsertIntoTableCommand.java Outdated Show resolved Hide resolved

feiniaofeiafei force-pushed the insert_select_empty_table branch from d87c9f6 to 05cb2c3 Compare May 8, 2024 06:53

feiniaofeiafei added 12 commits May 9, 2024 23:06

[Feat](nereids)when dealing insert into stmt with empty table source,…

7dbab0b

… the fe return directly

[Feat](nereids)when dealing insert into stmt with empty table source,…

37a21e3

… the fe return directly

modify tests

ef8db6e

modify tests

581cc8f

modify tests

942bb25

modify tests

19794e3

modify regression test

6d0aac0

modify regression test

6f9c9dc

modify regression test

256e618

modify regression test

d98c869

modify regression test

420dfc1

modify regression test

4a4b6c0

feiniaofeiafei force-pushed the insert_select_empty_table branch from 409aaad to 4a4b6c0 Compare May 9, 2024 16:02

modify regression test

4d97407

morrySnow reviewed May 11, 2024

View reviewed changes

seawinde approved these changes May 11, 2024

View reviewed changes

github-actions bot added the reviewed label May 11, 2024

morrySnow approved these changes May 14, 2024

View reviewed changes

github-actions bot added the approved Indicates a PR has been approved by one committer. label May 14, 2024

morrySnow merged commit dae90d3 into apache:master May 14, 2024
29 of 33 checks passed

yiguolei added the 2.1.0-conflict label May 18, 2024

feiniaofeiafei mentioned this pull request May 24, 2024

[Fix](nereids) fix bug when insert into select empty table with agg #35291

Merged

morrySnow added the dev/2.1.x label May 24, 2024

feiniaofeiafei mentioned this pull request May 24, 2024

[Feat](nereids)when dealing insert into stmt with empty table source, fe returns directly #35333

Merged

yiguolei added the dev/2.1.4-merged label May 24, 2024

morrySnow removed the 2.1.0-conflict label May 27, 2024

morrySnow removed the dev/2.1.x label Jun 6, 2024

morningman mentioned this pull request Jun 22, 2024

Release Notes 2.1.4 #35752

Closed

dataroaring added the dev/3.0.0-merged label Aug 5, 2024

TangSiyang2001 mentioned this pull request Aug 6, 2024

[Bug] Insert from empty source will begin an infinite transaction #38956

Closed

3 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Feat](nereids)when dealing insert into stmt with empty table source, fe returns directly #34418

[Feat](nereids)when dealing insert into stmt with empty table source, fe returns directly #34418

feiniaofeiafei commented May 6, 2024 •

edited

Loading

doris-robot commented May 6, 2024

gavinchou May 6, 2024

feiniaofeiafei commented May 6, 2024

doris-robot commented May 6, 2024

doris-robot commented May 6, 2024

feiniaofeiafei commented May 8, 2024

feiniaofeiafei commented May 8, 2024

github-actions bot commented May 8, 2024

feiniaofeiafei commented May 8, 2024

feiniaofeiafei commented May 9, 2024

feiniaofeiafei commented May 9, 2024

feiniaofeiafei commented May 9, 2024

feiniaofeiafei commented May 9, 2024

doris-robot commented May 9, 2024

doris-robot commented May 9, 2024

doris-robot commented May 9, 2024

doris-robot commented May 9, 2024

feiniaofeiafei commented May 10, 2024

feiniaofeiafei commented May 10, 2024

englefly commented May 11, 2024

morrySnow May 11, 2024

seawinde May 11, 2024

github-actions bot commented May 11, 2024

github-actions bot commented May 14, 2024

[Feat](nereids)when dealing insert into stmt with empty table source, fe returns directly #34418

[Feat](nereids)when dealing insert into stmt with empty table source, fe returns directly #34418

Conversation

feiniaofeiafei commented May 6, 2024 • edited Loading

doris-robot commented May 6, 2024

gavinchou May 6, 2024

Choose a reason for hiding this comment

feiniaofeiafei commented May 6, 2024

doris-robot commented May 6, 2024

doris-robot commented May 6, 2024

feiniaofeiafei commented May 8, 2024

feiniaofeiafei commented May 8, 2024

github-actions bot commented May 8, 2024

sh-checker report

feiniaofeiafei commented May 8, 2024

feiniaofeiafei commented May 9, 2024

feiniaofeiafei commented May 9, 2024

feiniaofeiafei commented May 9, 2024

feiniaofeiafei commented May 9, 2024

doris-robot commented May 9, 2024

doris-robot commented May 9, 2024

doris-robot commented May 9, 2024

doris-robot commented May 9, 2024

feiniaofeiafei commented May 10, 2024

feiniaofeiafei commented May 10, 2024

englefly commented May 11, 2024

morrySnow May 11, 2024

Choose a reason for hiding this comment

seawinde May 11, 2024

Choose a reason for hiding this comment

github-actions bot commented May 11, 2024

github-actions bot commented May 14, 2024

feiniaofeiafei commented May 6, 2024 •

edited

Loading

`sh-checker report`