Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[enhancement](compression) prints the specific exception when ZSTD compress fails #39433

Merged
merged 2 commits into from
Aug 20, 2024

Conversation

luwei16
Copy link
Contributor

@luwei16 luwei16 commented Aug 15, 2024

No description provided.

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

dataroaring
dataroaring previously approved these changes Aug 15, 2024
Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Aug 15, 2024
Copy link
Contributor

PR approved by at least one committer and no changes requested.

Copy link
Contributor

PR approved by anyone and no changes requested.

@github-actions github-actions bot removed the approved Indicates a PR has been approved by one committer. label Aug 15, 2024
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

1 similar comment
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@dataroaring
Copy link
Contributor

run buildall

Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Aug 16, 2024
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@doris-robot
Copy link

TPC-H: Total hot run time: 38190 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 3e24e7cd8b1d502a3f9606cd0d80546cee318cda, data reload: false

------ Round 1 ----------------------------------
q1	17932	4497	4308	4308
q2	2066	207	202	202
q3	10465	1315	1154	1154
q4	10168	755	716	716
q5	7798	2876	2822	2822
q6	271	156	160	156
q7	1003	669	652	652
q8	9393	2070	2099	2070
q9	7275	6585	6555	6555
q10	7069	2158	2251	2158
q11	483	266	276	266
q12	526	244	245	244
q13	17939	2992	3061	2992
q14	299	255	254	254
q15	561	516	525	516
q16	547	406	405	405
q17	1015	694	656	656
q18	7475	6875	6810	6810
q19	5621	1058	1053	1053
q20	720	362	359	359
q21	4487	2976	2819	2819
q22	1092	1037	1023	1023
Total cold run time: 114205 ms
Total hot run time: 38190 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4565	4282	4272	4272
q2	405	301	299	299
q3	2850	2648	2649	2648
q4	1943	1680	1695	1680
q5	5663	5736	5649	5649
q6	248	151	148	148
q7	2185	1806	1812	1806
q8	3372	3514	3445	3445
q9	8653	8699	8787	8699
q10	3584	3338	3294	3294
q11	618	544	530	530
q12	846	663	632	632
q13	16329	3146	3161	3146
q14	328	312	312	312
q15	557	539	505	505
q16	502	457	457	457
q17	1841	1569	1536	1536
q18	8283	7958	7790	7790
q19	1750	1540	1699	1540
q20	3485	1938	1918	1918
q21	13544	5314	5483	5314
q22	1270	1071	1075	1071
Total cold run time: 82821 ms
Total hot run time: 56691 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 189737 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 3e24e7cd8b1d502a3f9606cd0d80546cee318cda, data reload: false

query1	910	411	395	395
query2	6728	2001	1933	1933
query3	6428	219	235	219
query4	32230	23766	23352	23352
query5	4287	621	666	621
query6	305	201	200	200
query7	4442	308	309	308
query8	433	360	357	357
query9	8313	2478	2469	2469
query10	465	322	305	305
query11	17896	14998	15052	14998
query12	171	124	122	122
query13	1607	417	415	415
query14	9840	6837	6594	6594
query15	243	184	190	184
query16	7798	541	562	541
query17	1538	590	580	580
query18	1970	331	335	331
query19	217	169	165	165
query20	141	133	128	128
query21	249	142	139	139
query22	4476	4031	4222	4031
query23	34181	33008	33361	33008
query24	11475	2950	2928	2928
query25	665	414	407	407
query26	1246	178	183	178
query27	2602	300	304	300
query28	7467	2131	2108	2108
query29	945	485	435	435
query30	322	183	181	181
query31	1067	852	845	845
query32	115	79	77	77
query33	794	346	341	341
query34	923	494	497	494
query35	861	756	743	743
query36	1102	964	955	955
query37	168	105	101	101
query38	3995	3846	3844	3844
query39	1518	1460	1474	1460
query40	239	156	151	151
query41	138	136	138	136
query42	131	118	117	117
query43	532	492	504	492
query44	1273	783	784	783
query45	225	196	221	196
query46	1130	798	797	797
query47	1959	1874	1879	1874
query48	408	338	338	338
query49	1182	579	574	574
query50	865	468	462	462
query51	6889	6813	6778	6778
query52	127	112	107	107
query53	299	221	227	221
query54	915	520	496	496
query55	90	86	92	86
query56	339	307	321	307
query57	1215	1154	1105	1105
query58	305	292	300	292
query59	2967	2813	2735	2735
query60	365	326	331	326
query61	148	146	147	146
query62	856	705	714	705
query63	263	225	226	225
query64	5252	2407	1860	1860
query65	3209	3207	3187	3187
query66	1125	689	665	665
query67	15490	14842	14865	14842
query68	6556	595	601	595
query69	720	398	335	335
query70	1270	1154	1187	1154
query71	568	303	310	303
query72	7533	2321	2069	2069
query73	850	360	359	359
query74	9270	8749	8838	8749
query75	4811	2774	2792	2774
query76	4734	1037	1069	1037
query77	902	448	438	438
query78	9876	9631	8972	8972
query79	8063	559	558	558
query80	1169	598	608	598
query81	619	261	255	255
query82	746	154	155	154
query83	379	205	207	205
query84	282	94	96	94
query85	1449	344	343	343
query86	467	326	321	321
query87	4459	4247	4270	4247
query88	4958	2533	2525	2525
query89	528	322	323	322
query90	2043	236	232	232
query91	153	124	123	123
query92	85	74	75	74
query93	5963	563	558	558
query94	1003	331	329	329
query95	386	294	301	294
query96	626	291	284	284
query97	3245	3074	3092	3074
query98	252	234	233	233
query99	1590	1353	1334	1334
Total cold run time: 313624 ms
Total hot run time: 189737 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.91 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 3e24e7cd8b1d502a3f9606cd0d80546cee318cda, data reload: false

query1	0.04	0.05	0.04
query2	0.08	0.05	0.05
query3	0.23	0.05	0.05
query4	1.67	0.07	0.08
query5	0.48	0.48	0.48
query6	1.14	0.73	0.73
query7	0.02	0.01	0.02
query8	0.06	0.05	0.05
query9	0.55	0.49	0.50
query10	0.54	0.55	0.54
query11	0.16	0.12	0.12
query12	0.16	0.13	0.13
query13	0.62	0.62	0.58
query14	0.76	0.77	0.78
query15	0.86	0.83	0.82
query16	0.39	0.37	0.37
query17	1.06	1.04	1.05
query18	0.24	0.22	0.23
query19	1.81	1.74	1.80
query20	0.02	0.02	0.01
query21	15.40	0.84	0.65
query22	4.80	7.09	1.65
query23	18.29	1.37	1.30
query24	2.13	0.22	0.23
query25	0.15	0.08	0.08
query26	0.30	0.21	0.21
query27	0.45	0.23	0.23
query28	13.31	1.03	1.01
query29	12.61	3.41	3.35
query30	0.38	0.19	0.20
query31	2.81	0.42	0.40
query32	3.23	0.48	0.49
query33	2.98	2.94	2.94
query34	16.93	4.34	4.43
query35	4.42	4.40	4.42
query36	0.66	0.49	0.49
query37	0.20	0.17	0.16
query38	0.16	0.16	0.16
query39	0.06	0.04	0.05
query40	0.18	0.15	0.13
query41	0.11	0.07	0.06
query42	0.07	0.06	0.06
query43	0.06	0.05	0.05
Total cold run time: 110.58 s
Total hot run time: 30.91 s

@dataroaring dataroaring merged commit eb5cd76 into apache:master Aug 20, 2024
28 of 31 checks passed
hello-stephen pushed a commit that referenced this pull request Aug 20, 2024
## Proposed changes

Issue Number: close #xxx

```
FAILED: src/util/CMakeFiles/Util.dir/block_compression.cpp.o
ccache /var/local/ldb-toolchain/bin/g++ -DBOOST_DATE_TIME_POSIX_TIME_STD_CONFIG -DBOOST_STACKTRACE_USE_BACKTRACE -DBOOST_SYSTEM_NO_DEPRECATED -DBOOST_UUID_RANDOM_PROVIDER_FORCE_POSIX=1 -DBRPC_ENABLE_CPU_PROFILER -DGLOG_CUSTOM_PREFIX_SUPPORT -DHAVE_INTTYPES_H -DHAVE_NETINET_IN_H -DLIBJVM -DS2_USE_GFLAGS -DS2_USE_GLOG -DUSE_AZURE -DUSE_HADOOP_HDFS -DUSE_JEMALLOC -DUSE_MEM_TRACKER -DUSE_UNWIND -D__STDC_FORMAT_MACROS -I/root/doris/be/src/apache-orc/c++/include -I/root/doris/be/build_release/src/apache-orc/c++/include -I/root/doris/be/build_release/src/clucene/src/shared -I/root/doris/be/src/clucene/src/core -I/root/doris/be/src/clucene/src/shared -I/root/doris/be/src/clucene/src/contribs-lib -I/root/doris/be/src -I/root/doris/be/test -I/usr/lib/jvm/jdk-17.0.2/include -I/usr/lib/jvm/jdk-17.0.2/include/linux -isystem /root/doris/be/../common -isystem /root/doris/be/../gensrc/build -isystem /var/local/thirdparty/installed/include -isystem /var/local/thirdparty/installed/gperftools/include -O3 -DNDEBUG -O3 -O3 -DNDEBUG   -D OS_LINUX -g -Wall -Wextra -Werror -pthread -fstrict-aliasing -fno-omit-frame-pointer -Wnon-virtual-dtor -Wno-unused-parameter -Wno-sign-compare -fdiagnostics-color=always -Wno-nonnull -Wno-stringop-overread -Wno-stringop-overflow -Wno-array-bounds -msse4.2 -mavx2 -std=gnu++20 -MD -MT src/util/CMakeFiles/Util.dir/block_compression.cpp.o -MF src/util/CMakeFiles/Util.dir/block_compression.cpp.o.d -o src/util/CMakeFiles/Util.dir/block_compression.cpp.o -c /root/doris/be/src/util/block_compression.cpp
/root/doris/be/src/util/block_compression.cpp: In member function 'virtual doris::Status doris::ZstdBlockCompression::compress(const std::vector<doris::Slice>&, size_t, doris::faststring*)':
/root/doris/be/src/util/block_compression.cpp:956:33: error: catching polymorphic type 'class std::exception' by value [-Werror=catch-value=]
956 |         } catch (std::exception e) {
    |                                 ^
cc1plus: all warnings being treated as errors
```

introduced by #39433
luwei16 added a commit to luwei16/incubator-doris that referenced this pull request Aug 27, 2024
dataroaring pushed a commit that referenced this pull request Aug 27, 2024
dataroaring pushed a commit that referenced this pull request Oct 9, 2024
## Proposed changes

Issue Number: close #xxx

```
FAILED: src/util/CMakeFiles/Util.dir/block_compression.cpp.o
ccache /var/local/ldb-toolchain/bin/g++ -DBOOST_DATE_TIME_POSIX_TIME_STD_CONFIG -DBOOST_STACKTRACE_USE_BACKTRACE -DBOOST_SYSTEM_NO_DEPRECATED -DBOOST_UUID_RANDOM_PROVIDER_FORCE_POSIX=1 -DBRPC_ENABLE_CPU_PROFILER -DGLOG_CUSTOM_PREFIX_SUPPORT -DHAVE_INTTYPES_H -DHAVE_NETINET_IN_H -DLIBJVM -DS2_USE_GFLAGS -DS2_USE_GLOG -DUSE_AZURE -DUSE_HADOOP_HDFS -DUSE_JEMALLOC -DUSE_MEM_TRACKER -DUSE_UNWIND -D__STDC_FORMAT_MACROS -I/root/doris/be/src/apache-orc/c++/include -I/root/doris/be/build_release/src/apache-orc/c++/include -I/root/doris/be/build_release/src/clucene/src/shared -I/root/doris/be/src/clucene/src/core -I/root/doris/be/src/clucene/src/shared -I/root/doris/be/src/clucene/src/contribs-lib -I/root/doris/be/src -I/root/doris/be/test -I/usr/lib/jvm/jdk-17.0.2/include -I/usr/lib/jvm/jdk-17.0.2/include/linux -isystem /root/doris/be/../common -isystem /root/doris/be/../gensrc/build -isystem /var/local/thirdparty/installed/include -isystem /var/local/thirdparty/installed/gperftools/include -O3 -DNDEBUG -O3 -O3 -DNDEBUG   -D OS_LINUX -g -Wall -Wextra -Werror -pthread -fstrict-aliasing -fno-omit-frame-pointer -Wnon-virtual-dtor -Wno-unused-parameter -Wno-sign-compare -fdiagnostics-color=always -Wno-nonnull -Wno-stringop-overread -Wno-stringop-overflow -Wno-array-bounds -msse4.2 -mavx2 -std=gnu++20 -MD -MT src/util/CMakeFiles/Util.dir/block_compression.cpp.o -MF src/util/CMakeFiles/Util.dir/block_compression.cpp.o.d -o src/util/CMakeFiles/Util.dir/block_compression.cpp.o -c /root/doris/be/src/util/block_compression.cpp
/root/doris/be/src/util/block_compression.cpp: In member function 'virtual doris::Status doris::ZstdBlockCompression::compress(const std::vector<doris::Slice>&, size_t, doris::faststring*)':
/root/doris/be/src/util/block_compression.cpp:956:33: error: catching polymorphic type 'class std::exception' by value [-Werror=catch-value=]
956 |         } catch (std::exception e) {
    |                                 ^
cc1plus: all warnings being treated as errors
```

introduced by #39433
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by one committer. dev/2.1.6-merged dev/3.0.2-merged reviewed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants