Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[enhancement](memory) add exception handle logic in memtable flush #39913

Merged
merged 3 commits into from
Aug 27, 2024
Merged

[enhancement](memory) add exception handle logic in memtable flush #39913

merged 3 commits into from
Aug 27, 2024

Conversation

yiguolei
Copy link
Contributor

Proposed changes

Issue Number: close #xxx

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@yiguolei
Copy link
Contributor Author

run buildall

@github-actions GitHub Actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@yiguolei
Copy link
Contributor Author

run buildall

@github-actions GitHub Actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@yiguolei
Copy link
Contributor Author

run buildall

@github-actions GitHub Actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TPC-H: Total hot run time: 38167 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 00852fc06bc133420ed3fdc1f5605b8d51e24516, data reload: false

------ Round 1 ----------------------------------
q1	17957	5695	4473	4473
q2	2726	183	174	174
q3	11110	1271	1162	1162
q4	10544	757	783	757
q5	8351	2853	2773	2773
q6	229	149	137	137
q7	973	630	596	596
q8	9317	2072	2044	2044
q9	7264	6531	6510	6510
q10	6993	2280	2187	2187
q11	454	243	242	242
q12	400	222	214	214
q13	18822	3067	3049	3049
q14	275	240	245	240
q15	530	501	497	497
q16	491	385	390	385
q17	973	705	802	705
q18	7293	6769	6865	6769
q19	1402	1022	1001	1001
q20	713	347	346	346
q21	3928	2908	3054	2908
q22	1126	1031	998	998
Total cold run time: 111871 ms
Total hot run time: 38167 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4371	4290	4254	4254
q2	366	277	269	269
q3	2879	2622	2640	2622
q4	1945	1683	1652	1652
q5	5387	5419	5419	5419
q6	222	130	129	129
q7	2098	1679	1795	1679
q8	3218	3345	3338	3338
q9	8458	8475	8421	8421
q10	3449	3198	3172	3172
q11	612	500	498	498
q12	789	615	602	602
q13	11942	3050	3043	3043
q14	308	267	281	267
q15	539	487	479	479
q16	475	435	425	425
q17	1780	1468	1482	1468
q18	8010	7488	7498	7488
q19	1662	1591	1481	1481
q20	2043	1822	1821	1821
q21	5436	5211	5283	5211
q22	1109	1053	1025	1025
Total cold run time: 67098 ms
Total hot run time: 54763 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 186636 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 00852fc06bc133420ed3fdc1f5605b8d51e24516, data reload: false

query1	904	368	364	364
query2	6460	1973	1978	1973
query3	6648	212	212	212
query4	30901	23292	23058	23058
query5	4177	496	487	487
query6	251	169	177	169
query7	4593	283	280	280
query8	255	204	197	197
query9	8437	2530	2501	2501
query10	445	285	266	266
query11	17626	15034	15111	15034
query12	146	101	95	95
query13	1622	381	366	366
query14	9780	7180	6882	6882
query15	245	173	169	169
query16	7516	461	412	412
query17	1589	581	571	571
query18	1889	298	299	298
query19	223	146	151	146
query20	123	113	111	111
query21	217	105	100	100
query22	4536	4166	3985	3985
query23	34568	33570	34382	33570
query24	12187	2872	2819	2819
query25	533	376	392	376
query26	1018	153	155	153
query27	2695	280	274	274
query28	7174	2055	2051	2051
query29	645	417	396	396
query30	290	154	148	148
query31	980	762	770	762
query32	98	55	56	55
query33	769	278	279	278
query34	979	471	493	471
query35	864	741	748	741
query36	1092	935	951	935
query37	139	85	82	82
query38	4071	3938	3833	3833
query39	1433	1388	1382	1382
query40	283	118	116	116
query41	47	47	44	44
query42	123	103	99	99
query43	496	488	478	478
query44	1195	736	756	736
query45	196	165	169	165
query46	1116	741	745	741
query47	1851	1770	1806	1770
query48	366	289	300	289
query49	1141	437	451	437
query50	808	410	406	406
query51	7267	7016	6979	6979
query52	97	87	88	87
query53	250	183	182	182
query54	876	445	444	444
query55	78	79	77	77
query56	282	246	262	246
query57	1180	1062	1068	1062
query58	236	225	238	225
query59	2903	2848	2849	2848
query60	289	257	264	257
query61	105	98	99	98
query62	842	637	674	637
query63	213	187	183	183
query64	6331	2322	1755	1755
query65	3187	3127	3113	3113
query66	1220	335	339	335
query67	15824	15221	15257	15221
query68	3343	561	566	561
query69	392	280	282	280
query70	1090	1113	1136	1113
query71	342	278	286	278
query72	2600	2056	2041	2041
query73	691	318	314	314
query74	9422	8893	8834	8834
query75	3394	2607	2735	2607
query76	1843	1014	1010	1010
query77	475	390	311	311
query78	9652	9073	9022	9022
query79	1049	534	534	534
query80	708	492	507	492
query81	485	229	229	229
query82	290	145	136	136
query83	171	152	154	152
query84	230	78	71	71
query85	678	295	290	290
query86	316	297	297	297
query87	4421	4355	4205	4205
query88	2875	2292	2287	2287
query89	381	298	290	290
query90	1953	251	190	190
query91	122	98	99	98
query92	60	50	53	50
query93	1036	534	527	527
query94	796	301	308	301
query95	380	251	262	251
query96	589	272	267	267
query97	3212	3112	3104	3104
query98	215	206	201	201
query99	1551	1319	1261	1261
Total cold run time: 281682 ms
Total hot run time: 186636 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.49 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 00852fc06bc133420ed3fdc1f5605b8d51e24516, data reload: false

query1	0.05	0.05	0.04
query2	0.08	0.04	0.04
query3	0.22	0.05	0.06
query4	1.68	0.08	0.08
query5	0.50	0.50	0.49
query6	1.13	0.73	0.74
query7	0.02	0.01	0.02
query8	0.05	0.05	0.05
query9	0.54	0.49	0.48
query10	0.53	0.52	0.54
query11	0.16	0.12	0.12
query12	0.14	0.12	0.12
query13	0.60	0.58	0.57
query14	0.76	0.80	0.78
query15	0.88	0.82	0.82
query16	0.37	0.38	0.37
query17	1.04	1.07	1.06
query18	0.21	0.20	0.20
query19	1.88	1.84	1.75
query20	0.01	0.00	0.01
query21	15.40	0.65	0.65
query22	4.56	7.66	1.68
query23	18.28	1.35	1.31
query24	2.16	0.22	0.21
query25	0.15	0.08	0.09
query26	0.26	0.18	0.18
query27	0.08	0.07	0.08
query28	13.25	1.01	1.00
query29	12.66	3.28	3.30
query30	0.24	0.06	0.05
query31	2.86	0.40	0.40
query32	3.29	0.48	0.47
query33	2.97	3.01	2.98
query34	17.05	4.38	4.41
query35	4.43	4.40	4.42
query36	0.65	0.49	0.47
query37	0.19	0.16	0.16
query38	0.16	0.14	0.15
query39	0.05	0.04	0.04
query40	0.17	0.13	0.14
query41	0.10	0.05	0.06
query42	0.05	0.06	0.04
query43	0.05	0.04	0.05
Total cold run time: 109.91 s
Total hot run time: 30.49 s

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Aug 27, 2024
@github-actions GitHub Actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions GitHub Actions
Copy link
Contributor

PR approved by anyone and no changes requested.

mrhhsg
mrhhsg approved these changes Aug 27, 2024
View reviewed changes
Copy link
Member

@mrhhsg mrhhsg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@yiguolei yiguolei merged commit d11c6e5 into apache:master Aug 27, 2024
27 of 30 checks passed
yiguolei added a commit to yiguolei/incubator-doris that referenced this pull request Sep 9, 2024
@yiguolei @Doris-Extras
[enhancement](memory) add exception handle logic in memtable flush (a…
9c036a0 
…pache#39913)

## Proposed changes

Issue Number: close #xxx

<!--Describe your changes.-->

---------

Co-authored-by: yiguolei <yiguolei@gmail.com>
@yiguolei yiguolei mentioned this pull request Sep 9, 2024
Closed
@xinyiZzz xinyiZzz mentioned this pull request Sep 9, 2024
Closed
yiguolei added a commit to yiguolei/incubator-doris that referenced this pull request Sep 15, 2024
@yiguolei @Doris-Extras
[enhancement](memory) add exception handle logic in memtable flush (a…
fbd4411 
…pache#39913)

## Proposed changes

Issue Number: close #xxx

<!--Describe your changes.-->

---------

Co-authored-by: yiguolei <yiguolei@gmail.com>
@yiguolei yiguolei mentioned this pull request Sep 15, 2024
Merged
yiguolei added a commit that referenced this pull request Sep 16, 2024
@yiguolei @Doris-Extras
[cherrypick](branch21) pick some exception handleing pr in page io (#…
a6e2364 
…40883)

## Proposed changes

pick
#39536
#39913
#39964
#40020

---------

Co-authored-by: yiguolei <yiguolei@gmail.com>
dataroaring pushed a commit that referenced this pull request Oct 9, 2024
@yiguolei @Doris-Extras @dataroaring
[enhancement](memory) add exception handle logic in memtable flush (#…
1b7a5cc 
…39913)

## Proposed changes

Issue Number: close #xxx

<!--Describe your changes.-->

---------

Co-authored-by: yiguolei <yiguolei@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@mrhhsg mrhhsg mrhhsg approved these changes

@jacktengg jacktengg jacktengg approved these changes

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by one committer. dev/2.1.7-merged dev/3.0.3-merged reviewed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@yiguolei @doris-robot @mrhhsg @jacktengg @dataroaring @Doris-Extras