Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[enhancement](memory) return error if allocate memory failed during page builder allocate memory failed #39964

Merged
merged 2 commits into from
Aug 27, 2024
Merged

[enhancement](memory) return error if allocate memory failed during page builder allocate memory failed #39964

merged 2 commits into from
Aug 27, 2024

Conversation

yiguolei
Copy link
Contributor

Proposed changes

Issue Number: close #xxx

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@yiguolei
Copy link
Contributor Author

run buildall

@github-actions GitHub Actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TPC-H: Total hot run time: 38045 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit d06cdb6baed3257b32712fe033a026f50eee2ea4, data reload: false

------ Round 1 ----------------------------------
q1	17689	4613	4315	4315
q2	2027	186	179	179
q3	11843	978	1115	978
q4	10530	861	774	774
q5	7779	2910	2870	2870
q6	228	133	134	133
q7	977	625	619	619
q8	9330	2100	2127	2100
q9	7141	6572	6587	6572
q10	7017	2283	2256	2256
q11	450	253	243	243
q12	396	220	218	218
q13	18735	3063	3024	3024
q14	286	234	245	234
q15	531	487	489	487
q16	499	406	390	390
q17	984	657	704	657
q18	7437	6899	6942	6899
q19	1384	1123	1034	1034
q20	678	334	341	334
q21	3869	2910	2715	2715
q22	1137	1014	1061	1014
Total cold run time: 110947 ms
Total hot run time: 38045 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4952	4306	4256	4256
q2	376	285	278	278
q3	2901	2721	2693	2693
q4	1901	1667	1619	1619
q5	5743	5707	5761	5707
q6	225	131	129	129
q7	2218	1844	1816	1816
q8	3306	3453	3523	3453
q9	8857	8961	8883	8883
q10	3627	3459	3364	3364
q11	586	519	512	512
q12	826	666	655	655
q13	12902	3177	3246	3177
q14	330	290	296	290
q15	536	500	476	476
q16	484	441	452	441
q17	1849	1550	1551	1550
q18	8207	7828	7950	7828
q19	1754	1678	1553	1553
q20	2174	1951	1883	1883
q21	5739	5361	5596	5361
q22	1125	1054	1041	1041
Total cold run time: 70618 ms
Total hot run time: 56965 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 191337 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit d06cdb6baed3257b32712fe033a026f50eee2ea4, data reload: false

query1	1272	889	881	881
query2	6333	1859	1900	1859
query3	10765	4057	4138	4057
query4	60448	25970	23293	23293
query5	5330	486	503	486
query6	398	156	162	156
query7	5771	285	292	285
query8	288	203	217	203
query9	8895	2511	2502	2502
query10	491	274	270	270
query11	18007	15181	15339	15181
query12	158	103	102	102
query13	1539	392	387	387
query14	11161	6943	7182	6943
query15	233	166	168	166
query16	7443	461	480	461
query17	1125	558	559	558
query18	2055	304	303	303
query19	296	146	159	146
query20	120	114	110	110
query21	225	101	102	101
query22	4582	4464	4538	4464
query23	34335	33642	33344	33344
query24	5936	2814	2794	2794
query25	543	396	397	396
query26	688	160	156	156
query27	1787	280	274	274
query28	3792	2056	2047	2047
query29	694	422	418	418
query30	240	157	154	154
query31	931	809	740	740
query32	82	57	58	57
query33	453	293	297	293
query34	850	457	467	457
query35	874	741	750	741
query36	1078	929	922	922
query37	144	89	82	82
query38	4014	3882	3867	3867
query39	1430	1383	1394	1383
query40	200	117	120	117
query41	44	48	46	46
query42	120	99	98	98
query43	498	455	458	455
query44	1082	737	746	737
query45	199	181	167	167
query46	1099	754	756	754
query47	1873	1774	1814	1774
query48	364	302	306	302
query49	774	437	456	437
query50	826	409	431	409
query51	7262	7171	6932	6932
query52	98	90	90	90
query53	253	184	178	178
query54	569	456	451	451
query55	78	80	82	80
query56	291	269	274	269
query57	1204	1097	1084	1084
query58	220	236	277	236
query59	3020	2738	2914	2738
query60	307	282	273	273
query61	127	118	116	116
query62	753	668	656	656
query63	222	182	183	182
query64	3334	1847	1783	1783
query65	3270	3177	3168	3168
query66	656	340	345	340
query67	15602	15199	15279	15199
query68	4424	548	551	548
query69	403	290	282	282
query70	1188	1132	1047	1047
query71	355	271	285	271
query72	6619	2311	1903	1903
query73	792	319	319	319
query74	9282	8880	8685	8685
query75	3352	2666	2706	2666
query76	1618	966	949	949
query77	544	309	314	309
query78	9723	9110	9054	9054
query79	1584	535	526	526
query80	1020	516	506	506
query81	565	227	226	226
query82	441	134	134	134
query83	175	150	149	149
query84	255	71	86	71
query85	925	284	289	284
query86	414	307	285	285
query87	4389	4254	4253	4253
query88	3369	2280	2290	2280
query89	388	282	282	282
query90	1799	191	189	189
query91	120	98	100	98
query92	61	53	50	50
query93	2358	527	530	527
query94	779	303	296	296
query95	355	260	260	260
query96	601	264	264	264
query97	3206	3117	3066	3066
query98	227	239	196	196
query99	1825	1343	1252	1252
Total cold run time: 312307 ms
Total hot run time: 191337 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.8 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit d06cdb6baed3257b32712fe033a026f50eee2ea4, data reload: false

query1	0.04	0.04	0.04
query2	0.09	0.05	0.05
query3	0.22	0.05	0.05
query4	1.67	0.08	0.07
query5	0.52	0.50	0.51
query6	1.13	0.73	0.72
query7	0.02	0.01	0.01
query8	0.06	0.04	0.05
query9	0.54	0.48	0.49
query10	0.55	0.54	0.55
query11	0.15	0.12	0.11
query12	0.15	0.12	0.12
query13	0.62	0.58	0.58
query14	0.76	0.79	0.80
query15	0.84	0.81	0.80
query16	0.37	0.37	0.37
query17	0.98	1.01	1.05
query18	0.21	0.22	0.22
query19	1.97	1.82	1.80
query20	0.01	0.01	0.01
query21	15.40	0.67	0.67
query22	4.48	7.12	1.81
query23	18.26	1.41	1.31
query24	2.11	0.23	0.22
query25	0.15	0.08	0.08
query26	0.28	0.18	0.17
query27	0.07	0.08	0.07
query28	13.29	1.02	0.99
query29	12.60	3.42	3.37
query30	0.24	0.06	0.05
query31	2.89	0.40	0.39
query32	3.27	0.47	0.46
query33	2.98	2.97	2.99
query34	17.07	4.38	4.41
query35	4.48	4.52	4.47
query36	0.67	0.48	0.48
query37	0.18	0.16	0.16
query38	0.17	0.15	0.16
query39	0.05	0.04	0.04
query40	0.15	0.13	0.12
query41	0.09	0.04	0.04
query42	0.06	0.05	0.05
query43	0.05	0.04	0.04
Total cold run time: 109.89 s
Total hot run time: 30.8 s

zhannngchen
zhannngchen approved these changes Aug 27, 2024
View reviewed changes
Copy link
Contributor

@zhannngchen zhannngchen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Aug 27, 2024
@github-actions GitHub Actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions GitHub Actions
Copy link
Contributor

PR approved by anyone and no changes requested.

liaoxin01
liaoxin01 approved these changes Aug 27, 2024
View reviewed changes
Copy link
Contributor

@liaoxin01 liaoxin01 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@yiguolei yiguolei merged commit dc287d5 into apache:master Aug 27, 2024
28 of 31 checks passed
yiguolei added a commit to yiguolei/incubator-doris that referenced this pull request Sep 9, 2024
@yiguolei @Doris-Extras
[enhancement](memory) return error if allocate memory failed during p…
18dffc1 
…age builder allocate memory failed (apache#39964)

## Proposed changes

Issue Number: close #xxx

<!--Describe your changes.-->

---------

Co-authored-by: yiguolei <yiguolei@gmail.com>
@yiguolei yiguolei mentioned this pull request Sep 9, 2024
Closed
@xinyiZzz xinyiZzz mentioned this pull request Sep 9, 2024
Closed
yiguolei added a commit to yiguolei/incubator-doris that referenced this pull request Sep 15, 2024
@yiguolei @Doris-Extras
[enhancement](memory) return error if allocate memory failed during p…
c9e1a3d 
…age builder allocate memory failed (apache#39964)

## Proposed changes

Issue Number: close #xxx

<!--Describe your changes.-->

---------

Co-authored-by: yiguolei <yiguolei@gmail.com>
@yiguolei yiguolei mentioned this pull request Sep 15, 2024
Merged
yiguolei added a commit that referenced this pull request Sep 16, 2024
@yiguolei @Doris-Extras
[cherrypick](branch21) pick some exception handleing pr in page io (#…
a6e2364 
…40883)

## Proposed changes

pick
#39536
#39913
#39964
#40020

---------

Co-authored-by: yiguolei <yiguolei@gmail.com>
dataroaring pushed a commit that referenced this pull request Oct 9, 2024
@yiguolei @Doris-Extras @dataroaring
[enhancement](memory) return error if allocate memory failed during p…
777bd3c 
…age builder allocate memory failed (#39964)

## Proposed changes

Issue Number: close #xxx

<!--Describe your changes.-->

---------

Co-authored-by: yiguolei <yiguolei@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@liaoxin01 liaoxin01 liaoxin01 approved these changes

@zhannngchen zhannngchen zhannngchen approved these changes

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by one committer. dev/2.1.7-merged dev/3.0.3-merged reviewed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@yiguolei @doris-robot @liaoxin01 @zhannngchen @dataroaring @Doris-Extras