Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[opt](cpu-profile) enable cpu profile in BE webui #40330

Merged
merged 3 commits into from
Sep 5, 2024
Merged

[opt](cpu-profile) enable cpu profile in BE webui #40330

merged 3 commits into from
Sep 5, 2024

Conversation

morningman
Copy link
Contributor

Visit be_ip:webserver_port/cpu
You can get cpu profile like this:
image

or

img_v3_02ec_e8d45934-ddaf-45de-818e-a7e4899ec7cg

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Sep 3, 2024

clang-tidy review says "All clean, LGTM! 👍"

xinyiZzz
xinyiZzz previously approved these changes Sep 3, 2024
View reviewed changes
Copy link
Contributor

@xinyiZzz xinyiZzz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Sep 3, 2024

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added approved Indicates a PR has been approved by one committer. reviewed labels Sep 3, 2024
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Sep 3, 2024

PR approved by anyone and no changes requested.

@github-actions github-actions bot removed the approved Indicates a PR has been approved by one committer. label Sep 3, 2024
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Sep 3, 2024

clang-tidy review says "All clean, LGTM! 👍"

@morningman
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 36.83% (9385/25485)
Line Coverage: 28.27% (77436/273884)
Region Coverage: 27.67% (39962/144412)
Branch Coverage: 24.31% (20331/83624)
Coverage Report: http://coverage.selectdb-in.cc/coverage/371dfd930903c1ab9335814231e7a3607f742c5b_371dfd930903c1ab9335814231e7a3607f742c5b/report/index.html

@doris-robot
Copy link

TPC-H: Total hot run time: 38274 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 371dfd930903c1ab9335814231e7a3607f742c5b, data reload: false

------ Round 1 ----------------------------------
q1	17804	4419	4341	4341
q2	2024	186	177	177
q3	11886	931	1120	931
q4	10508	710	734	710
q5	7770	2897	2820	2820
q6	232	142	144	142
q7	974	637	604	604
q8	9571	2138	2132	2132
q9	7234	6641	6747	6641
q10	7524	2406	2315	2315
q11	458	258	246	246
q12	397	231	230	230
q13	19042	3107	3127	3107
q14	290	257	241	241
q15	529	488	503	488
q16	641	522	517	517
q17	981	633	702	633
q18	8100	6865	6977	6865
q19	1383	1071	1066	1066
q20	697	335	332	332
q21	3963	3102	2708	2708
q22	1120	1032	1028	1028
Total cold run time: 113128 ms
Total hot run time: 38274 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4364	4296	4258	4258
q2	387	280	271	271
q3	2877	2668	2692	2668
q4	1945	1657	1705	1657
q5	5712	5654	5723	5654
q6	229	134	134	134
q7	2224	1858	1806	1806
q8	3285	3502	3492	3492
q9	8904	8797	8746	8746
q10	3689	3379	3277	3277
q11	597	517	528	517
q12	824	683	656	656
q13	15914	3217	3354	3217
q14	328	300	295	295
q15	552	485	501	485
q16	630	584	594	584
q17	1835	1586	1567	1567
q18	8140	8001	7706	7706
q19	1728	1665	1494	1494
q20	2173	1912	1990	1912
q21	5586	5502	5515	5502
q22	1138	1022	1060	1022
Total cold run time: 73061 ms
Total hot run time: 56920 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 192872 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 371dfd930903c1ab9335814231e7a3607f742c5b, data reload: false

query1	1250	886	867	867
query2	6444	1972	1889	1889
query3	10640	3894	4009	3894
query4	59202	23875	23407	23407
query5	5315	518	500	500
query6	426	164	157	157
query7	5783	289	302	289
query8	291	214	214	214
query9	8891	2475	2460	2460
query10	489	280	279	279
query11	18073	15316	15301	15301
query12	179	100	101	100
query13	1567	393	371	371
query14	11040	7882	7248	7248
query15	237	170	167	167
query16	7193	470	465	465
query17	1101	570	583	570
query18	1443	291	293	291
query19	295	144	142	142
query20	118	113	116	113
query21	207	105	104	104
query22	4487	4454	4660	4454
query23	34441	33695	33641	33641
query24	5963	2909	2830	2830
query25	559	403	412	403
query26	689	158	157	157
query27	1806	278	274	274
query28	3764	2036	2025	2025
query29	668	429	425	425
query30	229	157	158	157
query31	955	750	780	750
query32	78	52	57	52
query33	449	295	294	294
query34	862	462	477	462
query35	856	720	761	720
query36	1057	931	928	928
query37	154	97	88	88
query38	3926	3872	3943	3872
query39	1490	1422	1424	1422
query40	197	116	114	114
query41	48	48	45	45
query42	118	95	95	95
query43	516	470	457	457
query44	1109	727	731	727
query45	193	171	167	167
query46	1088	735	748	735
query47	1902	1779	1793	1779
query48	364	308	294	294
query49	757	435	485	435
query50	815	407	414	407
query51	7069	7004	6786	6786
query52	99	84	87	84
query53	249	180	180	180
query54	566	467	448	448
query55	76	72	71	71
query56	273	260	253	253
query57	1180	1072	1056	1056
query58	226	240	218	218
query59	2911	2736	2777	2736
query60	288	264	273	264
query61	103	99	128	99
query62	747	656	642	642
query63	218	184	185	184
query64	2863	670	673	670
query65	3223	3122	3174	3122
query66	674	360	344	344
query67	15485	15258	15139	15139
query68	3031	552	614	552
query69	409	285	278	278
query70	1144	1069	1103	1069
query71	341	268	277	268
query72	6176	4066	4077	4066
query73	742	326	325	325
query74	9100	8856	8942	8856
query75	3392	2670	2719	2670
query76	1396	987	925	925
query77	552	323	311	311
query78	9781	9083	9042	9042
query79	1041	541	523	523
query80	690	539	504	504
query81	470	234	231	231
query82	246	145	145	145
query83	172	148	150	148
query84	256	82	74	74
query85	689	289	339	289
query86	300	294	301	294
query87	4522	4210	4300	4210
query88	3396	2317	2331	2317
query89	386	280	285	280
query90	2020	198	188	188
query91	137	100	98	98
query92	61	49	53	49
query93	1061	518	521	518
query94	724	298	288	288
query95	338	251	251	251
query96	589	261	271	261
query97	3236	3173	3069	3069
query98	225	208	206	206
query99	1571	1234	1273	1234
Total cold run time: 304168 ms
Total hot run time: 192872 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.15 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 371dfd930903c1ab9335814231e7a3607f742c5b, data reload: false

query1	0.04	0.04	0.04
query2	0.07	0.05	0.04
query3	0.23	0.06	0.06
query4	1.66	0.09	0.08
query5	0.51	0.50	0.49
query6	1.12	0.73	0.73
query7	0.02	0.02	0.02
query8	0.06	0.05	0.05
query9	0.54	0.51	0.49
query10	0.53	0.54	0.55
query11	0.16	0.12	0.12
query12	0.15	0.12	0.12
query13	0.61	0.60	0.58
query14	2.04	2.05	2.12
query15	0.89	0.83	0.82
query16	0.38	0.37	0.38
query17	1.05	1.00	1.07
query18	0.21	0.20	0.22
query19	1.93	1.88	1.84
query20	0.01	0.02	0.01
query21	15.43	0.66	0.66
query22	4.19	7.26	1.93
query23	18.29	1.44	1.31
query24	2.17	0.21	0.22
query25	0.15	0.09	0.08
query26	0.27	0.19	0.17
query27	0.07	0.08	0.07
query28	13.20	1.02	1.02
query29	12.60	3.31	3.27
query30	0.24	0.06	0.05
query31	2.90	0.40	0.39
query32	3.23	0.48	0.48
query33	3.05	2.96	3.05
query34	17.01	4.41	4.42
query35	4.43	4.46	4.44
query36	0.65	0.48	0.47
query37	0.18	0.15	0.16
query38	0.15	0.15	0.15
query39	0.04	0.03	0.04
query40	0.15	0.13	0.12
query41	0.09	0.05	0.05
query42	0.06	0.04	0.05
query43	0.04	0.04	0.04
Total cold run time: 110.8 s
Total hot run time: 32.15 s

xinyiZzz
xinyiZzz approved these changes Sep 5, 2024
View reviewed changes
Copy link
Contributor

@xinyiZzz xinyiZzz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Sep 5, 2024
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Sep 5, 2024

PR approved by at least one committer and no changes requested.

@morningman morningman merged commit 16e3c78 into apache:master Sep 5, 2024
26 of 29 checks passed
morningman added a commit to morningman/doris that referenced this pull request Sep 20, 2024
@morningman
[opt](cpu-profile) enable cpu profile in BE webui (apache#40330)
48f9951 
Visit `be_ip:webserver_port/cpu`
You can get cpu profile like this:
<img width="736" alt="image"
src="https://github.com/user-attachments/assets/05da73ca-c02a-4ec3-a2fb-83ef80ff9553">

or

![img_v3_02ec_e8d45934-ddaf-45de-818e-a7e4899ec7cg](https://github.com/user-attachments/assets/6b1a4e15-a219-456b-9d1f-e3be88177db5)
@morningman morningman mentioned this pull request Sep 20, 2024
Merged
morningman added a commit that referenced this pull request Sep 20, 2024
@morningman
[opt](cpu-profile) enable cpu profile in BE webui (#40330) (#41044)
596cfc9 
bp #40330
morningman added a commit to morningman/doris that referenced this pull request Oct 15, 2024
@morningman
[opt](cpu-profile) enable cpu profile in BE webui (apache#40330)
a880175 
Visit `be_ip:webserver_port/cpu`
You can get cpu profile like this:
<img width="736" alt="image"
src="https://github.com/user-attachments/assets/05da73ca-c02a-4ec3-a2fb-83ef80ff9553">

or

![img_v3_02ec_e8d45934-ddaf-45de-818e-a7e4899ec7cg](https://github.com/user-attachments/assets/6b1a4e15-a219-456b-9d1f-e3be88177db5)
@morningman morningman mentioned this pull request Oct 15, 2024
Merged
morningman added a commit to morningman/doris that referenced this pull request Oct 15, 2024
@morningman
[opt](cpu-profile) enable cpu profile in BE webui (apache#40330)
23001c7 
Visit `be_ip:webserver_port/cpu`
You can get cpu profile like this:
<img width="736" alt="image"
src="https://github.com/user-attachments/assets/05da73ca-c02a-4ec3-a2fb-83ef80ff9553">

or

![img_v3_02ec_e8d45934-ddaf-45de-818e-a7e4899ec7cg](https://github.com/user-attachments/assets/6b1a4e15-a219-456b-9d1f-e3be88177db5)
morningman added a commit to morningman/doris that referenced this pull request Oct 15, 2024
@morningman
[opt](cpu-profile) enable cpu profile in BE webui (apache#40330)
9ab3048 
Visit `be_ip:webserver_port/cpu`
You can get cpu profile like this:
<img width="736" alt="image"
src="https://github.com/user-attachments/assets/05da73ca-c02a-4ec3-a2fb-83ef80ff9553">

or

![img_v3_02ec_e8d45934-ddaf-45de-818e-a7e4899ec7cg](https://github.com/user-attachments/assets/6b1a4e15-a219-456b-9d1f-e3be88177db5)
morningman added a commit that referenced this pull request Oct 15, 2024
@morningman
[opt](cpu-profile) enable cpu profile in BE webui (#40330) (#41849)
b2d245c 
bp #40330
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@xinyiZzz xinyiZzz xinyiZzz approved these changes

@wuwenchi wuwenchi wuwenchi approved these changes

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by one committer. dev/2.1.7-merged dev/3.0.3-merged reviewed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@morningman @doris-robot @xinyiZzz @wuwenchi @dataroaring