Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[docs](partition) Improve auto partitions document #27662

Merged
merged 2 commits into from
Nov 29, 2023

Conversation

zclllyybb
Copy link
Contributor

Proposed changes

Issue Number: close #xxx

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@zclllyybb
Copy link
Contributor Author

run buildall

@zclllyybb
Copy link
Contributor Author

run buildall

@zclllyybb
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
Tpch sf100 test result on commit 1fe56b4bfef4a7b236e530063b74ba3a106df2b9, data reload: false

run tpch-sf100 query with default conf and session variables
q1	4921	4674	4645	4645
q2	357	160	155	155
q3	2043	1955	1970	1955
q4	1397	1306	1296	1296
q5	3956	3916	4014	3916
q6	264	130	124	124
q7	1421	895	879	879
q8	2809	2826	2782	2782
q9	9852	9709	9632	9632
q10	3469	3611	3562	3562
q11	372	249	244	244
q12	436	284	304	284
q13	4583	3778	3764	3764
q14	320	287	295	287
q15	586	541	528	528
q16	658	591	581	581
q17	1140	993	912	912
q18	7977	7508	7544	7508
q19	1685	1668	1677	1668
q20	533	296	320	296
q21	4458	4026	4033	4026
q22	467	385	387	385
Total cold run time: 53704 ms
Total hot run time: 49429 ms

run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1	4610	4563	4567	4563
q2	348	231	245	231
q3	4062	4035	4031	4031
q4	2725	2713	2716	2713
q5	9679	9698	9610	9610
q6	245	120	121	120
q7	3042	2488	2495	2488
q8	4465	4459	4490	4459
q9	13298	13148	13145	13145
q10	4090	4188	4172	4172
q11	784	651	646	646
q12	976	806	827	806
q13	4291	3525	3636	3525
q14	377	342	343	342
q15	569	517	511	511
q16	744	672	687	672
q17	3873	3890	3904	3890
q18	9660	9191	9197	9191
q19	1825	1801	1780	1780
q20	2411	2068	2056	2056
q21	8817	8677	8675	8675
q22	875	825	829	825
Total cold run time: 81766 ms
Total hot run time: 78451 ms

@doris-robot
Copy link

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.97 seconds
stream load tsv: 576 seconds loaded 74807831229 Bytes, about 123 MB/s
stream load json: 18 seconds loaded 2358488459 Bytes, about 124 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 33 seconds loaded 861443392 Bytes, about 24 MB/s
insert into select: 28.7 seconds inserted 10000000 Rows, about 348K ops/s
storage size: 17101515551 Bytes

@doris-robot
Copy link

TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
Tpch sf100 test result on commit 9b128de730d730006e75c52a2cacae80b21d2ff2, data reload: true

run tpch-sf100 query with default conf and session variables
q1	4814	4594	4581	4581
q2	381	130	119	119
q3	2062	1935	1865	1865
q4	1390	1267	1260	1260
q5	3948	3943	3948	3943
q6	247	126	125	125
q7	1349	872	853	853
q8	2793	2784	2768	2768
q9	9962	10371	10294	10294
q10	9936	3528	3512	3512
q11	381	236	234	234
q12	438	288	300	288
q13	16022	3801	3776	3776
q14	319	276	299	276
q15	577	538	512	512
q16	496	453	458	453
q17	1165	954	923	923
q18	8247	7468	7518	7468
q19	1686	1687	1686	1686
q20	569	295	293	293
q21	4373	3996	4052	3996
q22	461	361	367	361
Total cold run time: 71616 ms
Total hot run time: 49586 ms

run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1	4568	4572	4560	4560
q2	335	216	254	216
q3	4036	4033	3985	3985
q4	2713	2699	2702	2699
q5	9678	9703	9687	9687
q6	246	122	125	122
q7	2967	2517	2490	2490
q8	4465	4470	4449	4449
q9	12997	12935	12862	12862
q10	4056	4126	4155	4126
q11	790	654	634	634
q12	965	794	810	794
q13	4375	3534	3582	3534
q14	374	339	362	339
q15	580	504	517	504
q16	600	538	579	538
q17	3932	3916	3895	3895
q18	9781	9229	9084	9084
q19	1840	1779	1775	1775
q20	2399	2066	2069	2066
q21	8805	8526	8598	8526
q22	921	781	813	781
Total cold run time: 81423 ms
Total hot run time: 77666 ms

Copy link
Contributor

@HappenLee HappenLee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Nov 29, 2023
Copy link
Contributor

PR approved by at least one committer and no changes requested.

Copy link
Contributor

PR approved by anyone and no changes requested.

Copy link
Contributor

@LemonLiTree LemonLiTree left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@HappenLee HappenLee merged commit c30299c into apache:master Nov 29, 2023
17 of 18 checks passed
keanji-x pushed a commit to keanji-x/doris that referenced this pull request Nov 29, 2023
keanji-x pushed a commit to keanji-x/doris that referenced this pull request Nov 29, 2023
XuJianxu pushed a commit to XuJianxu/doris that referenced this pull request Dec 14, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by one committer. reviewed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants