Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[chore](backup) Fix the db name of the restored view #38075

Merged
merged 1 commit into from
Jul 18, 2024
Merged

[chore](backup) Fix the db name of the restored view #38075

merged 1 commit into from
Jul 18, 2024

Conversation

w41ter
Copy link
Contributor

@w41ter w41ter commented Jul 18, 2024

Cherry-pick #37412

Previously, during restore, the database name in the CREATE VIEW statement was not modified, causing the restored view to be unviewable with the SHOW VIEW command. This PR retains the original cluster's database name in the BackupMeta and manually replaces it with the new cluster's database name in the CREATE VIEW statement during restore.

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@w41ter
Copy link
Contributor Author

w41ter commented Jul 18, 2024

run buildall

@w41ter
[chore](backup) Fix the db name of the restored view
d6828a8 
Cherry-pick apache#37412

Previously, during restore, the database name in the CREATE VIEW
statement was not modified, causing the restored view to be unviewable
with the SHOW VIEW command. This PR retains the original cluster's
database name in the BackupMeta and manually replaces it with the new
cluster's database name in the CREATE VIEW statement during restore.
@w41ter w41ter force-pushed the pick/37412_to_2.0 branch from 2344082 to d6828a8 Compare July 18, 2024 08:37
@w41ter
Copy link
Contributor Author

w41ter commented Jul 18, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 49699 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit d6828a83a983938fa45c5120a0fe1711629fc58f, data reload: false

------ Round 1 ----------------------------------
q1	17612	4432	4361	4361
q2	2064	162	147	147
q3	10265	1920	1909	1909
q4	10389	1243	1315	1243
q5	8447	3856	3924	3856
q6	266	123	126	123
q7	2082	1624	1621	1621
q8	9296	2729	2697	2697
q9	10767	10277	10161	10161
q10	8653	3518	3503	3503
q11	408	240	255	240
q12	468	307	302	302
q13	18366	3968	4038	3968
q14	351	328	338	328
q15	495	464	462	462
q16	674	569	569	569
q17	1121	970	966	966
q18	7267	6833	6857	6833
q19	1837	1638	1606	1606
q20	536	308	299	299
q21	4425	4147	4069	4069
q22	549	445	436	436
Total cold run time: 116338 ms
Total hot run time: 49699 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4287	4309	4318	4309
q2	319	228	225	225
q3	4147	4175	4132	4132
q4	2746	2754	2763	2754
q5	7081	7087	7074	7074
q6	237	123	119	119
q7	3192	2876	2890	2876
q8	4356	4472	4476	4472
q9	16940	16804	16758	16758
q10	4207	4259	4256	4256
q11	751	682	697	682
q12	1044	858	878	858
q13	7224	3720	3772	3720
q14	450	437	439	437
q15	499	473	463	463
q16	738	688	688	688
q17	3847	3881	3809	3809
q18	8889	8670	8687	8670
q19	1705	1717	1627	1627
q20	2424	2158	2136	2136
q21	8586	8560	8496	8496
q22	1043	964	938	938
Total cold run time: 84712 ms
Total hot run time: 79499 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 204652 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit d6828a83a983938fa45c5120a0fe1711629fc58f, data reload: false

query1	919	411	382	382
query2	6550	2864	2656	2656
query3	6922	217	206	206
query4	20881	17941	17926	17926
query5	19753	6508	6498	6498
query6	291	222	229	222
query7	4167	316	325	316
query8	442	415	393	393
query9	3115	2726	2716	2716
query10	416	305	309	305
query11	11879	10713	10852	10713
query12	129	78	81	78
query13	5610	692	700	692
query14	17891	13829	13802	13802
query15	363	240	263	240
query16	6472	282	271	271
query17	1716	1450	886	886
query18	2310	418	408	408
query19	211	159	151	151
query20	77	78	80	78
query21	195	101	96	96
query22	5256	5056	5044	5044
query23	32568	32147	31950	31950
query24	7038	6559	6564	6559
query25	539	432	431	431
query26	532	168	163	163
query27	1870	299	306	299
query28	6215	2388	2328	2328
query29	3005	2800	3056	2800
query30	249	170	168	168
query31	884	741	742	741
query32	72	65	60	60
query33	413	257	272	257
query34	849	484	473	473
query35	1111	917	921	917
query36	1526	1211	1269	1211
query37	93	63	63	63
query38	3078	2882	2985	2882
query39	1381	1340	1325	1325
query40	214	92	97	92
query41	50	47	46	46
query42	85	91	85	85
query43	682	786	686	686
query44	1119	729	731	729
query45	245	238	239	238
query46	1250	986	973	973
query47	2017	1649	1785	1649
query48	1015	757	712	712
query49	617	381	374	374
query50	879	635	604	604
query51	4845	4703	4716	4703
query52	101	95	83	83
query53	462	338	331	331
query54	2701	2472	2505	2472
query55	94	78	94	78
query56	244	231	207	207
query57	1170	1083	1155	1083
query58	223	200	204	200
query59	4228	4106	4235	4106
query60	212	227	220	220
query61	98	95	98	95
query62	869	505	565	505
query63	482	352	348	348
query64	2520	1549	1500	1500
query65	3639	3562	3548	3548
query66	755	385	384	384
query67	16195	15934	14966	14966
query68	10815	662	662	662
query69	590	364	356	356
query70	1993	1508	1384	1384
query71	414	306	311	306
query72	6546	3499	3481	3481
query73	737	325	316	316
query74	6353	5830	5916	5830
query75	5538	3778	3736	3736
query76	6971	1158	1209	1158
query77	1193	253	268	253
query78	12491	11571	12396	11571
query79	8072	635	634	634
query80	1059	398	410	398
query81	459	246	233	233
query82	695	98	102	98
query83	167	133	134	133
query84	255	73	69	69
query85	877	332	334	332
query86	330	286	309	286
query87	3239	3068	3085	3068
query88	4394	2331	2315	2315
query89	444	305	285	285
query90	1980	202	209	202
query91	178	151	145	145
query92	58	53	55	53
query93	6125	627	591	591
query94	701	204	213	204
query95	1113	1068	1061	1061
query96	647	323	323	323
query97	6544	6400	6360	6360
query98	196	176	185	176
query99	3056	910	910	910
Total cold run time: 318229 ms
Total hot run time: 204652 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.42 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit d6828a83a983938fa45c5120a0fe1711629fc58f, data reload: false

query1	0.02	0.02	0.03
query2	0.07	0.02	0.02
query3	0.24	0.05	0.05
query4	1.80	0.07	0.06
query5	0.54	0.53	0.53
query6	1.30	0.62	0.61
query7	0.02	0.01	0.01
query8	0.03	0.02	0.02
query9	0.52	0.48	0.48
query10	0.54	0.53	0.53
query11	0.12	0.08	0.09
query12	0.12	0.09	0.09
query13	0.62	0.62	0.61
query14	0.79	0.80	0.79
query15	0.79	0.76	0.76
query16	0.36	0.38	0.35
query17	1.02	0.97	1.02
query18	0.21	0.28	0.24
query19	1.92	1.85	1.86
query20	0.02	0.01	0.01
query21	15.46	0.56	0.54
query22	2.13	2.48	1.46
query23	17.23	0.95	1.13
query24	4.65	0.86	1.38
query25	0.40	0.09	0.06
query26	0.52	0.15	0.16
query27	0.05	0.04	0.03
query28	8.23	0.78	0.75
query29	12.60	2.28	2.12
query30	0.54	0.60	0.54
query31	2.82	0.39	0.38
query32	3.37	0.51	0.50
query33	3.07	3.10	3.11
query34	15.27	4.81	4.81
query35	4.86	4.83	4.85
query36	1.07	1.02	1.03
query37	0.06	0.05	0.04
query38	0.04	0.02	0.02
query39	0.02	0.01	0.02
query40	0.16	0.14	0.14
query41	0.06	0.01	0.02
query42	0.02	0.01	0.02
query43	0.02	0.01	0.01
Total cold run time: 103.7 s
Total hot run time: 30.42 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit d6828a83a983938fa45c5120a0fe1711629fc58f with default session variables
Stream load json:         20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       21.4 seconds inserted 10000000 Rows, about 467K ops/s

@dataroaring dataroaring merged commit 2b8c6a0 into apache:branch-2.0 Jul 18, 2024
23 of 24 checks passed
@w41ter w41ter deleted the pick/37412_to_2.0 branch July 18, 2024 12:58
mongo360 pushed a commit to mongo360/doris that referenced this pull request Aug 16, 2024
@w41ter
[chore](backup) Fix the db name of the restored view (apache#38075)
a521e26 
Cherry-pick apache#37412

Previously, during restore, the database name in the CREATE VIEW
statement was not modified, causing the restored view to be unviewable
with the SHOW VIEW command. This PR retains the original cluster's
database name in the BackupMeta and manually replaces it with the new
cluster's database name in the CREATE VIEW statement during restore.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@w41ter @doris-robot @dataroaring