Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[fix](cloud version) fix version of information_schema table backward when query in follower node #43451

Merged
merged 1 commit into from
Nov 8, 2024
Merged

[fix](cloud version) fix version of information_schema table backward when query in follower node #43451

merged 1 commit into from
Nov 8, 2024

Conversation

sollhui
Copy link
Contributor

@sollhui sollhui commented Nov 7, 2024

Problem Summary:

For the follower node in cloud mode, when querying the information_schema table, the version needs to be updated.
Otherwise, the version will always be the old value unless there is a query for the table in the follower node.

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No colde files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.
  • Release note

    None

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.
  • Release note

    None

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@sollhui sollhui changed the title fix version of information_schema table backward when query in follower node [fix](cloud) fix version of information_schema table backward when query in follower node Nov 7, 2024
@sollhui
Copy link
Contributor Author

sollhui commented Nov 7, 2024

run buildall

@liaoxin01 liaoxin01 requested a review from gavinchou November 7, 2024 12:41
@doris-robot
Copy link

TPC-H: Total hot run time: 41062 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit b8a79d705e6c385876503a5b4f96f9c30600b495, data reload: false

------ Round 1 ----------------------------------
q1	17566	7351	7259	7259
q2	2069	176	171	171
q3	10711	1068	1162	1068
q4	10559	849	961	849
q5	7748	3042	3067	3042
q6	233	143	145	143
q7	995	605	600	600
q8	9383	1951	2060	1951
q9	6522	6428	6456	6428
q10	7107	2435	2437	2435
q11	473	256	260	256
q12	406	219	215	215
q13	17787	3014	2975	2975
q14	236	214	219	214
q15	594	517	511	511
q16	648	579	607	579
q17	959	548	515	515
q18	7232	6729	6795	6729
q19	1328	939	1024	939
q20	456	180	178	178
q21	4025	2999	3004	2999
q22	1108	1007	1006	1006
Total cold run time: 108145 ms
Total hot run time: 41062 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7292	7227	7217	7217
q2	337	245	246	245
q3	2914	2998	2930	2930
q4	2053	1853	1838	1838
q5	5718	5747	5796	5747
q6	229	142	146	142
q7	2227	1858	1830	1830
q8	3385	3622	3467	3467
q9	8924	8956	8916	8916
q10	3593	3565	3594	3565
q11	602	513	511	511
q12	835	620	612	612
q13	10024	3220	3274	3220
q14	297	273	285	273
q15	610	553	549	549
q16	712	639	652	639
q17	1861	1616	1602	1602
q18	8273	7819	7701	7701
q19	1699	1541	1486	1486
q20	2103	1852	1874	1852
q21	5688	5473	5481	5473
q22	1143	1067	1034	1034
Total cold run time: 70519 ms
Total hot run time: 60849 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 194389 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit b8a79d705e6c385876503a5b4f96f9c30600b495, data reload: false

query1	2263	2110	2075	2075
query2	6243	2058	2029	2029
query3	15091	9091	217	217
query4	33106	23721	23597	23597
query5	3467	425	428	425
query6	265	178	176	176
query7	3976	289	291	289
query8	278	233	229	229
query9	9349	2727	2718	2718
query10	466	253	245	245
query11	17939	15421	15215	15215
query12	154	102	98	98
query13	1583	411	428	411
query14	9445	6646	6677	6646
query15	227	181	175	175
query16	7887	490	497	490
query17	1609	583	623	583
query18	2338	625	621	621
query19	240	184	187	184
query20	129	118	129	118
query21	208	107	104	104
query22	4978	4343	4523	4343
query23	34919	34126	34135	34126
query24	11913	3280	3307	3280
query25	644	410	403	403
query26	1223	186	187	186
query27	2511	279	288	279
query28	7500	2448	2443	2443
query29	876	424	424	424
query30	409	312	309	309
query31	1045	791	789	789
query32	92	54	57	54
query33	781	273	284	273
query34	947	511	512	511
query35	886	759	719	719
query36	1106	966	953	953
query37	147	78	80	78
query38	4404	4292	4285	4285
query39	1508	1422	1433	1422
query40	211	99	98	98
query41	48	46	49	46
query42	105	95	101	95
query43	524	497	478	478
query44	1293	808	803	803
query45	185	166	169	166
query46	1145	693	698	693
query47	1969	1860	1886	1860
query48	403	330	317	317
query49	986	399	400	399
query50	804	391	394	391
query51	7331	7140	7187	7140
query52	100	87	86	86
query53	258	181	182	181
query54	1023	418	396	396
query55	76	72	74	72
query56	252	236	237	236
query57	1299	1158	1147	1147
query58	222	204	197	197
query59	3152	3111	2965	2965
query60	278	242	251	242
query61	111	114	117	114
query62	886	676	678	676
query63	218	186	183	183
query64	3973	649	625	625
query65	3245	3219	3190	3190
query66	817	298	292	292
query67	15930	15888	15666	15666
query68	4468	592	582	582
query69	424	248	250	248
query70	1155	1133	1121	1121
query71	326	254	255	254
query72	6290	3988	4034	3988
query73	754	363	356	356
query74	10120	9002	9185	9002
query75	3464	2695	2698	2695
query76	2753	1031	1054	1031
query77	412	272	277	272
query78	10491	9442	9406	9406
query79	1914	598	588	588
query80	1267	428	418	418
query81	568	238	241	238
query82	977	112	112	112
query83	250	164	153	153
query84	242	74	73	73
query85	1291	318	292	292
query86	396	300	306	300
query87	4896	4699	4625	4625
query88	3652	2202	2182	2182
query89	440	287	290	287
query90	1961	188	182	182
query91	145	113	104	104
query92	57	50	50	50
query93	2393	530	539	530
query94	876	286	299	286
query95	344	247	241	241
query96	621	286	293	286
query97	2908	2724	2715	2715
query98	209	202	189	189
query99	1718	1337	1307	1307
Total cold run time: 306778 ms
Total hot run time: 194389 ms

backward when query in follower node
@sollhui
Copy link
Contributor Author

sollhui commented Nov 7, 2024

run buildall

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Nov 7, 2024
Copy link
Contributor

github-actions bot commented Nov 7, 2024

PR approved by at least one committer and no changes requested.

Copy link
Contributor

github-actions bot commented Nov 7, 2024

PR approved by anyone and no changes requested.

@doris-robot
Copy link

TPC-H: Total hot run time: 41367 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit daff2f6e358926dfb310822282da9f4411618c2d, data reload: false

------ Round 1 ----------------------------------
q1	17599	7671	7307	7307
q2	2072	182	189	182
q3	10597	1129	1181	1129
q4	10459	843	824	824
q5	7789	3128	3065	3065
q6	234	150	150	150
q7	1002	612	599	599
q8	9402	2012	2073	2012
q9	6754	6487	6474	6474
q10	7079	2398	2450	2398
q11	479	264	259	259
q12	407	217	210	210
q13	17758	3063	3011	3011
q14	248	214	215	214
q15	581	540	513	513
q16	686	594	579	579
q17	999	569	549	549
q18	7375	6649	6849	6649
q19	1338	1047	967	967
q20	483	183	180	180
q21	4013	3231	3098	3098
q22	1112	998	1020	998
Total cold run time: 108466 ms
Total hot run time: 41367 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7272	7281	7263	7263
q2	344	249	247	247
q3	2947	2823	2807	2807
q4	1939	1685	1743	1685
q5	5450	5521	5561	5521
q6	219	136	136	136
q7	2145	1719	1683	1683
q8	3234	3453	3455	3453
q9	8560	8600	8605	8600
q10	3512	3451	3466	3451
q11	604	493	519	493
q12	797	562	569	562
q13	11175	3024	3037	3024
q14	301	260	260	260
q15	586	537	540	537
q16	688	630	634	630
q17	1804	1601	1545	1545
q18	7867	7646	7570	7570
q19	1679	1495	1526	1495
q20	2056	1883	1821	1821
q21	5502	5201	5293	5201
q22	1104	1023	1011	1011
Total cold run time: 69785 ms
Total hot run time: 58995 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 192383 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit daff2f6e358926dfb310822282da9f4411618c2d, data reload: false

query1	969	377	378	377
query2	5932	2030	2110	2030
query3	6716	210	212	210
query4	33895	23741	23582	23582
query5	4324	460	446	446
query6	262	184	193	184
query7	4582	291	288	288
query8	295	222	222	222
query9	9448	2737	2740	2737
query10	461	259	251	251
query11	18240	15421	15336	15336
query12	151	106	99	99
query13	1667	430	402	402
query14	10452	6627	6603	6603
query15	222	181	184	181
query16	7012	479	447	447
query17	1672	579	556	556
query18	2337	558	609	558
query19	410	179	181	179
query20	119	112	108	108
query21	210	101	102	101
query22	4557	4365	4484	4365
query23	35040	34116	34107	34107
query24	11779	3299	3365	3299
query25	579	399	415	399
query26	1121	174	176	174
query27	2881	286	275	275
query28	8254	2460	2433	2433
query29	698	437	424	424
query30	463	314	319	314
query31	1043	801	810	801
query32	68	57	60	57
query33	753	297	296	296
query34	924	504	518	504
query35	876	761	720	720
query36	1126	941	953	941
query37	120	75	70	70
query38	4317	4198	4233	4198
query39	1469	1442	1426	1426
query40	284	104	104	104
query41	52	48	47	47
query42	112	102	100	100
query43	543	496	498	496
query44	1286	817	816	816
query45	184	170	171	170
query46	1144	715	689	689
query47	1934	1832	1828	1828
query48	422	321	338	321
query49	1157	393	395	393
query50	815	380	388	380
query51	7464	7152	7059	7059
query52	101	93	92	92
query53	264	186	179	179
query54	1063	417	409	409
query55	80	79	76	76
query56	276	253	258	253
query57	1295	1178	1169	1169
query58	221	229	212	212
query59	3280	2954	2828	2828
query60	270	244	239	239
query61	115	106	107	106
query62	864	675	691	675
query63	210	190	183	183
query64	3516	658	614	614
query65	3600	3239	3239	3239
query66	1019	321	304	304
query67	16178	15621	15904	15621
query68	4960	590	573	573
query69	432	257	248	248
query70	1199	1163	1094	1094
query71	368	257	254	254
query72	6190	4018	4043	4018
query73	783	360	368	360
query74	10254	9051	8970	8970
query75	3425	2700	2680	2680
query76	2257	1170	1064	1064
query77	381	290	270	270
query78	10536	9396	9436	9396
query79	1342	579	592	579
query80	844	439	438	438
query81	553	244	240	240
query82	1185	123	118	118
query83	236	157	158	157
query84	239	77	78	77
query85	1257	304	296	296
query86	372	303	302	302
query87	4841	4640	4709	4640
query88	3522	2239	2184	2184
query89	401	285	295	285
query90	1938	190	186	186
query91	139	105	106	105
query92	58	52	48	48
query93	1061	534	541	534
query94	901	294	284	284
query95	346	246	250	246
query96	611	278	276	276
query97	2875	2676	2751	2676
query98	213	192	200	192
query99	1462	1293	1291	1291
Total cold run time: 298374 ms
Total hot run time: 192383 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.72 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit daff2f6e358926dfb310822282da9f4411618c2d, data reload: false

query1	0.04	0.03	0.02
query2	0.06	0.03	0.03
query3	0.24	0.06	0.06
query4	1.65	0.10	0.10
query5	0.42	0.40	0.40
query6	1.12	0.65	0.66
query7	0.02	0.02	0.01
query8	0.04	0.02	0.03
query9	0.56	0.50	0.50
query10	0.57	0.56	0.56
query11	0.14	0.10	0.11
query12	0.15	0.11	0.11
query13	0.61	0.59	0.60
query14	2.80	2.82	2.86
query15	0.88	0.82	0.81
query16	0.39	0.39	0.39
query17	1.05	0.99	1.07
query18	0.20	0.19	0.20
query19	1.89	1.85	2.03
query20	0.02	0.01	0.01
query21	15.36	0.60	0.57
query22	2.83	2.14	1.85
query23	17.08	1.05	0.84
query24	2.63	1.17	1.48
query25	0.27	0.05	0.19
query26	0.35	0.13	0.12
query27	0.04	0.04	0.04
query28	10.44	1.09	1.07
query29	12.52	3.26	3.23
query30	0.25	0.05	0.06
query31	2.88	0.38	0.36
query32	3.29	0.46	0.45
query33	2.99	3.03	3.17
query34	17.18	4.43	4.45
query35	4.47	4.58	4.47
query36	0.66	0.48	0.47
query37	0.10	0.06	0.06
query38	0.05	0.03	0.04
query39	0.03	0.02	0.02
query40	0.18	0.12	0.12
query41	0.08	0.03	0.03
query42	0.03	0.02	0.02
query43	0.03	0.03	0.03
Total cold run time: 106.59 s
Total hot run time: 32.72 s

Copy link
Contributor

@liaoxin01 liaoxin01 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@sollhui sollhui changed the title [fix](cloud) fix version of information_schema table backward when query in follower node [fix](cloud txn version) fix version of information_schema table backward when query in follower node Nov 8, 2024
@sollhui sollhui changed the title [fix](cloud txn version) fix version of information_schema table backward when query in follower node [fix](cloud version) fix version of information_schema table backward when query in follower node Nov 8, 2024
@liaoxin01 liaoxin01 merged commit c57041a into apache:master Nov 8, 2024
28 of 30 checks passed
github-actions bot pushed a commit that referenced this pull request Nov 8, 2024
… when query in follower node (#43451)

Problem Summary:

For the follower node in cloud mode, when querying the
information_schema table, the version needs to be updated.
Otherwise, the version will always be the old value unless there is a
query for the table in the follower node.
@gavinchou gavinchou added cir and removed approved Indicates a PR has been approved by one committer. labels Nov 11, 2024
gavinchou pushed a commit that referenced this pull request Nov 13, 2024
…ble backward when query in follower node (#43496)

Cherry-picked from #43451

Co-authored-by: hui lai <1353307710@qq.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants