Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[fix](Nereids) not release lock if schema changed when insert #47733

Merged
merged 1 commit into from
Feb 12, 2025
Merged

[fix](Nereids) not release lock if schema changed when insert #47733

merged 1 commit into from
Feb 12, 2025

Conversation

morrySnow
Copy link
Contributor

What problem does this PR solve?

Related PR: #45045

Problem Summary:

not release lock if schema changed when insert

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@hello-stephen
Copy link
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@dataroaring
Copy link
Contributor

run buildall

morrySnow added a commit that referenced this pull request Feb 11, 2025
@morrySnow
branch-3.0: [fix](Nereids) not release lock if schema changed when in…
8f69825 
…sert #47733

pick from master #47733
@morrySnow morrySnow mentioned this pull request Feb 11, 2025
Merged
@shuke987
Copy link
Collaborator

run buildall

@morrySnow
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 31588 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 2bdc399234c2fa5b67d28b63ee0b3e09cedc42c7, data reload: false

------ Round 1 ----------------------------------
q1	17580	5217	5181	5181
q2	2052	312	174	174
q3	10446	1296	706	706
q4	10261	1007	533	533
q5	7900	2306	2368	2306
q6	191	166	133	133
q7	885	739	601	601
q8	9310	1270	1022	1022
q9	4868	4787	4884	4787
q10	6836	2283	1911	1911
q11	483	273	266	266
q12	354	350	219	219
q13	17770	3679	3074	3074
q14	225	226	205	205
q15	507	479	448	448
q16	633	617	592	592
q17	582	867	341	341
q18	6726	6191	6135	6135
q19	1208	943	538	538
q20	313	319	185	185
q21	2753	2134	1910	1910
q22	357	326	321	321
Total cold run time: 102240 ms
Total hot run time: 31588 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5121	5161	5100	5100
q2	237	328	229	229
q3	2134	2623	2266	2266
q4	1408	1802	1353	1353
q5	4222	4105	4152	4105
q6	204	162	124	124
q7	1874	1833	1682	1682
q8	2626	2601	2620	2601
q9	7278	7073	6958	6958
q10	2953	3194	2791	2791
q11	568	525	501	501
q12	692	766	605	605
q13	3493	3923	3227	3227
q14	276	299	270	270
q15	516	463	465	463
q16	642	667	635	635
q17	1125	1623	1319	1319
q18	7466	7420	7123	7123
q19	801	852	934	852
q20	1939	1998	1870	1870
q21	5402	4811	4831	4811
q22	623	607	577	577
Total cold run time: 51600 ms
Total hot run time: 49462 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 182905 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 2bdc399234c2fa5b67d28b63ee0b3e09cedc42c7, data reload: false

query1	962	394	383	383
query2	6524	1823	1805	1805
query3	6799	205	221	205
query4	26301	23351	23089	23089
query5	4385	660	486	486
query6	307	203	193	193
query7	4620	499	313	313
query8	296	243	227	227
query9	8602	2498	2498	2498
query10	449	341	246	246
query11	15409	15080	14794	14794
query12	153	103	103	103
query13	1649	535	381	381
query14	8857	6819	6376	6376
query15	215	186	172	172
query16	7235	620	442	442
query17	1187	689	538	538
query18	1466	379	285	285
query19	195	182	154	154
query20	115	123	115	115
query21	205	120	103	103
query22	4243	4223	4428	4223
query23	34364	33457	32908	32908
query24	7744	2383	2383	2383
query25	522	449	378	378
query26	1232	269	151	151
query27	2146	479	318	318
query28	3958	2388	2368	2368
query29	693	526	411	411
query30	229	184	152	152
query31	936	826	782	782
query32	71	67	59	59
query33	576	342	301	301
query34	777	866	516	516
query35	807	802	740	740
query36	962	989	883	883
query37	123	102	81	81
query38	4085	4140	4096	4096
query39	1449	1427	1430	1427
query40	206	114	104	104
query41	58	58	57	57
query42	129	103	107	103
query43	500	523	472	472
query44	1321	793	817	793
query45	179	168	167	167
query46	856	1044	651	651
query47	1744	1791	1717	1717
query48	369	417	297	297
query49	788	526	445	445
query50	663	725	405	405
query51	4156	4135	4109	4109
query52	109	107	101	101
query53	222	249	181	181
query54	480	476	411	411
query55	82	81	81	81
query56	277	258	253	253
query57	1128	1121	1062	1062
query58	243	232	242	232
query59	2779	2814	2407	2407
query60	281	267	248	248
query61	163	124	117	117
query62	801	689	672	672
query63	227	186	182	182
query64	4373	998	643	643
query65	3169	3153	3170	3153
query66	1147	381	308	308
query67	15941	15590	15356	15356
query68	5146	782	516	516
query69	491	297	261	261
query70	1178	1124	1117	1117
query71	395	283	277	277
query72	5781	3571	3719	3571
query73	746	720	353	353
query74	9273	9067	8783	8783
query75	3158	3134	2664	2664
query76	3205	1164	752	752
query77	466	375	276	276
query78	9889	10075	9275	9275
query79	1776	843	593	593
query80	583	542	437	437
query81	496	274	241	241
query82	215	147	120	120
query83	170	172	160	160
query84	247	95	75	75
query85	740	341	299	299
query86	368	312	277	277
query87	4430	4586	4535	4535
query88	3350	2260	2202	2202
query89	383	318	287	287
query90	2020	189	188	188
query91	132	137	111	111
query92	73	56	63	56
query93	1857	1011	580	580
query94	675	380	303	303
query95	354	267	253	253
query96	483	550	269	269
query97	2731	2864	2737	2737
query98	228	208	206	206
query99	1288	1385	1319	1319
Total cold run time: 265566 ms
Total hot run time: 182905 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.45 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 2bdc399234c2fa5b67d28b63ee0b3e09cedc42c7, data reload: false

query1	0.04	0.03	0.03
query2	0.07	0.03	0.03
query3	0.23	0.07	0.07
query4	1.61	0.10	0.10
query5	0.42	0.42	0.39
query6	1.18	0.66	0.65
query7	0.02	0.02	0.02
query8	0.04	0.04	0.03
query9	0.58	0.51	0.53
query10	0.57	0.58	0.57
query11	0.16	0.10	0.10
query12	0.15	0.11	0.11
query13	0.62	0.59	0.60
query14	2.77	2.84	2.72
query15	0.91	0.85	0.85
query16	0.38	0.38	0.38
query17	1.03	1.01	1.00
query18	0.22	0.20	0.19
query19	1.92	1.83	2.04
query20	0.01	0.01	0.01
query21	15.35	0.90	0.54
query22	0.74	1.04	0.61
query23	15.13	1.36	0.63
query24	6.88	1.16	0.84
query25	0.49	0.25	0.06
query26	0.46	0.17	0.13
query27	0.05	0.05	0.05
query28	10.07	0.87	0.43
query29	12.54	4.01	3.30
query30	0.24	0.09	0.06
query31	2.83	0.58	0.38
query32	3.23	0.56	0.47
query33	2.95	3.04	3.00
query34	15.78	5.07	4.45
query35	4.53	4.52	4.51
query36	0.66	0.49	0.49
query37	0.10	0.06	0.06
query38	0.05	0.04	0.04
query39	0.03	0.03	0.02
query40	0.17	0.14	0.13
query41	0.08	0.03	0.03
query42	0.03	0.02	0.02
query43	0.03	0.02	0.03
Total cold run time: 105.35 s
Total hot run time: 30.45 s

dataroaring pushed a commit that referenced this pull request Feb 11, 2025
@morrySnow
branch-3.0: [fix](Nereids) not release lock if schema changed when in…
24c98c5 
…sert #47733 (#47761)

pick from master #47733
@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Feb 11, 2025
@github-actions GitHub Actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions GitHub Actions
Copy link
Contributor

PR approved by anyone and no changes requested.

@morrySnow morrySnow merged commit d774a06 into apache:master Feb 12, 2025
26 of 27 checks passed
github-actions bot pushed a commit that referenced this pull request Feb 12, 2025
@morrySnow
[fix](Nereids) not release lock if schema changed when insert (#47733)
5d9c595
@github-actions github-actions bot mentioned this pull request Feb 12, 2025
Merged
@morrySnow morrySnow deleted the fix_lock_unlock branch February 12, 2025 10:42
morrySnow added a commit that referenced this pull request Feb 12, 2025
@github-actions @morrySnow
branch-2.1: [fix](Nereids) not release lock if schema changed when in…
f8fb5f9 
…sert #47733 (#47833)

Cherry-picked from #47733

Co-authored-by: morrySnow <zhangwenxin@selectdb.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@924060929 924060929 924060929 approved these changes

@feiniaofeiafei feiniaofeiafei feiniaofeiafei approved these changes

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by one committer. dev/2.1.9-merged dev/3.0.4-merged p0_b reviewed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
7 participants
@morrySnow @hello-stephen @dataroaring @shuke987 @doris-robot @924060929 @feiniaofeiafei