Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[improvement](jdbc catalog) Added support for Oracle Raw type #37078

Merged
merged 2 commits into from
Jul 9, 2024

Conversation

zy-kkk
Copy link
Member

@zy-kkk zy-kkk commented Jul 1, 2024

In previous versions, we adopted the strategy of reading the object address for Oracle's raw type, which would lead to unstable and meaningless results. Here I changed it to read hexadecimal or UTF8

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@zy-kkk
Copy link
Member Author

zy-kkk commented Jul 1, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 39551 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 03f623aa0e43a4a1f7a4cbe1f0201429ce456c37, data reload: false

------ Round 1 ----------------------------------
q1	17957	4462	4377	4377
q2	2552	195	192	192
q3	11117	1228	1066	1066
q4	10396	780	814	780
q5	7992	2642	2567	2567
q6	223	144	137	137
q7	963	622	626	622
q8	9373	2075	2065	2065
q9	8871	6471	6438	6438
q10	8988	3711	3702	3702
q11	463	242	237	237
q12	431	246	232	232
q13	17767	3013	2961	2961
q14	263	216	225	216
q15	538	483	477	477
q16	491	374	375	374
q17	968	647	700	647
q18	8131	7433	7275	7275
q19	3047	1509	1339	1339
q20	679	318	331	318
q21	4955	3195	4017	3195
q22	410	334	345	334
Total cold run time: 116575 ms
Total hot run time: 39551 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4392	4252	4241	4241
q2	395	272	266	266
q3	3000	2713	2730	2713
q4	1836	1600	1579	1579
q5	5239	5251	5284	5251
q6	217	128	129	128
q7	2122	1709	1746	1709
q8	3200	3308	3353	3308
q9	8248	8234	8308	8234
q10	3879	3643	3656	3643
q11	582	470	478	470
q12	800	631	593	593
q13	17565	3000	2998	2998
q14	292	263	263	263
q15	527	472	484	472
q16	462	407	442	407
q17	1767	1470	1486	1470
q18	7653	7606	7400	7400
q19	1702	1581	1501	1501
q20	2004	1779	1791	1779
q21	4794	4844	4620	4620
q22	658	559	582	559
Total cold run time: 71334 ms
Total hot run time: 53604 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 173550 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 03f623aa0e43a4a1f7a4cbe1f0201429ce456c37, data reload: false

query1	912	402	365	365
query2	6455	2380	2373	2373
query3	6656	211	224	211
query4	18852	17517	17425	17425
query5	4194	481	494	481
query6	273	176	170	170
query7	4564	286	295	286
query8	303	301	289	289
query9	8483	2398	2348	2348
query10	597	290	282	282
query11	10429	10278	10210	10210
query12	138	84	84	84
query13	1588	372	356	356
query14	9945	7624	6913	6913
query15	224	181	185	181
query16	7944	272	265	265
query17	1908	518	522	518
query18	2020	274	266	266
query19	195	153	156	153
query20	94	83	82	82
query21	214	149	134	134
query22	4580	4191	4170	4170
query23	33912	33275	33189	33189
query24	12045	2808	2977	2808
query25	689	386	385	385
query26	1786	154	156	154
query27	3024	324	313	313
query28	7630	2064	2059	2059
query29	1112	641	633	633
query30	275	158	156	156
query31	947	734	745	734
query32	99	55	55	55
query33	787	314	313	313
query34	922	472	481	472
query35	778	672	615	615
query36	1106	937	948	937
query37	286	71	73	71
query38	2851	2747	2776	2747
query39	883	788	792	788
query40	281	126	123	123
query41	57	52	54	52
query42	124	97	101	97
query43	578	538	506	506
query44	1240	716	722	716
query45	186	171	175	171
query46	1075	727	715	715
query47	1844	1791	1771	1771
query48	371	296	297	296
query49	1200	430	399	399
query50	761	389	401	389
query51	6889	7058	6703	6703
query52	117	93	93	93
query53	363	294	306	294
query54	906	435	443	435
query55	75	77	73	73
query56	279	267	268	267
query57	1172	1055	1040	1040
query58	270	245	263	245
query59	3325	3107	3249	3107
query60	304	277	266	266
query61	93	101	122	101
query62	651	428	429	428
query63	329	305	293	293
query64	9851	2217	1778	1778
query65	3165	3093	3110	3093
query66	1395	359	339	339
query67	15715	14932	15000	14932
query68	5067	539	552	539
query69	614	381	321	321
query70	1068	1080	1078	1078
query71	448	281	268	268
query72	7766	5947	5246	5246
query73	747	327	328	327
query74	5966	5560	5509	5509
query75	3767	2690	2687	2687
query76	3726	997	939	939
query77	657	296	319	296
query78	10384	9947	9913	9913
query79	3237	515	517	515
query80	2060	472	468	468
query81	581	220	223	220
query82	1247	113	110	110
query83	334	176	171	171
query84	272	93	87	87
query85	1593	299	276	276
query86	469	333	319	319
query87	3295	3077	3084	3077
query88	3828	2371	2376	2371
query89	475	394	381	381
query90	1782	188	203	188
query91	129	98	168	98
query92	64	49	53	49
query93	2946	512	503	503
query94	1175	190	186	186
query95	406	317	316	316
query96	597	271	271	271
query97	3175	3084	3065	3065
query98	208	198	194	194
query99	1261	863	849	849
Total cold run time: 281660 ms
Total hot run time: 173550 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.81 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 03f623aa0e43a4a1f7a4cbe1f0201429ce456c37, data reload: false

query1	0.04	0.03	0.03
query2	0.08	0.04	0.04
query3	0.22	0.05	0.06
query4	1.66	0.10	0.10
query5	0.50	0.49	0.47
query6	1.14	0.73	0.73
query7	0.02	0.02	0.01
query8	0.05	0.05	0.04
query9	0.54	0.48	0.48
query10	0.54	0.54	0.54
query11	0.15	0.10	0.12
query12	0.15	0.13	0.12
query13	0.59	0.60	0.60
query14	0.78	0.79	0.76
query15	0.82	0.82	0.81
query16	0.36	0.38	0.37
query17	1.00	0.98	1.03
query18	0.22	0.23	0.28
query19	1.76	1.68	1.72
query20	0.01	0.01	0.01
query21	15.43	0.75	0.65
query22	4.41	6.08	2.20
query23	18.28	1.36	1.29
query24	2.17	0.23	0.22
query25	0.15	0.08	0.10
query26	0.29	0.17	0.17
query27	0.08	0.08	0.08
query28	13.17	1.03	1.01
query29	12.62	3.29	3.26
query30	0.26	0.06	0.06
query31	2.87	0.38	0.39
query32	3.29	0.46	0.48
query33	2.83	2.92	2.91
query34	17.17	4.43	4.37
query35	4.50	4.46	4.54
query36	0.64	0.46	0.48
query37	0.18	0.16	0.16
query38	0.15	0.15	0.15
query39	0.04	0.03	0.04
query40	0.16	0.15	0.14
query41	0.10	0.05	0.04
query42	0.06	0.05	0.04
query43	0.04	0.04	0.04
Total cold run time: 109.52 s
Total hot run time: 30.81 s

@zy-kkk
Copy link
Member Author

zy-kkk commented Jul 2, 2024

run feut

@zy-kkk
Copy link
Member Author

zy-kkk commented Jul 5, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 40149 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit c23e423544fa8471d542a4788c0dcdfb63d55125, data reload: false

------ Round 1 ----------------------------------
q1	17823	4777	4409	4409
q2	2505	203	196	196
q3	11114	1238	1180	1180
q4	10350	858	779	779
q5	7553	2754	2709	2709
q6	222	142	143	142
q7	981	615	630	615
q8	9589	2093	2111	2093
q9	9146	6500	6500	6500
q10	8929	3741	3700	3700
q11	479	245	234	234
q12	414	237	232	232
q13	17760	3000	3042	3000
q14	275	242	242	242
q15	549	496	495	495
q16	508	380	375	375
q17	977	655	667	655
q18	8118	7529	7392	7392
q19	5128	1534	1313	1313
q20	698	330	328	328
q21	4980	3222	3870	3222
q22	401	343	338	338
Total cold run time: 118499 ms
Total hot run time: 40149 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4325	4265	4217	4217
q2	377	267	259	259
q3	2999	2801	2737	2737
q4	1896	1634	1641	1634
q5	5302	5325	5310	5310
q6	224	133	137	133
q7	2123	1694	1772	1694
q8	3217	3397	3335	3335
q9	8352	8345	8394	8345
q10	3903	3650	3702	3650
q11	575	510	491	491
q12	792	625	593	593
q13	16412	3018	3028	3018
q14	292	283	253	253
q15	522	482	492	482
q16	477	411	435	411
q17	1793	1492	1475	1475
q18	7716	7597	7336	7336
q19	1702	1554	1512	1512
q20	2025	1809	1790	1790
q21	4839	4809	4644	4644
q22	636	560	533	533
Total cold run time: 70499 ms
Total hot run time: 53852 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 172816 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit c23e423544fa8471d542a4788c0dcdfb63d55125, data reload: false

query1	913	375	376	375
query2	6468	2468	2284	2284
query3	6655	231	211	211
query4	28466	17699	17292	17292
query5	4199	492	487	487
query6	294	175	174	174
query7	4610	293	285	285
query8	310	292	284	284
query9	8639	2413	2401	2401
query10	605	279	292	279
query11	11963	10038	10045	10038
query12	145	93	86	86
query13	1643	377	376	376
query14	10226	7881	7668	7668
query15	298	188	194	188
query16	8095	319	317	317
query17	1801	575	539	539
query18	2101	294	284	284
query19	204	174	163	163
query20	93	86	87	86
query21	217	127	128	127
query22	4351	3979	3962	3962
query23	34047	33087	33254	33087
query24	11185	2826	2770	2770
query25	632	382	394	382
query26	1238	160	166	160
query27	2775	324	321	321
query28	7451	2084	2075	2075
query29	880	650	637	637
query30	286	150	149	149
query31	976	747	755	747
query32	89	56	56	56
query33	794	317	301	301
query34	930	506	478	478
query35	736	669	623	623
query36	1119	930	942	930
query37	137	80	76	76
query38	2864	2741	2720	2720
query39	869	802	828	802
query40	292	132	133	132
query41	56	55	53	53
query42	127	99	106	99
query43	602	566	548	548
query44	1227	759	728	728
query45	192	170	162	162
query46	1075	729	727	727
query47	1847	1766	1760	1760
query48	395	298	296	296
query49	1106	445	437	437
query50	773	403	398	398
query51	6901	6667	6692	6667
query52	107	96	98	96
query53	372	306	295	295
query54	929	463	465	463
query55	79	79	76	76
query56	312	300	302	300
query57	1171	1048	1038	1038
query58	272	276	268	268
query59	3484	3159	3380	3159
query60	298	284	294	284
query61	95	117	93	93
query62	650	445	441	441
query63	329	295	291	291
query64	9655	2201	1637	1637
query65	3164	3127	3150	3127
query66	1405	336	335	335
query67	15653	14842	14728	14728
query68	7267	559	548	548
query69	734	459	383	383
query70	1213	1091	1131	1091
query71	529	285	318	285
query72	8913	5353	5892	5353
query73	810	323	324	323
query74	5983	5443	5446	5443
query75	5095	2673	2664	2664
query76	5015	960	1007	960
query77	792	318	315	315
query78	9521	8956	12879	8956
query79	6483	515	510	510
query80	986	470	517	470
query81	576	220	222	220
query82	361	110	109	109
query83	335	165	169	165
query84	275	85	88	85
query85	1425	308	311	308
query86	442	332	309	309
query87	3269	3143	3096	3096
query88	4060	2349	2344	2344
query89	477	377	391	377
query90	2034	185	186	185
query91	137	103	143	103
query92	65	49	51	49
query93	1731	515	505	505
query94	1264	210	204	204
query95	408	319	313	313
query96	590	269	268	268
query97	3242	3016	3003	3003
query98	220	202	191	191
query99	1162	823	833	823
Total cold run time: 297286 ms
Total hot run time: 172816 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.46 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit c23e423544fa8471d542a4788c0dcdfb63d55125, data reload: false

query1	0.03	0.03	0.03
query2	0.07	0.04	0.04
query3	0.23	0.05	0.05
query4	1.68	0.07	0.07
query5	0.49	0.48	0.48
query6	1.14	0.74	0.73
query7	0.02	0.02	0.01
query8	0.05	0.04	0.04
query9	0.54	0.49	0.49
query10	0.55	0.55	0.54
query11	0.16	0.11	0.11
query12	0.14	0.12	0.13
query13	0.59	0.58	0.58
query14	0.77	0.78	0.78
query15	0.85	0.82	0.82
query16	0.38	0.36	0.36
query17	1.03	0.95	1.01
query18	0.21	0.25	0.26
query19	1.83	1.79	1.71
query20	0.01	0.01	0.01
query21	15.40	0.77	0.66
query22	3.83	7.28	1.85
query23	18.29	1.39	1.30
query24	2.10	0.22	0.21
query25	0.17	0.10	0.08
query26	0.31	0.21	0.20
query27	0.46	0.23	0.23
query28	13.31	1.03	1.00
query29	12.65	3.24	3.32
query30	0.26	0.06	0.06
query31	2.85	0.39	0.40
query32	3.27	0.48	0.47
query33	2.90	2.98	2.86
query34	17.05	4.36	4.33
query35	4.43	4.39	4.50
query36	0.66	0.47	0.46
query37	0.18	0.15	0.15
query38	0.15	0.14	0.14
query39	0.05	0.03	0.04
query40	0.16	0.12	0.13
query41	0.10	0.04	0.04
query42	0.05	0.05	0.04
query43	0.04	0.04	0.04
Total cold run time: 109.44 s
Total hot run time: 30.46 s

@zy-kkk
Copy link
Member Author

zy-kkk commented Jul 8, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 39924 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 007f4e9bcf900df0caa2f3cd4be972a468eb3027, data reload: false

------ Round 1 ----------------------------------
q1	17598	4385	4247	4247
q2	2022	188	196	188
q3	10435	1138	1216	1138
q4	10180	821	815	815
q5	7489	2657	2661	2657
q6	220	134	135	134
q7	946	590	600	590
q8	9239	2056	2095	2056
q9	8956	6487	6476	6476
q10	8902	3694	3687	3687
q11	441	234	235	234
q12	445	221	219	219
q13	17842	2963	2951	2951
q14	274	232	243	232
q15	516	482	504	482
q16	513	380	373	373
q17	952	659	641	641
q18	7973	7480	7483	7480
q19	5936	1534	1550	1534
q20	672	317	332	317
q21	4866	3135	3947	3135
q22	400	338	343	338
Total cold run time: 116817 ms
Total hot run time: 39924 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4383	4223	4233	4223
q2	369	263	270	263
q3	2978	2839	2870	2839
q4	1945	1709	1712	1709
q5	5676	5469	5461	5461
q6	223	128	130	128
q7	2191	1879	1848	1848
q8	3318	3388	3384	3384
q9	8674	8764	8823	8764
q10	4074	3792	3764	3764
q11	597	496	505	496
q12	827	636	636	636
q13	16273	3155	3133	3133
q14	297	277	275	275
q15	538	494	477	477
q16	481	433	440	433
q17	1807	1511	1499	1499
q18	8116	7857	7758	7758
q19	1803	1640	1722	1640
q20	2188	1849	1866	1849
q21	5226	5025	4600	4600
q22	612	573	559	559
Total cold run time: 72596 ms
Total hot run time: 55738 ms

Copy link
Contributor

@morningman morningman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jul 8, 2024
Copy link
Contributor

github-actions bot commented Jul 8, 2024

PR approved by at least one committer and no changes requested.

Copy link
Contributor

github-actions bot commented Jul 8, 2024

PR approved by anyone and no changes requested.

@doris-robot
Copy link

TPC-DS: Total hot run time: 173978 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 007f4e9bcf900df0caa2f3cd4be972a468eb3027, data reload: false

query1	907	366	363	363
query2	6446	2750	2585	2585
query3	6636	205	224	205
query4	21095	17438	17471	17438
query5	3577	468	474	468
query6	253	172	170	170
query7	4588	296	288	288
query8	343	282	293	282
query9	8517	2349	2341	2341
query10	597	311	272	272
query11	10996	9858	9930	9858
query12	112	86	79	79
query13	1641	368	370	368
query14	9446	7624	7609	7609
query15	228	187	186	186
query16	7737	312	303	303
query17	1533	538	544	538
query18	1984	304	265	265
query19	189	144	143	143
query20	89	81	81	81
query21	207	133	126	126
query22	4217	3952	4113	3952
query23	33845	33463	33614	33463
query24	11235	2893	2826	2826
query25	592	400	398	398
query26	833	156	152	152
query27	2378	318	324	318
query28	6212	2123	2137	2123
query29	885	638	636	636
query30	254	152	150	150
query31	995	754	740	740
query32	103	60	60	60
query33	758	324	324	324
query34	963	490	498	490
query35	747	622	637	622
query36	1140	985	967	967
query37	142	84	84	84
query38	2950	2883	2854	2854
query39	916	840	847	840
query40	224	130	124	124
query41	58	53	51	51
query42	114	101	95	95
query43	602	574	573	573
query44	1187	724	735	724
query45	187	163	164	163
query46	1083	720	697	697
query47	1855	1744	1785	1744
query48	368	305	303	303
query49	840	405	416	405
query50	768	382	388	382
query51	6963	6872	6785	6785
query52	129	96	90	90
query53	357	290	283	283
query54	955	443	444	443
query55	73	71	70	70
query56	278	256	271	256
query57	1176	1077	1052	1052
query58	238	242	243	242
query59	3696	3417	3235	3235
query60	315	270	267	267
query61	94	91	94	91
query62	597	447	439	439
query63	324	290	282	282
query64	9129	2269	1690	1690
query65	3142	3095	3128	3095
query66	750	326	334	326
query67	15458	14929	14884	14884
query68	4474	526	535	526
query69	514	369	322	322
query70	1192	1061	1075	1061
query71	394	278	274	274
query72	7330	5762	5838	5762
query73	765	331	330	330
query74	6298	5696	5482	5482
query75	3316	2684	2655	2655
query76	2349	924	982	924
query77	451	299	311	299
query78	9576	8990	8878	8878
query79	2876	510	522	510
query80	2424	466	475	466
query81	620	218	226	218
query82	1488	108	102	102
query83	305	177	162	162
query84	278	100	83	83
query85	1384	390	307	307
query86	463	326	304	304
query87	3317	3064	3111	3064
query88	4200	2439	2442	2439
query89	473	398	376	376
query90	1729	188	184	184
query91	130	100	101	100
query92	57	49	48	48
query93	2637	498	496	496
query94	1096	205	210	205
query95	413	314	317	314
query96	597	272	270	270
query97	3266	3028	3030	3028
query98	223	196	195	195
query99	1146	836	837	836
Total cold run time: 273027 ms
Total hot run time: 173978 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.47 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 007f4e9bcf900df0caa2f3cd4be972a468eb3027, data reload: false

query1	0.04	0.03	0.03
query2	0.08	0.04	0.03
query3	0.23	0.05	0.05
query4	1.68	0.06	0.07
query5	0.51	0.50	0.48
query6	1.12	0.72	0.74
query7	0.02	0.01	0.01
query8	0.05	0.04	0.04
query9	0.55	0.48	0.49
query10	0.55	0.52	0.55
query11	0.16	0.11	0.11
query12	0.14	0.12	0.12
query13	0.59	0.58	0.58
query14	0.76	0.78	0.78
query15	0.83	0.81	0.80
query16	0.38	0.37	0.37
query17	0.94	0.99	1.02
query18	0.23	0.24	0.26
query19	1.84	1.82	1.70
query20	0.01	0.01	0.01
query21	15.39	0.74	0.67
query22	3.75	6.49	2.88
query23	18.31	1.38	1.33
query24	2.13	0.22	0.23
query25	0.17	0.08	0.09
query26	0.29	0.21	0.20
query27	0.46	0.23	0.24
query28	13.24	1.01	0.97
query29	12.57	3.32	3.22
query30	0.25	0.06	0.06
query31	2.86	0.38	0.38
query32	3.28	0.49	0.46
query33	2.91	2.86	2.93
query34	17.10	4.42	4.35
query35	4.45	4.43	4.49
query36	0.64	0.48	0.50
query37	0.18	0.15	0.14
query38	0.15	0.14	0.15
query39	0.04	0.03	0.03
query40	0.16	0.11	0.12
query41	0.10	0.04	0.04
query42	0.06	0.05	0.04
query43	0.04	0.04	0.03
Total cold run time: 109.24 s
Total hot run time: 31.47 s

@zy-kkk zy-kkk merged commit 270f592 into apache:master Jul 9, 2024
26 of 29 checks passed
@zy-kkk zy-kkk deleted the oracle_raw branch July 9, 2024 08:13
zy-kkk added a commit to zy-kkk/doris that referenced this pull request Jul 15, 2024
…#37078)

In previous versions, we adopted the strategy of reading the object
address for Oracle's raw type, which would lead to unstable and
meaningless results. Here I changed it to read hexadecimal or UTF8
zy-kkk added a commit that referenced this pull request Jul 15, 2024
…type (#37776)

pick (#37078)
In previous versions, we adopted the strategy of reading the object
address for Oracle's raw type, which would lead to unstable and
meaningless results. Here I changed it to read hexadecimal or UTF8
dataroaring pushed a commit that referenced this pull request Jul 17, 2024
In previous versions, we adopted the strategy of reading the object
address for Oracle's raw type, which would lead to unstable and
meaningless results. Here I changed it to read hexadecimal or UTF8
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants