Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[string performence improvement] #3288

Merged
merged 5 commits into from
Dec 8, 2021

Conversation

Veeupup
Copy link
Contributor

@Veeupup Veeupup commented Dec 7, 2021

I hereby agree to the terms of the CLA available at: https://databend.rs/policies/cla/

Summary

improve string function performance

  • almost 50% speed improvement in release mode
  • 100% extra memory copy improvement in base64/quote/trim/reverse string function

before:
image

now:
image

Changelog

  • Performance Improvement

Related Issues

Fixes #3259

Test Plan

Unit Tests

Stateless Tests

@vercel
Copy link

vercel bot commented Dec 7, 2021

This pull request is being automatically deployed with Vercel (learn more).
To see the status of your deployment, click below or on the icon next to each commit.

🔍 Inspect: https://vercel.com/databend/databend/FPa2TDzJkuyXDr1nnRJsAUtMQxpX
✅ Preview: Canceled

[Deployment for 4046f6b canceled]

@Veeupup Veeupup marked this pull request as ready for review December 8, 2021 06:15
@Veeupup Veeupup requested a review from BohuTANG as a code owner December 8, 2021 06:15
@Veeupup Veeupup requested a review from sundy-li December 8, 2021 06:18
@Veeupup Veeupup marked this pull request as draft December 8, 2021 06:19
@codecov-commenter
Copy link

codecov-commenter commented Dec 8, 2021

Codecov Report

Merging #3288 (77d61f1) into main (7096d74) will increase coverage by 0%.
The diff coverage is 36%.

Impacted file tree graph

@@          Coverage Diff           @@
##            main   #3288    +/-   ##
======================================
  Coverage     65%     65%            
======================================
  Files        705     710     +5     
  Lines      38272   38634   +362     
======================================
+ Hits       24928   25183   +255     
- Misses     13344   13451   +107     
Impacted Files Coverage Δ
common/datavalues/src/arrays/string/mod.rs 55% <ø> (ø)
common/functions/src/scalars/strings/base_64.rs 0% <0%> (ø)
common/functions/src/scalars/strings/quote.rs 0% <0%> (ø)
common/functions/src/scalars/strings/reverse.rs 0% <0%> (ø)
common/datavalues/src/arrays/string/transform.rs 44% <44%> (ø)
...mon/functions/src/scalars/strings/string2string.rs 60% <54%> (-9%) ⬇️
common/functions/src/scalars/strings/trim.rs 84% <69%> (-7%) ⬇️
common/datablocks/src/data_block.rs 75% <0%> (-14%) ⬇️
query/src/interpreters/interpreter_insert.rs 72% <0%> (-11%) ⬇️
common/datavalues/src/data_value.rs 39% <0%> (-3%) ⬇️
... and 28 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7096d74...77d61f1. Read the comment docs.

@Veeupup Veeupup marked this pull request as ready for review December 8, 2021 09:02
@Veeupup Veeupup changed the title [draft] [string performence improvement] [string performence improvement] Dec 8, 2021
@sundy-li
Copy link
Member

sundy-li commented Dec 8, 2021

Is the performance in debug mode or release mode?

@Veeupup
Copy link
Contributor Author

Veeupup commented Dec 8, 2021

Is the performance in debug mode or release mode?

sorry, I test it in debug mode ... I try make it in release mode later

@databend-bot
Copy link
Member

Thanks for the contribution!
I have applied any labels matching special text in your PR Changelog.

Please review the labels and make any necessary changes.

Copy link
Member

@sundy-li sundy-li left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/LGTM

And I think it's time to add transfrom_from_primitive function.

@databend-bot
Copy link
Member

Wait for another reviewer approval

@BohuTANG BohuTANG merged commit d0d34ba into databendlabs:main Dec 8, 2021
@Veeupup Veeupup deleted the string-improvement branch December 9, 2021 01:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Improve performance in functions working with String
5 participants