Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add multi-row support for Database.insert #32

Merged
merged 1 commit into from
Sep 28, 2023
Merged

Conversation

a-hurst
Copy link
Owner

@a-hurst a-hurst commented Sep 28, 2023

PR Description

This PR adds adds multi-row insertion support to the KLDatabase.Database.insert method, providing massive speedups when inserting many rows of data at a a time.

Previously, inserting rows one-by-one would commit the changes to the database after every row, which is a slow I/O operation (especially on computers with slower storage). When inserting hundreds of rows at once, it could take over 2 seconds in some cases! With these changes, when providing multiple rows to insert in a list, the database changes are only committed once at the end, increasing performance to ~30ms for the same data that took over 2 seconds during testing.

Merge Checklist

  • the PR has been reviewed and all comments are resolved
  • all CI checks pass
  • (if applicable): the PR description includes the phrase closes #<issue-number> to automatically close an issue
  • (if applicable): bug fixes, new features, or API changes are documented in CHANGELOG.rst

@a-hurst a-hurst merged commit 69a60cd into testing Sep 28, 2023
18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant