Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

why tx.db.grow() is not needed in tx.root.spill() #285

Closed
uvletter opened this issue Jul 5, 2021 · 3 comments
Closed

why tx.db.grow() is not needed in tx.root.spill() #285

uvletter opened this issue Jul 5, 2021 · 3 comments

Comments

@uvletter
Copy link
Contributor

uvletter commented Jul 5, 2021

I'm learning the source code of boltdb recently. In Commit tx.root.spill() would shift tx.meta.pgid possiblely when there is no enough room in freelist, but without a tx.db.grow(). If the following freelist serialization find a feat page in the freelist, the tx.db.grow() would be eliminated, is it the expected behave?
Thanks a lot

@ptabor
Copy link
Contributor

ptabor commented Dec 30, 2022

It's interesting question...

What I wanted to say is that 'grow' is postponed till tx.commitFreelist() is called:

bbolt/tx.go

Lines 239 to 240 in b654ce9

if tx.meta.pgid > opgid {
if err := tx.db.grow(int(tx.meta.pgid+1) * tx.db.pageSize); err != nil {

But... the commitFreelist is only called when we are NOT operating in NoFreelistSync (that's getting more popular):

bbolt/tx.go

Lines 174 to 175 in b654ce9

if !tx.db.NoFreelistSync {
err := tx.commitFreelist()

Intuitively the growing part should be migrated out of the 'commitFreelist' call.

I think it justifies an attempt to create a repro test for this issue.

@uvletter
Copy link
Contributor Author

From the git history grow is introduced in boltdb/bolt#284, to solve the issue fdatasync is unsafe in some particular FS and OS version. As far as I can tell this issue has been fixed in the newer Linux kernel. As a side effect Truncate improve the performance by mitigatethe possibility of sync the metadata when flush the dirty page. But all the above will have no effect if NoFreelistSync is on...

@ahrtr
Copy link
Member

ahrtr commented Feb 1, 2023

My understanding is both spill and commitFreelist are operating on virtual memory, it's only necessary to resize the file before syncing data to db file.

@ahrtr ahrtr closed this as completed Feb 1, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

No branches or pull requests

3 participants