Skip to content

Automatic blob garbage collection #157

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Draft
wants to merge 62 commits into
base: 3.0.0
Choose a base branch
from
Draft

Automatic blob garbage collection #157

wants to merge 62 commits into from

Conversation

marvin-j97
Copy link
Contributor

@marvin-j97 marvin-j97 commented Sep 28, 2025 •
edited
Loading

  • Add blob fragmentation map to Version (+ ser/de)
  • Add optional callback to CompactionStream to listen for expired KVs
  • Install callback to listen for expired blobs if tree is key-value separated
  • Store referenced blob file IDs in table file on flush and compaction
  • Remove all superseded old vlog + GC code
  • Add a bunch of tests for blob tree

@marvin-j97 marvin-j97 added this to the 3.0.0 milestone Sep 28, 2025
@marvin-j97 marvin-j97 self-assigned this Sep 28, 2025
@marvin-j97 marvin-j97 changed the base branch from main to 3.0.0 September 28, 2025 15:26
@marvin-j97 marvin-j97 mentioned this pull request Oct 5, 2025
Open
12 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees

@marvin-j97 marvin-j97

Labels
api enhancement New feature or request epic performance refactor type:compaction type:table type:vlog
Projects
None yet
Milestone
3.0.0
Development

Successfully merging this pull request may close these issues.
1 participant
@marvin-j97