Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Better Tombstone handling in Iter #21

Merged
merged 3 commits into from
Feb 9, 2022
Merged

Better Tombstone handling in Iter #21

merged 3 commits into from
Feb 9, 2022

Conversation

bbengfort
Copy link
Contributor

Adds an option for iterating with tombstones but by default the Iter function skips tombstones to match the Get functionality. Also fixes the colons in namespace issue by removing the namespace prefix rather than splitting on the separator. Increases tests to ensure that Iter covers undead objects.

Note, this will require a change to Trtl replication, it must use the WithTombstone option when it is iterating in initiator phase 1 and remote phase 2; otherwise deletes will not be replicated.

Adds an option for iterating with tombstones but by default the Iter
function skips tombstones to match the Get functionality. Also fixes the
colons in namespace issue by removing the namespace prefix rather than
splitting on the separator. Increases tests to ensure that Iter covers
undead objects.

Note, this will require a change to Trtl replication, it must use the
WithTombstone option when it is iterating in initiator phase 1 and
remote phase 2; otherwise deletes will not be replicated.
@bbengfort bbengfort requested a review from pdeziel February 9, 2022 14:17
@codecov
Copy link

codecov bot commented Feb 9, 2022

Codecov Report

Merging #21 (d4b4b13) into main (59182a3) will increase coverage by 0.63%.
The diff coverage is 57.14%.

❗ Current head d4b4b13 differs from pull request most recent head f39d7c1. Consider uploading reports for the commit f39d7c1 to get more accurate results

Impacted file tree graph

@@            Coverage Diff             @@
##             main      #21      +/-   ##
==========================================
+ Coverage   55.77%   56.41%   +0.63%     
==========================================
  Files          10       10              
  Lines         554      569      +15     
==========================================
+ Hits          309      321      +12     
- Misses        200      203       +3     
  Partials       45       45              
Impacted Files Coverage Δ
engines/leveldb/iter.go 45.00% <47.82%> (-3.28%) ⬇️
engines/leveldb/leveldb.go 84.70% <100.00%> (ø)
options/options.go 93.54% <100.00%> (+15.77%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 59182a3...f39d7c1. Read the comment docs.

}
}

func TestTombstonesMultipleNamespaces(t *testing.T) {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@pdeziel I'm hoping this resolves SC-3389; let me know if the logic doesn't look right to you.

Copy link
Contributor

@pdeziel pdeziel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for fixing the problem and adding the test cases, I have a lot more confidence that we are handling the tombstones correctly. I just had a few questions/comments below.

engines/leveldb/leveldb_test.go Outdated Show resolved Hide resolved
// Teardown after finishing the test
defer os.RemoveAll(ldbPath)
defer ldbEngine.Close()
ldbEngine, _ := setupLevelDBEngine(t)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for adding the cleanup func, it makes the teardown code a lot neater.

iter.Release()

require.Equal(t, expected, actual, "database key count does not match")
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea to add these utilities.

func TestTombstonesMultipleNamespaces(t *testing.T) {
// Create a test database
db, _ := setupHonuDB(t)
namespaces := []string{"graveyard", "cemetery", "catacombs"}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 for the theming

// If we aren't including Tombstones, we need to check if the next version is a
// tombstone before we know if we have a next value or not.
if !i.options.Tombstones {
if obj, err := i.Object(); err != nil || obj.Tombstone() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are we ignoring the error here or does it get captured somewhere else?

@bbengfort bbengfort merged commit 396d314 into main Feb 9, 2022
@bbengfort bbengfort deleted the tombstone-iter branch February 9, 2022 18:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants