move from timestamps to epochs for folding and siblings #140
Conversation
- pull in more aggressive pulse tunings from #128 - remove all test sleeps related to old 1 second timestamps, so that things will break if old code is retained.
mutations and iterations. this adds 8 bytes per key, but it's better to use slightly more memory than to be vulnerable to clock skews, and gives us hard freezing boundaries to make folds work as they have done traditionally.
|
force updated a rebase on top of develop. unit tests and cursory pulsing passing again. |
| &ret->itr, &ret->pending_entry) | ||
| && (tstamp >= ret->pending_entry->tstamp)) | ||
| &ret->itr, &ret->pending_entry) | ||
| && (epoch > ret->pending_entry->epoch)) |
There was a problem hiding this comment.
Oops, this should be >= instead.
There was a problem hiding this comment.
All epochs should be unique, if not there is a bug.
|
Ok, so even though the way epochs are used seems a bit counterintuitive to me, they are consistent with current use: they get incremented on iterator creation, not only on update operations. Since they are used on iterators only (normal gets use max_epoch always), it doesn't matter that they pick up only entries with an epoch < than theirs. If we were to forget that in the future and write an operation that puts and uses the generated epoch to view a snapshot including that put, you would have to use epoch+1 to see your put. I hope we remember that if we ever do :) I have been testing this branch in combination with a rebased branch of the merge/delete race fix, the other known issue with Bitcask at the moment. All eunit tests pass, and pulse ran for more than a day without finding a single counterexample. I will be opening that second rebase branch as a separate PR to finish reviewing and testing after this is merged. 👍 and merge away |
move from timestamps to epochs for folding and siblings
move away from timestamps to a per-run epoch counter for all keydir
mutations and iterations. this adds 8 bytes per key, but it's better
to use slightly more memory than to be vulnerable to clock skews, and
gives us hard freezing boundaries to make folds work as they have
done traditionally.