core: add support for bloom filters #99
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
arpitbbhayani
requested changes
Dec 24, 2022
core/bloom.go
Outdated
Comment on lines
329
to
358
| func setBit(buf []byte, b int) { | ||
| if b < 0 { | ||
| return | ||
| } | ||
|
|
||
| idx, offset := b/8, 7-b%8 | ||
| if idx < 0 || idx >= len(buf) { | ||
| return | ||
| } | ||
|
|
||
| buf[idx] = buf[idx] | 1<<offset | ||
| } | ||
|
|
||
| // isBitSet checks if the bit at index `b` is set to "1" or not in `buf`. | ||
| func isBitSet(buf []byte, b int) bool { | ||
| if b < 0 { | ||
| return false | ||
| } | ||
|
|
||
| idx, offset := b/8, 7-b%8 | ||
| if idx >= len(buf) { | ||
| return false | ||
| } | ||
|
|
||
| if buf[idx]&(1<<offset) == 1<<offset { | ||
| return true | ||
| } else { | ||
| return false | ||
| } | ||
| } |
There was a problem hiding this comment.
There is a util directory where you can put byte level utils; also we have something similar for varint and bytelist. Check that implementation once and see if we need to converge.
There was a problem hiding this comment.
Hi, at the moment, I don't think there's any code which can be converged. Bytelist holds some extra metadata and we might not need any of it for bloom bytes. Although I can move some logic to bloom_utils to separate the logic.
arpitbbhayani
approved these changes
Jan 20, 2023
| @@ -24,6 +24,9 @@ var OBJ_ENCODING_QREF uint8 = 1 | |||
| var OBJ_ENCODING_STACKINT uint8 = 2 | |||
| var OBJ_ENCODING_STACKREF uint8 = 3 | |||
|
|
|||
| var OBJ_TYPE_BITSET uint8 = 1 << 5 // 00100000 | |||
There was a problem hiding this comment.
This is a great decision; as we cannot use go strings as byte arrays easily.
Comment on lines
+22
to
+24
| } else { | ||
| return false | ||
| } |
There was a problem hiding this comment.
don't need an else here; just a blanket return false
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR attempts to add bloom filters in dice db (See issue: #36).
The commands supported are mentioned below:
Returns "OK" in case of successful bloom filter creation and relevant error if not.
Returns "1" in case of successful addition to the filter and "0" if the element (or some other elements which set the same bits) was already added before. It also returns "-1" in case of errors. Moreover, if the filter does not exist, it will create one with defaults.
Returns "1" is the element may (or may not) exist in the filter and "0" is the element surely doesn't exist. It also returns "-1" in case of errors.
Returns the parameters and metadata of the filter.
Sufficient unit tests for most of the functions have been added in this PR.
Future work (not necessary in this PR)