Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement iterator over nodes; refactor key-value iterator #517

Merged
merged 90 commits into from
Feb 13, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
90 commits
Select commit Hold shift + click to select a range
0e66425
WIP add node iterator types/scaffold
Feb 5, 2024
40a1407
nit
Feb 5, 2024
bb4c8ca
change return type of poll_next to be node
Feb 5, 2024
f386809
WIP node iterator implementation
Feb 5, 2024
b4eaec6
WIP node iterator implementation
Feb 5, 2024
b0a17a9
WIP node iterator implementation
Feb 5, 2024
587c295
WIP node iterator implementation
Feb 5, 2024
995d9f8
WIP node iterator implementation
Feb 5, 2024
b284a47
WIP node iterator implementation
Feb 5, 2024
f5f379b
WIP node iterator implementation
Feb 5, 2024
07ab154
WIP node iterator implementation
Feb 6, 2024
48b0991
WIP node iterator implementation
Feb 6, 2024
2fb5963
WIP node iterator implementation
Feb 6, 2024
ad99a65
WIP node iterator implementation
Feb 6, 2024
7bab840
finish(?) key-value stream iterator; untested
Feb 6, 2024
093fd00
remove dead code
Feb 6, 2024
8f5e642
remove dead code
Feb 6, 2024
a5655cb
skip nibble from sentinel node
Feb 6, 2024
9d86ecf
16 of 20 UT passing
Feb 6, 2024
ec1d2fa
key-value tests passing
Feb 6, 2024
a97bc42
remove stale todo
Feb 6, 2024
deebfd9
comments
Feb 6, 2024
95258e8
make BranchIterator an enum with Unvisited/Visited states
Feb 6, 2024
05d80af
cleanup
Feb 6, 2024
fbd8b32
consolidate cases / duplicate code
Feb 6, 2024
ff22328
cleanup
Feb 6, 2024
f83ad7e
comment
Feb 6, 2024
9689f9e
nit rename Box<[u8]> to Key
Feb 7, 2024
1f6cade
comments
Feb 7, 2024
3dea540
cleanup
Feb 7, 2024
d519d3b
tweak dont copy iter
Feb 7, 2024
dd8f838
add todo
Feb 7, 2024
bdebf53
tweak dont copy iter
Feb 7, 2024
120e329
cleanup
Feb 7, 2024
0f30daf
remove unused helpers
Feb 7, 2024
99173a1
comments
Feb 7, 2024
599f5df
write helper function for is_prefix comparison
Feb 7, 2024
550bcdf
use is_prefix helper
Feb 7, 2024
d9c20bd
cleanup
Feb 7, 2024
4b49b52
add test
Feb 7, 2024
e0d740f
appease clippy
Feb 7, 2024
0694e19
nit renaming
Feb 7, 2024
1e2e1b4
remove unused errors
Feb 7, 2024
fbac933
appease docs
Feb 7, 2024
1fdd292
nit comments
Feb 7, 2024
b5ba4bf
put node inside Unvisited state instead of getting node twice
Feb 7, 2024
ace6f08
nit
Feb 7, 2024
bf7b983
add node_iter and node_iter_from methods on merkle
Feb 7, 2024
a5f2a92
prepend key-value iterator tests with key_value
Feb 7, 2024
aed1f16
make node iterator return raw key instead of nibbles
Feb 7, 2024
0d3d903
add tests
Feb 7, 2024
f5fe858
nit /// for docs
Feb 7, 2024
dfec30d
comment
Feb 7, 2024
78aaf99
remove errant assignment
Feb 7, 2024
d47c3df
rename is_prefix to compare_partial_path
Feb 7, 2024
1d64121
remove errant assignment
Feb 7, 2024
6eb9371
nit
Feb 7, 2024
1d3016a
remove unused node_addr
Feb 7, 2024
7f2c8b0
match --> if
Feb 7, 2024
3711a3a
remove TODO
Feb 7, 2024
c5568b6
Merge branch 'main' into node-iterator
Feb 7, 2024
a707350
appease clippy
Feb 7, 2024
2813236
Merge branch 'node-iterator' of github.com:ava-labs/firewood into nod…
Feb 7, 2024
7921dc9
rename key_value_iter_from to key_value_iter_from_key
Feb 7, 2024
347194d
nit rename MerkleNodeStreamState to NodeStreamState
Feb 7, 2024
3ade6ee
rename NodeStreamState enum states
Feb 7, 2024
58adeda
nit use Box::new
Feb 7, 2024
bd3c842
remove comment
Feb 7, 2024
d6c5851
panic! --> unreachable
Feb 7, 2024
347653b
nit rename var
Feb 7, 2024
9bb2622
nit rename var
Feb 7, 2024
649a189
add comment
Feb 7, 2024
6dd2f91
nit
Feb 7, 2024
64eaf67
box key differently
Feb 7, 2024
32395d6
simplify error handling on get_node by adding From<MerkleError> for a…
Feb 7, 2024
549e0e9
nit
Feb 7, 2024
6ff6ec1
nit
Feb 7, 2024
57f6e91
split out test into multiple
Feb 7, 2024
5584e4c
remove dead code; rename new methods
Feb 8, 2024
e3f2ffb
comment
Feb 8, 2024
b1935a0
nit
Feb 8, 2024
99992ea
Remove dead `IntoBytes` code (#520)
Feb 9, 2024
7a9e0c1
nits
Feb 9, 2024
3616545
rename get_children_iter --> as_enumerated_children_iter
Feb 9, 2024
0df9c73
reverse order of children iterator return values
Feb 9, 2024
8f931b3
nit
Feb 9, 2024
42ae4f5
Merge branch 'main' into node-iterator
rkuris Feb 9, 2024
95eedc1
Merge branch 'main' into node-iterator
rkuris Feb 9, 2024
811dfd6
Merge branch 'main' into node-iterator
Feb 12, 2024
2d873cb
Merge branch 'main' into node-iterator
richardpringle Feb 13, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions firewood/src/db.rs
Original file line number Diff line number Diff line change
Expand Up @@ -320,14 +320,15 @@ impl<S: ShaleStore<Node> + Send + Sync> api::DbView for DbRev<S> {

impl<S: ShaleStore<Node> + Send + Sync> DbRev<S> {
pub fn stream(&self) -> merkle::MerkleKeyValueStream<'_, S, Bincode> {
self.merkle.iter(self.header.kv_root)
self.merkle.key_value_iter(self.header.kv_root)
}

pub fn stream_from(
&self,
start_key: Box<[u8]>,
) -> merkle::MerkleKeyValueStream<'_, S, Bincode> {
self.merkle.iter_from(self.header.kv_root, start_key)
self.merkle
.key_value_iter_from_key(self.header.kv_root, start_key)
}

fn flush_dirty(&mut self) -> Option<()> {
Expand Down
10 changes: 6 additions & 4 deletions firewood/src/merkle.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1715,11 +1715,11 @@ impl<S: ShaleStore<Node> + Send + Sync, T> Merkle<S, T> {
self.store.flush_dirty()
}

pub(crate) fn iter(&self, root: DiskAddress) -> MerkleKeyValueStream<'_, S, T> {
pub(crate) fn key_value_iter(&self, root: DiskAddress) -> MerkleKeyValueStream<'_, S, T> {
MerkleKeyValueStream::new(self, root)
}

pub(crate) fn iter_from(
pub(crate) fn key_value_iter_from_key(
&self,
root: DiskAddress,
key: Box<[u8]>,
Expand Down Expand Up @@ -1750,8 +1750,10 @@ impl<S: ShaleStore<Node> + Send + Sync, T> Merkle<S, T> {

let mut stream = match first_key {
// TODO: fix the call-site to force the caller to do the allocation
Some(key) => self.iter_from(root, key.as_ref().to_vec().into_boxed_slice()),
None => self.iter(root),
Some(key) => {
self.key_value_iter_from_key(root, key.as_ref().to_vec().into_boxed_slice())
}
None => self.key_value_iter(root),
};

// fetch the first key from the stream
Expand Down
Loading
Loading