-
Notifications
You must be signed in to change notification settings - Fork 33
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Introduce data access recording capabilities #162
Conversation
src/data.rs
Outdated
} | ||
|
||
// TODO: Consider if we are happy with the fact that duplicates are counted twice in both | ||
// `len(..)` and `iter(..)` methods. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If we are unhappy with double counting then we can introduce a counter for duplicates which we deduct when len
is called. To make the iter(..)
method only return unique values we could use itertools
or alternatively copy the Unique
implementation we require .
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we do want to handle this properly (especially for iteration) as it may affect how things work downstream. Depending on how much work it is, copying the implementation would be better (though, if it proves rather complex, adding itertools
as a dependency may be OK.
Another potential alternative is to change how this data structure keeps track of updates. Specifically, it could look something like this:
pub struct DataRecorder<K, V> {
data: BTreeMap<K, V>,
updates: BTreeSet<K>,
trace: RefCell<BTreeMap<K, V>>,
}
And something like get()
method could then work like so:
pub fn get(&self, key: &K) -> Option<&V> {
match self.state.get(key) {
None => None,
Some(value) => {
if !self.updates.contains(key) {
self.trace.borrow_mut().insert(key.clone(), value);
}
Some(value)
}
}
}
This does do more work - so, not sure if it is worth it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What does state represent in this instance?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
oh - sorry, state
should have been data
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think your proposal will work. It will always return the original value even if it has been updated. I implemented an alternate scheme, this involves the introduction of updated_keys
which keeps track of the keys from the original set which have been updated.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've implemented this approach in 80a7f01 as it simplified the implementation a bit. The tests seem to be passing (and I've added some more checks) - but take a look and see if I missed anything.
9680fc8
to
d8ec9d9
Compare
c75dd1c
to
591b95d
Compare
591b95d
to
679a30e
Compare
80a7f01
to
f08644e
Compare
I pushed a commit which simplifies things a bit. This includes:
|
Describe your changes
This PR introduces the required changes to support recording of data access. This includes the introduction of the following components and changes:
KvMap
: A trait that defines the interface for a key-value map.RecordingMap
: A [RecordingMap] that records read requests to the underlying key-value map. The data recorder is used to generate a proof for read requests.MerkleMapT
: A supertrait that defines the required traits for a type to be used as a data map backend for the [GenericMerkleStore]MerkleStore
->GenericMerkleStore
: MadeMerkleStore
generic over the key value map backend.