Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update to new empty amt serialization #623

Merged
merged 3 commits into from
Aug 19, 2020
Merged

Update to new empty amt serialization #623

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
9b2d432
Update to new empty amt serialization and remove unnecessary allocations
austinabell Aug 13, 2020
5790ac5
update string deserializations to use Cow to handle escape characters
austinabell Aug 13, 2020
3428be1
Merge branch 'main' into austin/amtserfix
austinabell Aug 19, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion blockchain/chain_sync/src/sync.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1115,7 +1115,7 @@ mod tests {
compute_msg_meta(&blockstore, &usm, &sm)
.unwrap()
.to_string(),
"bafy2bzacecgw6dqj4bctnbnyqfujltkwu7xc7ttaaato4i5miroxr4bayhfea"
"bafy2bzacecmda75ovposbdateg7eyhwij65zklgyijgcjwynlklmqazpwlhba"
);
}
}
7 changes: 5 additions & 2 deletions crypto/src/vrf.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@ impl VRFProof {
pub mod json {
use super::*;
use serde::{de, Deserialize, Deserializer, Serialize, Serializer};
use std::borrow::Cow;

pub fn serialize<S>(m: &VRFProof, serializer: S) -> Result<S::Ok, S::Error>
where
Expand All @@ -49,7 +50,9 @@ pub mod json {
where
D: Deserializer<'de>,
{
let s: String = Deserialize::deserialize(deserializer)?;
Ok(VRFProof::new(base64::decode(s).map_err(de::Error::custom)?))
let s: Cow<'de, str> = Deserialize::deserialize(deserializer)?;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cow!!!!!

Ok(VRFProof::new(
base64::decode(s.as_ref()).map_err(de::Error::custom)?,
))
}
}
21 changes: 8 additions & 13 deletions ipld/amt/src/bitmap.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,12 +15,7 @@ impl ser::Serialize for BitMap {
where
S: ser::Serializer,
{
if self.b == 0 {
<[u8] as serde_bytes::Serialize>::serialize(&[], s)
} else {
let bitmap_bz = self.to_byte_array();
<[u8] as serde_bytes::Serialize>::serialize(&bitmap_bz, s)
}
<[u8] as serde_bytes::Serialize>::serialize(&[self.b], s)
}
}

Expand All @@ -29,17 +24,17 @@ impl<'de> de::Deserialize<'de> for BitMap {
where
D: de::Deserializer<'de>,
{
let bz: Vec<u8> = serde_bytes::Deserialize::deserialize(deserializer)?;
let bz: &[u8] = serde_bytes::Deserialize::deserialize(deserializer)?;

if bz.is_empty() {
return Ok(BitMap::default());
if bz.len() != 1 {
return Err(de::Error::custom(&format!(
"Expected 1 bitmap byte, was {}",
bz.len()
)));
}

// Get bitmap byte from serialized bytes
let bmap: BitMap = bz
.get(0)
.map(|b| BitMap::new(*b))
.ok_or_else(|| de::Error::custom("Expected bitmap byte"))?;
let bmap = BitMap::new(bz[0]);

Ok(bmap)
}
Expand Down
4 changes: 2 additions & 2 deletions ipld/tests/walk_tests.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -56,8 +56,8 @@ mod path_json {
where
D: Deserializer<'de>,
{
let s: String = Deserialize::deserialize(deserializer)?;
Ok(Path::from(s.as_str()))
let s: std::borrow::Cow<'de, str> = Deserialize::deserialize(deserializer)?;
Ok(Path::from(s.as_ref()))
}
}

Expand Down
6 changes: 3 additions & 3 deletions utils/bigint/src/bigint_ser.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,11 +37,11 @@ pub fn deserialize<'de, D>(deserializer: D) -> Result<BigInt, D::Error>
where
D: serde::Deserializer<'de>,
{
let mut bz: Vec<u8> = serde_bytes::Deserialize::deserialize(deserializer)?;
let bz: &[u8] = serde_bytes::Deserialize::deserialize(deserializer)?;
if bz.is_empty() {
return Ok(BigInt::default());
}
let sign_byte = bz.remove(0);
let sign_byte = bz[0];
let sign: Sign = match sign_byte {
1 => Sign::Minus,
0 => Sign::Plus,
Expand All @@ -51,5 +51,5 @@ where
));
}
};
Ok(BigInt::from_bytes_be(sign, &bz))
Ok(BigInt::from_bytes_be(sign, &bz[1..]))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What happens if there's only a sign byte? This will panic i think, but what is the expected behaviour? I would assume 0 or something, since empty bytes -> bigint deserializes to 0?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

on the 0 case, it is serialized as empty bytes, even if it was a sign byte and nothing else, this would just deserialize as zero. Also the functionality is basically the same as it was before.

}
6 changes: 3 additions & 3 deletions utils/bigint/src/biguint_ser.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,16 +35,16 @@ pub fn deserialize<'de, D>(deserializer: D) -> Result<BigUint, D::Error>
where
D: serde::Deserializer<'de>,
{
let mut bz: Vec<u8> = serde_bytes::Deserialize::deserialize(deserializer)?;
let bz: &[u8] = serde_bytes::Deserialize::deserialize(deserializer)?;
if bz.is_empty() {
return Ok(BigUint::default());
}

if bz.remove(0) != 0 {
if bz.get(0) != Some(&0) {
return Err(serde::de::Error::custom(
"First byte must be 0 to decode as BigUint",
));
}

Ok(BigUint::from_bytes_be(&bz))
Ok(BigUint::from_bytes_be(&bz[1..]))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same thing here

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

empty bytes get serialized as 0 and this would still not have an issue (but again Lotus and our client would not ever serialize as this case)

}
2 changes: 1 addition & 1 deletion utils/bitfield/src/rleplus/mod.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -85,7 +85,7 @@ impl<'de> Deserialize<'de> for BitField {
where
D: Deserializer<'de>,
{
let bytes: Vec<u8> = serde_bytes::deserialize(deserializer)?;
let bytes: &[u8] = serde_bytes::deserialize(deserializer)?;
Self::from_bytes(&bytes).map_err(serde::de::Error::custom)
}
}
Expand Down
2 changes: 1 addition & 1 deletion vm/address/src/lib.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -238,7 +238,7 @@ impl<'de> de::Deserialize<'de> for Address {
where
D: de::Deserializer<'de>,
{
let bz: Vec<u8> = serde_bytes::Deserialize::deserialize(deserializer)?;
let bz: &[u8] = serde_bytes::Deserialize::deserialize(deserializer)?;

// Create and return created address of unmarshalled bytes
Address::from_bytes(&bz).map_err(de::Error::custom)
Expand Down