Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use custom derive ser/de instead of std::io #533

Merged
merged 75 commits into from
Sep 3, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
75 commits
Select commit Hold shift + click to select a range
4873de8
Use custom derive ser/de instead of std::io
Dentosal Jul 27, 2023
ed47458
Make fuel-tx no_std compatible
Dentosal Aug 14, 2023
1c9053a
Merge branch 'master' into dento/nostd-serialization
Dentosal Aug 14, 2023
dbb8b14
Add changelog entry
Dentosal Aug 14, 2023
0ea5f91
Add a FIXME on unsound code
Dentosal Aug 14, 2023
0c79eaf
WIP: remove std::io::{Read, Write} impls
Dentosal Aug 14, 2023
0a458e0
WIP: remove even more store_* and restore_* -style serde
Dentosal Aug 14, 2023
fd8bdfb
WIP: migrate to new ser/de
Dentosal Aug 14, 2023
f6be726
WIP: Finalize encode/decode changes
Dentosal Aug 14, 2023
bed4264
Add canonical(skip) attribute to new ser/de
Dentosal Aug 15, 2023
4c16ccd
Merge branch 'master' into dento/nostd-serialization
Dentosal Aug 15, 2023
55cbbed
WIP: Improve canonical(skip) handling
Dentosal Aug 15, 2023
70ad9e7
Add custom discriminant and deserialization overrides
Dentosal Aug 16, 2023
7be3223
Add custom serialization, and lots of debug prints
Dentosal Aug 17, 2023
80fc61f
Clean up a test case
Dentosal Aug 17, 2023
9248189
Fix primitive type alignment
Dentosal Aug 17, 2023
55bbf02
Fix alignment of types, add inner_discriminant support
Dentosal Aug 17, 2023
0594342
Fix tests (partially by removing canonical tests, must still readd th…
Dentosal Aug 18, 2023
e50f221
WIP: serialized_size rework
Dentosal Aug 19, 2023
b1330a8
Fix an alignment bug; all tests pass
Dentosal Aug 19, 2023
0a30af6
Remove even more mem_layout!
Dentosal Aug 19, 2023
5a8ba11
Separate SerializedSize impl
Dentosal Aug 19, 2023
5c74b5e
Use autogenerated receipt enum variant size calculation
Dentosal Aug 19, 2023
1736f87
Remove store_* and restore_*
Dentosal Aug 19, 2023
cb61f69
Remove more of the old bytes module
Dentosal Aug 19, 2023
553c8bf
Simplify specialization hack for bytes
Dentosal Aug 19, 2023
61152de
Fix enum size padding/alignment
Dentosal Aug 19, 2023
9007295
Merge branch 'master' into dento/nostd-serialization
Dentosal Aug 19, 2023
069c120
sort deps
Dentosal Aug 19, 2023
ecc4624
Remove debug prints
Dentosal Aug 19, 2023
f5e95e0
More docs
Dentosal Aug 19, 2023
1f475c1
clippy
Dentosal Aug 19, 2023
41c814d
Limit memory allocation during decode
Dentosal Aug 19, 2023
0c4c2d8
Remove obsolete todos
Dentosal Aug 19, 2023
05bf3bd
Handle some imports correctly on no_std
Dentosal Aug 19, 2023
39339b2
Remove SizedBytes in favor of Serialize::size
Dentosal Aug 19, 2023
274c4bf
Relocate some tests
Dentosal Aug 19, 2023
864013c
Add some missing docs
Dentosal Aug 19, 2023
4d45aed
Update fuel-derive/Cargo.toml
Dentosal Aug 22, 2023
0eaad47
Update fuel-derive/Cargo.toml
Dentosal Aug 22, 2023
1c296db
Clean up imports and mutual dependencies
Dentosal Aug 22, 2023
41a6b6a
Make fuel-derive a workspace-level dependency
Dentosal Aug 22, 2023
34e6a04
Do not hude behind feature flag; fix an array compacting bug
Dentosal Aug 22, 2023
1d444e8
Require even less alloc
Dentosal Aug 22, 2023
df32bbb
Depend even less on alloc
Dentosal Aug 22, 2023
f329eac
Add more imports for fuel_types::canonical::*;
Dentosal Aug 22, 2023
9f35bf2
WIP: before SerializedSizeVariable => SerializedSize
Dentosal Aug 29, 2023
d55cd96
WIP: change size traits, all tests passing again
Dentosal Aug 29, 2023
c862d3d
Finalize size calculcation rework
Dentosal Aug 29, 2023
912a278
Move changelog entry to breaking
Dentosal Aug 29, 2023
a24f9a7
Remove rendundant alloc feature from fuel-asm
Dentosal Aug 29, 2023
8496cf3
Add whitespace
Dentosal Aug 29, 2023
5bbdb45
Feature cleanup
Dentosal Aug 29, 2023
ddf1630
Update CI to latest Rust version, deps with MSRV = current - 2 break …
Dentosal Aug 29, 2023
004b959
Remove debug_peek code
Dentosal Aug 29, 2023
635dc63
Change fuel_types::path to ::fuel_types::path
Dentosal Aug 29, 2023
f14c3ef
Fix clippy warnings on the latest stable
Dentosal Aug 29, 2023
cf1870d
Check for duplicate and unknown canonical attributes on top-level
Dentosal Aug 29, 2023
50259cb
Remove unchecked type size arithmetic from generated code
Dentosal Aug 29, 2023
2ebc70a
Various fixes on no_std targets
Dentosal Aug 29, 2023
31f7019
Clippy lint fix
Dentosal Aug 29, 2023
04561de
Cleanup
Dentosal Aug 29, 2023
f606618
fmt
Dentosal Aug 29, 2023
e048864
Clean up some #[canonical] attribute handling
Dentosal Aug 29, 2023
f96ef89
Fix typo
Dentosal Aug 29, 2023
4af7c63
Validate struct #[canonical(prefix)] when decoding
Dentosal Aug 29, 2023
d1e50bc
Merge branch 'master' into dento/nostd-serialization
Dentosal Aug 29, 2023
5e5a8f0
Merge branch 'master' into dento/nostd-serialization
Dentosal Aug 30, 2023
d58df38
Purge mem_layout! and related types
Dentosal Sep 1, 2023
1e14e95
Remove #[canonical(discriminant = ...)] attribute
Dentosal Sep 1, 2023
bb79eb0
Merge branch 'dento/nostd-serialization' of https://github.com/FuelLa…
Dentosal Sep 1, 2023
d431ca5
Remove old testing files
Dentosal Sep 1, 2023
b16268c
Add readme to fuel-derive
Dentosal Sep 1, 2023
79d5717
Change some empty field initializations to ..Default::default()
Dentosal Sep 1, 2023
6f9be61
Improve documentation of the canonical module
Dentosal Sep 1, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,15 +6,15 @@ and this project adheres to [Semantic Versioning](http://semver.org/).

## [Unreleased]

### Changed

#### Breaking

- [#554](https://github.com/FuelLabs/fuel-vm/pull/554): Removed `debug` feature from the `fuel-vm`. The debugger is always available and becomes active after calling any `set_*` method.
- [#537](https://github.com/FuelLabs/fuel-vm/pull/537): Use dependent cost for `k256`, `s256`, `mcpi`, `scwq`, `swwq` opcodes.
These opcodes charged inadequately low costs in comparison to the amount of work.
This change should make all transactions that used these opcodes much more expensive than before.

- [#533](https://github.com/FuelLabs/fuel-vm/pull/533): Use custom serialization for fuel-types to allow no_std compilation.

## [Version 0.36.1]

### Changed
Expand Down
1 change: 1 addition & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ version = "0.36.1"
[workspace.dependencies]
fuel-asm = { version = "0.36.1", path = "fuel-asm", default-features = false }
fuel-crypto = { version = "0.36.1", path = "fuel-crypto", default-features = false }
fuel-derive = { version = "0.36.1", path = "fuel-derive", default-features = false }
fuel-merkle = { version = "0.36.1", path = "fuel-merkle", default-features = false }
fuel-storage = { version = "0.36.1", path = "fuel-storage", default-features = false }
fuel-tx = { version = "0.36.1", path = "fuel-tx", default-features = false }
Expand Down
3 changes: 2 additions & 1 deletion fuel-asm/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ description = "Atomic types of the FuelVM."
[dependencies]
arbitrary = { version = "1.1", features = ["derive"], optional = true }
bitflags = "1.3"
fuel-types = { workspace = true }
serde = { version = "1.0", default-features = false, features = ["derive"], optional = true }
strum = { version = "0.24", default-features = false, features = ["derive"] }
wasm-bindgen = { version = "0.2.87", optional = true }
Expand All @@ -26,7 +27,7 @@ rstest = "0.16"
[features]
default = ["std"]
typescript = ["wasm-bindgen", "wee_alloc"]
std = ["serde?/default"]
std = ["serde?/default", "fuel-types/std"]
serde = ["dep:serde"]

# docs.rs-specific configuration
Expand Down
1 change: 1 addition & 0 deletions fuel-asm/src/panic_instruction.rs
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ use crate::{
#[derive(Clone, Copy, PartialEq, Eq, Hash)]
#[cfg_attr(feature = "typescript", wasm_bindgen::prelude::wasm_bindgen)]
#[cfg_attr(feature = "serde", derive(serde::Serialize, serde::Deserialize))]
#[derive(fuel_types::canonical::Deserialize, fuel_types::canonical::Serialize)]
#[cfg_attr(feature = "arbitrary", derive(arbitrary::Arbitrary))]
/// Describe a panic reason with the instruction that generated it
pub struct PanicInstruction {
Expand Down
1 change: 1 addition & 0 deletions fuel-asm/src/panic_reason.rs
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ enum_from! {
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, strum::EnumIter)]
#[cfg_attr(feature = "typescript", wasm_bindgen::prelude::wasm_bindgen)]
#[cfg_attr(feature = "serde", derive(serde::Serialize, serde::Deserialize))]
#[derive(fuel_types::canonical::Serialize, fuel_types::canonical::Deserialize)]
#[cfg_attr(feature = "arbitrary", derive(arbitrary::Arbitrary))]
#[repr(u8)]
#[non_exhaustive]
Expand Down
2 changes: 1 addition & 1 deletion fuel-crypto/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ sha2 = "0.10"

[features]
default = ["fuel-types/default", "std"]
alloc = ["rand?/alloc", "secp256k1/alloc"]
alloc = ["rand?/alloc", "secp256k1/alloc", "fuel-types/alloc"]
random = ["fuel-types/random", "rand"]
serde = ["dep:serde", "fuel-types/serde"]
# `rand-std` is used to further protect the blinders from side-channel attacks and won't compromise
Expand Down
20 changes: 20 additions & 0 deletions fuel-derive/Cargo.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
[package]
Dentosal marked this conversation as resolved.
Show resolved Hide resolved
name = "fuel-derive"
version = { workspace = true }
authors = { workspace = true }
categories = { workspace = true }
edition = { workspace = true }
homepage = { workspace = true }
keywords = ["blockchain", "cryptocurrencies", "fuel-vm", "vm"]
license = { workspace = true }
repository = { workspace = true }
description = "FuelVM (de)serialization derive macros for `fuel-vm` data structures."

[lib]
proc-macro = true

[dependencies]
quote = "1"
syn = { version = "2", features = ["full"] }
proc-macro2 = "1"
synstructure = "0.13"
8 changes: 8 additions & 0 deletions fuel-derive/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
# Fuel VM custom serialization derive macros

[![build](https://github.com/FuelLabs/fuel-vm/actions/workflows/ci.yml/badge.svg)](https://github.com/FuelLabs/fuel-vm/actions/workflows/ci.yml)
[![crates.io](https://img.shields.io/crates/v/fuel-derive?label=latest)](https://crates.io/crates/fuel-derive)
[![docs](https://docs.rs/fuel-derive/badge.svg)](https://docs.rs/fuel-derive/)
[![discord](https://img.shields.io/badge/chat%20on-discord-orange?&logo=discord&logoColor=ffffff&color=7389D8&labelColor=6A7EC2)](https://discord.gg/xfpK4Pe)

This crate contains derive macros for canonical serialization and deserialization. This is used with [`fuel-types/src/canonical.rs`](fuel-types/src/canonical.rs) module which contains the associated traits and their implementations for native Rust types.
172 changes: 172 additions & 0 deletions fuel-derive/src/attribute.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,172 @@
// TODO: nice error messages instead of panics, see: https://stackoverflow.com/a/54394014/2867076

use std::collections::HashMap;

use proc_macro2::{
TokenStream,
TokenTree,
};
use syn::{
AttrStyle,
Attribute,
Meta,
};
use synstructure::BindingInfo;

fn parse_attrs(s: &synstructure::Structure) -> HashMap<String, TokenStream> {
let mut attrs = HashMap::new();

for attr in &s.ast().attrs {
if attr.style != AttrStyle::Outer {
continue
}
if let Meta::List(ml) = &attr.meta {
if ml.path.segments.len() == 1 && ml.path.segments[0].ident == "canonical" {
bvrooman marked this conversation as resolved.
Show resolved Hide resolved
let mut tt = ml.tokens.clone().into_iter().peekable();
if let Some(key) = tt.next() {
let key = key.to_string();
if let Some(eq_sign) = tt.peek() {
if eq_sign.to_string() == "=" {
let _ = tt.next();
}
} else {
// Single token, no `=`, so it's a boolean flag.
let old = attrs.insert(key.clone(), TokenStream::new());
if old.is_some() {
panic!("duplicate canonical attribute: {}", key);
}
continue
}

// Key-value pair
let value = TokenStream::from_iter(tt);
let old = attrs.insert(key.clone(), value);
if old.is_some() {
panic!("duplicate canonical attribute: {}", key);
}
continue
}
panic!("enum-level canonical attribute must be a `key = value` pair");
}
}
}

attrs
}

/// Pop-level `canonical` attributes for a struct
pub struct StructAttrs {
/// The struct is prefixed with the given word.
/// Useful with`#[canonical(inner_discriminant)]`.
/// The type must implement `Into<u64>`, which is used to serialize it.
pub prefix: Option<TokenStream>,
}

impl StructAttrs {
pub fn parse(s: &synstructure::Structure) -> Self {
let mut attrs = parse_attrs(s);

let prefix = attrs.remove("prefix");
bvrooman marked this conversation as resolved.
Show resolved Hide resolved

if !attrs.is_empty() {
panic!("unknown canonical attributes: {:?}", attrs.keys())
}

Self { prefix }
}
}

/// Pop-level `canonical` attributes for an enum
#[allow(non_snake_case)]
pub struct EnumAttrs {
/// This is a wrapper enum where every variant can be recognized from it's first
/// word. This means that the enum itself doesn't have to serialize the
/// discriminant, but the field itself does so. This can be done using
/// `#[canonical(prefix = ...)]` attribute. `TryFromPrimitive` traits are used to
/// convert the raw bytes into the given type.
pub inner_discriminant: Option<TokenStream>,
/// Replaces calculation of the serialized static size with a custom function.
pub serialized_size_static_with: Option<TokenStream>,
/// Replaces calculation of the serialized dynamic size with a custom function.
pub serialized_size_dynamic_with: Option<TokenStream>,
/// Replaces serialization with a custom function.
pub serialize_with: Option<TokenStream>,
/// Replaces deserialization with a custom function.
pub deserialize_with: Option<TokenStream>,
/// Determines whether the enum has a dynamic size when `serialize_with` is used.
pub SIZE_NO_DYNAMIC: Option<TokenStream>,
}

impl EnumAttrs {
#[allow(non_snake_case)]
pub fn parse(s: &synstructure::Structure) -> Self {
let mut attrs = parse_attrs(s);

let inner_discriminant = attrs.remove("inner_discriminant");
let serialized_size_static_with = attrs.remove("serialized_size_static_with");
let serialized_size_dynamic_with = attrs.remove("serialized_size_dynamic_with");
let serialize_with = attrs.remove("serialize_with");
let deserialize_with = attrs.remove("deserialize_with");
let SIZE_NO_DYNAMIC = attrs.remove("SIZE_NO_DYNAMIC");

if !attrs.is_empty() {
panic!("unknown canonical attributes: {:?}", attrs.keys())
}

Self {
inner_discriminant,
serialized_size_static_with,
serialized_size_dynamic_with,
serialize_with,
deserialize_with,
SIZE_NO_DYNAMIC,
}
}
}

/// Parse `#[repr(int)]` attribute for an enum.
pub fn parse_enum_repr(attrs: &[Attribute]) -> Option<String> {
for attr in attrs {
if attr.style != AttrStyle::Outer {
continue
}
if let Meta::List(ml) = &attr.meta {
if ml.path.segments.len() == 1 && ml.path.segments[0].ident == "repr" {
if let Some(TokenTree::Ident(ident)) =
ml.tokens.clone().into_iter().next()
{
return Some(ident.to_string())
}
}
}
}
None
}

/// Parse `#[canonical(skip)]` attribute for a binding field.
pub fn should_skip_field_binding(binding: &BindingInfo<'_>) -> bool {
should_skip_field(&binding.ast().attrs)
}

/// Parse `#[canonical(skip)]` attribute for a struct field.
pub fn should_skip_field(attrs: &[Attribute]) -> bool {
for attr in attrs {
if attr.style != AttrStyle::Outer {
continue
}
if let Meta::List(ml) = &attr.meta {
if ml.path.segments.len() == 1 && ml.path.segments[0].ident == "canonical" {
for token in ml.tokens.clone() {
if let TokenTree::Ident(ident) = &token {
if ident == "skip" {
return true
} else {
panic!("unknown canonical attribute: {}", ident)
}
}
}
}
}
}
false
}
Loading
Loading