-
Notifications
You must be signed in to change notification settings - Fork 220
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Saveload custom derive #460
Conversation
`IntoSerialize` and `FromDeserialize` relied on `Component` which was somewhat pointless. This wouldn't be a problem, but due to how proc macros work, this is necessary to create something like `\#[derive(IntoSerialize)]` because then we can ensure every field, transitively, is `IntoSerialize`. For the actual (De)`SerializeComponents` traits, this is changed into `IntoSerialize+Component` and `FromDeserialize+Component` as a bound to ensure correctness (it doesn't compile without it anyway). We can then add `#[derive(Saveload)]` which auto-implements `IntoSerialize`/`FromDeserialize` for most types (excluding those with references/pointers, and ones with generics for now) by automatically constructing a matching proxy "Data" type and converting all the fields with a recursive call to `into`/`from`.
953c5ed
to
8b9ea4f
Compare
There is no much sense in deriving only one trait anyway. We could merge them altogether. |
I split them earlier, I'm sure there was a reason for that. Deriving two traits at once doesn't seem odd to me. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
First review, mostly formatting. The missing generic implementation should be rather easy to add, I think we should do that.
I didn't review all the files yet.
specs-derive/Cargo.toml
Outdated
@@ -9,8 +9,12 @@ keywords = ["gamedev", "parallel", "specs", "ecs", "derive"] | |||
license = "MIT/Apache-2.0" | |||
|
|||
[dependencies] | |||
syn = "0.12" | |||
syn = {version="0.12", features=["extra-traits"]} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Formatting / spacing
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also, isn't there a newer version already? Might be wrong
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Tried to update this, but the previous derive code was using synom
which was removed from re-exports in newer one and trying to depend on it manually didn't work out of the box. Will need more work to do this.
specs-derive/Cargo.toml
Outdated
quote = "0.4" | ||
|
||
[dev-dependencies] | ||
specs = { path = "..", features=["serde"] } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Same as above
specs-derive/src/impl_saveload.rs
Outdated
@@ -0,0 +1,487 @@ | |||
//! Contains implementations for #[derive(Saveload)] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
missing backticks
specs-derive/src/impl_saveload.rs
Outdated
//! Contains implementations for #[derive(Saveload)] | ||
//! which derives both `IntoSerialize` and `FromDeserialize`. | ||
//! | ||
//! Since this requires a proxy "Data" `struct`/`enum`, these two need |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
spacing
specs-derive/src/impl_saveload.rs
Outdated
//! to be derived together or it begins requiring unwieldy attribute explosion. | ||
//! | ||
//! Currently there is one major limitation to this implementation: it cannot handle | ||
//! generic types such as `struct Foo<T> { x: T, e: Entity }`. I'm not good enough at |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This should be fixed, shouldn't be too hard.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just starting with the review, is this fixed now?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes. Forgot to remove push commit that removes that documentation
specs-derive/src/impl_saveload.rs
Outdated
let derive = match ast.data { | ||
Data::Struct(ref mut data) => saveload_struct(data, &mut ast.ident, &mut ast.generics), | ||
Data::Enum(ref data) => saveload_enum(data, &ast.ident, &ast.generics), | ||
Data::Union(_) => panic!("Unions cannot derive IntoSerialize/FromDeserialize at present"), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
won't in the future (missing tag); missing backticks + spacing
specs-derive/src/impl_saveload.rs
Outdated
} | ||
|
||
// Implements all elements of saveload common to structs of any type | ||
// (except unit structs like `struct Foo;` since they're auto-saveload); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't quite understand this sentence.
specs-derive/src/impl_saveload.rs
Outdated
)), | ||
clause => *clause = Some(parse_quote!( | ||
where MA: ::serde::Serialize + ::serde::de::DeserializeOwned + ::specs::saveload::Marker, | ||
for <'deser> MA: ::serde::Deserialize<'deser> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why is this necessary? Isn't this guaranteed by DeserializeOwned
?
specs-derive/src/impl_saveload.rs
Outdated
&saveload_fields, | ||
) | ||
} else { | ||
panic!("This derive does not support unit structs (Hint: they automatically derive FromDeserialize and IntoSerialize)"); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah now I get the above sentence; both places should describe it more like "every unit struct can derive Serialize
and Deserialize
, thus has a blanket impl fulfill their IntoSerialize
/ IntoDeserialize
".
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, I think I wasn't thinking when I wrote this. TBH, we could probably just silently do nothing because of this instead of panicking. I think I decided to be explicit for some reason.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good otherwise 👍
panic!("Slices are unsupported, use owned types like Vecs or Arrays instead") | ||
} | ||
Type::Reference(_) => panic!("References are unsupported"), | ||
Type::Ptr(_) => panic!("Raw pointer types are unsupported"), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why is that?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I didn't want to deal with reference/pointer graphs within serialized components (especially with cycles).
The biggest problem is it's not exactly clear what to even do with types that hold references while deserializing, since you'd need to somehow deserialize the contents of the reference and then pass that handle back to the callsite as well.
The funny thing is, we have a pointer that solves all these problems for us -- Entity, which is just a generational weak pointer. So I see no reason to try and solve these issues when we can tell the user "either write your own impl
to explain what you mean by "(de)serialize this pointer" or split this into multiple components and use an Entity
ID lookup".
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oh right. I don't know what I was thinking there.
specs-derive/tests/saveload.rs
Outdated
@@ -0,0 +1,70 @@ | |||
// These are mostly just compile tests | |||
#![allow(dead_code)] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please just check the result of the method calls, so we've got some additional coverage and don't need to disable warnings. Would be nice to have some more "exotic" types involving entities, like Vec<Entity>
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
See the overall thread because this is really important.
src/saveload/ser.rs
Outdated
use storage::{GenericReadStorage, ReadStorage, WriteStorage}; | ||
use world::{Component, EntitiesRes, Entity}; | ||
|
||
/// Converts a component into its serialized form. | ||
/// Converts a data type (usually a [`Component`]) into its serialized form. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not caused by this PR, but this isn't accurate. It gets converted into its serializable form.
src/saveload/ser.rs
Outdated
@@ -191,7 +219,7 @@ macro_rules! serialize_components { | |||
M: Marker, | |||
$( | |||
$sto: GenericReadStorage<Component = $comp>, | |||
$comp : IntoSerialize<M>, | |||
$comp : IntoSerialize<M>+Component, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
spacing
@WaDelma Are you going to be working on this? |
Yeah I'll work on this. |
@torkleyy says
I can tell you that Crap, I forgot code like
Not sure how to solve this from a design standpoint, seems like there are only really two options:
I feel like 1 is more painful but probably ultimately more flexible, it seems closer to how I'm still not sure how to fix it. |
Ah right I just read it. What I don't understand though: |
The reason is Rust can't prove the implementations won't conflict. |
I don't think that'd be such a big issue. Suggested solution
|
specs-derive/src/impl_saveload.rs
Outdated
/// ```nobuild | ||
/// fn into<F: FnMut(Entity) -> Option<MA>>(&self, mut ids: F) -> Result<Self::Data, Self::Error> { | ||
/// FooSaveloadData ( | ||
/// e: IntoSerialize::into(&self.0, &mut ids)? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Tuple struct, shouldn't have a name here.
I stumbled into problem while trying to make custom derive to work with generics: I need to assert that So we would need to either wait for that to stabilize, merge the traits or figure some other workaround for it. This is my current macro expansion |
Merging the traits seems adequate, to me. I don't see much near-term utility in having them split with how all the |
After merging the traits and getting code gen work I tried to implement error[E0119]: conflicting implementations of trait `saveload::ConvertSaveload<_>` for type `std::vec::Vec<world::entity::Entity>`:
--> src/saveload/mod.rs:187:1
|
143 | / impl<C, M> ConvertSaveload<M> for C
144 | | where
145 | | C: Clone + Serialize + DeserializeOwned,
146 | | {
... |
162 | | }
163 | | }
| |_- first implementation here
...
187 | / impl<M> ConvertSaveload<M> for Vec<Entity>
188 | | where
189 | | M: Serialize + DeserializeOwned,
190 | | {
... |
206 | | }
207 | | }
| |_^ conflicting implementation for `std::vec::Vec<world::entity::Entity>`
|
= note: upstream crates may add new impl of trait `serde::Serialize` for type `std::vec::Vec<world::entity::Entity>` in future versions
= note: upstream crates may add new impl of trait `serde::Deserialize<'de>` for type `std::vec::Vec<world::entity::Entity>` in future versions |
Looks like since in the future they're allowing upstream impls of |
I've checked and annoyingly, even specialization doesn't solve this. |
Can addition of |
Adding Real solution would be adding bound for: auto trait NoEntity {}
impl !NoEntity for Entity {} but auto traits are unstable so that's out of the picture |
I see. This trait is for deriving. |
Since this seems rather experimental, please move this into an optional crate / hide it behind a feature flag. If this only works for nightly, that would also be a way to get a first implementation. |
I was going to leave it out of this PR anyways. Is there still something missing here otherwise? |
src/saveload/mod.rs
Outdated
/// [`Clone`], [`Serialize`] and [`DeserializeOwned`]. | ||
/// | ||
/// Implementing this yourself is usually only needed if you | ||
/// have a component that points to another Entity, or has a field which does, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Entity -> [Entity
] (and subsequent link at the bottom)
src/saveload/mod.rs
Outdated
/// and you wish to [`Serialize`] it. | ||
/// | ||
/// *Note*: if you're using `specs_derive` | ||
/// and your struct does not have a generic bound (i.e. `struct Foo<T>`), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The generic bound thing was fixed, so we can remove this part.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewed 1 of 2 files at r2, 3 of 6 files at r5, 1 of 1 files at r6, 1 of 1 files at r7.
Reviewable status: 6 of 7 files reviewed, 8 unresolved discussions (waiting on @torkleyy, @Zaerei, and @WaDelma)
specs-derive/Cargo.toml, line 16 at r2 (raw file):
Previously, torkleyy (Thomas Schaller) wrote…
Same as above
Done
specs-derive/src/impl_saveload.rs, line 1 at r2 (raw file):
Previously, torkleyy (Thomas Schaller) wrote…
missing backticks
Done
specs-derive/src/impl_saveload.rs, line 4 at r2 (raw file):
Previously, torkleyy (Thomas Schaller) wrote…
spacing
Done.
specs-derive/src/impl_saveload.rs, line 56 at r2 (raw file):
Previously, torkleyy (Thomas Schaller) wrote…
won't in the future (missing tag); missing backticks + spacing
Done.
specs-derive/src/impl_saveload.rs, line 112 at r2 (raw file):
Previously, torkleyy (Thomas Schaller) wrote…
I don't quite understand this sentence.
Done.
specs-derive/src/impl_saveload.rs, line 129 at r2 (raw file):
Previously, torkleyy (Thomas Schaller) wrote…
Why is this necessary? Isn't this guaranteed by
DeserializeOwned
?
OK, now invalid.
specs-derive/src/impl_saveload.rs, line 479 at r2 (raw file):
Previously, torkleyy (Thomas Schaller) wrote…
Oh right. I don't know what I was thinking there.
OK, nothing left here.
specs-derive/src/impl_saveload.rs, line 117 at r8 (raw file):
.cloned() .map(|mut f| { replace_entity_type(&mut f.ty);
In the future, it would be nice if we had a compiletest that checks if a serde derive would do it, too.
specs-derive/tests/saveload.rs, line 9 at r8 (raw file):
#[derive(ConvertSaveload)] struct OneFieldNamed { e: ::specs::Entity,
Please import the prelude
.
specs-derive/tests/saveload.rs, line 68 at r8 (raw file):
// so no need to test anything but unit black_box::<U64Marker, _>(AnEnum::Unit);
whitespace
src/saveload/de.rs, line 156 at r8 (raw file):
/// ``` /// pub trait FromDeserialize<M>: Component {
That's what I'm a bit more skeptical about.. There was no split before the overhaul, then I added it for some reason (don't remember), now I'm not sure why I did that. And yeah, maybe we can find a better trait name.
src/saveload/mod.rs, line 58 at r7 (raw file):
Previously, Zaerei (Zoe Juozapaitis) wrote…
Entity -> [
Entity
] (and subsequent link at the bottom)
Did you check the links actually work? It would be good if all of them were absolute, since relative ones break for re-exports. But then, versions might differ..
src/saveload/mod.rs, line 62 at r7 (raw file):
Previously, Zaerei (Zoe Juozapaitis) wrote…
The generic bound thing was fixed, so we can remove this part.
Is this resolved?
src/saveload/ser.rs, line 12 at r2 (raw file):
Previously, torkleyy (Thomas Schaller) wrote…
Not caused by this PR, but this isn't accurate. It gets converted into its serializable form.
OK, no longer valid.
src/saveload/ser.rs, line 222 at r2 (raw file):
Previously, torkleyy (Thomas Schaller) wrote…
spacing
This is still an issue.
"Please import the prelude.": re relative paths: |
Absolute links also break if the user is using locally generated documentation |
Reviewable still not working for you? Please ping me when I should review again. |
@WaDelma I don't know what you mean by that rn. I'll review both PRs tomorrow if possible. |
Currently the custom derive does |
I think I speak for the Amethst team when I say that relative paths are
better for Amethyst. Please just make sure to document the needed imports.
…On Wed, Oct 3, 2018, 10:06 WaDelma ***@***.***> wrote:
Currently the custom derive does ::specs::Entity, but in that PR the
other custom derive was changed to do Entity so you have to import
everything the derive needs. In exchange you get ability to (easily) use
without direct specs dependency. (You can still make top level module named
specs I think)
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#460 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/AWFFuZYZSiz7knQDhVonxp8saEnaP2TIks5uhHAXgaJpZM4Wf9Ki>
.
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewed 1 of 2 files at r2, 1 of 1 files at r8, 1 of 1 files at r9, 2 of 4 files at r10, 3 of 3 files at r11.
Reviewable status: all files reviewed, 3 unresolved discussions (waiting on @Zaerei)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewable status: all files reviewed, 3 unresolved discussions (waiting on @torkleyy and @Zaerei)
specs-derive/src/impl_saveload.rs, line 241 at r3 (raw file):
Previously, Zaerei (Zoe Juozapaitis) wrote…
Tuple struct, shouldn't have a name here.
Not sure what this one is about.
src/saveload/mod.rs, line 58 at r7 (raw file):
Previously, torkleyy (Thomas Schaller) wrote…
Did you check the links actually work? It would be good if all of them were absolute, since relative ones break for re-exports. But then, versions might differ..
OK, let's ignore this.
src/saveload/mod.rs, line 62 at r7 (raw file):
Previously, torkleyy (Thomas Schaller) wrote…
Is this resolved?
LGTM
specs-derive/src/impl_saveload.rs, line 241 at r3 (raw file): Previously, torkleyy (Thomas Schaller) wrote…
It's just a minor documentation error. We have a tuple struct, i.e. |
bors r+ Ah yeah, this PR. |
460: Saveload custom derive r=WaDelma a=WaDelma Split from #434 A custom derive for IntoSerialize and FromDeserialize named #[derive(Saveload)]. It may seem odd to derive them together, but after spending significant time trying to do it the normal way (one derive per trait) it became clear that due to the requirement of "proxy" types that fill the IntoSerialize::Data and FromDeserialize::Data fields, it made sense to do it this way or you start getting into attribute hell. <!-- Reviewable:start --> --- This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/slide-rs/specs/460) <!-- Reviewable:end --> Co-authored-by: Zoe Juozapaitis <jragonmiris@gmail.com> Co-authored-by: Delma <delma@del.ma>
Build succeeded |
Split from #434
A custom derive for IntoSerialize and FromDeserialize named #[derive(Saveload)]. It may seem odd to derive them together, but after spending significant time trying to do it the normal way (one derive per trait) it became clear that due to the requirement of "proxy" types that fill the IntoSerialize::Data and FromDeserialize::Data fields, it made sense to do it this way or you start getting into attribute hell.
This change is