Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replace RPC request union with bytes #1119

Closed
wants to merge 1 commit into from

Conversation

wemeetagain
Copy link
Contributor

Using a union creates an unnecessary duplication with the method id (with
the union index of the serialized union as a stand-in for the request
method_id) and ultimately must be rechecked against the method_id to
ensure that the proper type has been/will be deserialized.

Instead, the union has been changed to a byte list. This allows for
simple programmatic switching on method_id to determine the deserialized
type as the sole determination, rather than having multiple coupled
switches, eg the method_id AND the union prefix. This also aligns the
Request type with the Response type, as both now have a "body" that is
a byte list and can be decoded with the method_id (and response_code).

Because request method_ids must be checked against body deserialized
forms to ensure the proper deserialized type, it makes sense to perform
the body deserialization only after we know which type is needed.
Otherwise we require extra validation after-the-fact to determine that
the decoded type is indeed what we want.

Using a union creates an unnecessary duplication of the method id (with
the union index of the serialized union as a stand-in for the request
method_id) and ultimately must be checked against the method_id to
ensure that the proper type has been deserialized.

Instead, the union has been changed to a byte list. This allows for
simple programmatic switching on method_id to determine the deserialized
type as the sole determination, rather than having multiple coupled
switches, eg the method_id AND the union prefix. This also aligns the
Request type with the Response type, as both now have a "body" that can
be decoded with the method_id (and response_code).
@dankrad
Copy link
Contributor

dankrad commented May 25, 2019

The duplication does not seem a very strong argument for changing the structured union type to just unstructured "bytes" with a potentially new serialisation format (we're talking about 4 extra bytes here). Alternatively, can the request type be uniquely inferred from the message body type?

@wemeetagain
Copy link
Contributor Author

Its less a matter of duplication, and more a matter of the ultimate primacy of the method id in the current scheme, and where the mapping of request types to request type bodies should be stored. Do we want it stored programmatically, where method ids are mapped to types? or do we want it stored in the union type?
I tend to think method ids are more friendly to experimentation, as they aren't necessarily tied to the order of an array that must have consensus. Eg: a client may add a nonstandard method id and it won't break if a new consensus method id is added. If the methods are implicit to the union type, any new addition of a consensus method id will break nonstandard methods (by changing the index of the nonstandard response body).

I don't think I'm proposing any new serialization format, we would still technically be using SSZ everywhere.

Interesting alternative, its similar in spirit. If the method id is removed, the request type can still be known by the decoded request body type. Or in another interpretation, the union prefix bytes become the new method id. See above why I think just using method ids is a better idea.
But if we go that direction, we might look at replacing the response result with a union as well, in the vein of unstructured bytes == bad, single-pass ssz == good.

@protolambda
Copy link
Collaborator

Not decoding the body immediately with standard SSZ can be a nice side-effect. E.g. you may not support the data types used in a specific method (i.e. you don't know how to decode), or you may want to proxy the method to some service that handles the specific method id.

@wemeetagain
Copy link
Contributor Author

E.g. you may not support the data types used in a specific method (i.e. you don't know how to decode)

Exactly. If we want to let the rpc stream be extensible by client implementations, to allow for client-specific methods with client-specific request bodies, we should not also be relying on the union.
Doing so makes it difficult for other clients to differentiate incompatible methods from invalid request bodies, because unknown request bodies will trigger ssz parse errors before the parsed method id can be checked against known methods.

@hwwhww
Copy link
Contributor

hwwhww commented Aug 9, 2019

Hey @wemeetagain, it seems outdated after #1328, can we close it?

@djrtwo
Copy link
Contributor

djrtwo commented Aug 12, 2019

Closing this. Please bring up in an issue for discussion if still relevant

@djrtwo djrtwo closed this Aug 12, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants