Type safe proto clients and servers

[Mirko-von-Leipzig]

The proto crate provides the generated rpc client and server implementations, and conversions between miden-base and the generated protobuf types.

Should we take it a step further and wrap the various client and servers, to hide the protobuf types entirely? The wrappers would essentially just perform the conversions.

This has a couple of benefits

• smaller api surface in proto, particularly no generated code
• validation is performed automatically i.e. this is no longer required as it implicitly becomes part of the client/server api

and downsides

• quite a lot more (glue) code
• extra conversions can be unnecessary in the rpc case where its just being a proxy

[bobbinth]

Generally, I like this approach - but a few questions:

smaller api surface in proto, particularly no generated code

There would still be generated code, it would just not be exposed publicly - right?

extra conversions can be unnecessary in the rpc case where its just being a proxy

My understanding was that this is now happening anyway - but maybe if we go with the proposes approach we limit the ability to do something clever here in the future?

Also, one other question: would we still be able to have streaming API endpoints with this approach?

[Mirko-von-Leipzig]

smaller api surface in proto, particularly no generated code

There would still be generated code, it would just not be exposed publicly - right?

Correct

extra conversions can be unnecessary in the rpc case where its just being a proxy

My understanding was that this is now happening anyway - but maybe if we go with the proposes approach we limit the ability to do something clever here in the future?

Yes this is currently happening anyway - so long as we remember to add it each time. Its possible that we might find some optimization in the future which requires raw protobuf access for some reason. In this case we can expose the required parts again, though in the worst case it would regress to what we have now again. Though I struggle to think of an example of how this would happen.

Also, one other question: would we still be able to have streaming API endpoints with this approach?

If for some reason we struggle to wrap it properly, we can always fallback to exposing the streaming methods as is. Or with a trivial new-type wrapped around it.

I think the counter-argument to implementing this is simply: do we anticipate requiring raw access to most of the protobuf definitions for some reason? If yes, then this hiding business becomes unergonomic.

[bobbinth]

As we discussed earlier today, I think there are really 3 categories of endpoints here:

• RPC endpoints: these are not consumed internally in the node and so we don't need to worry about creating type-safe wrappers for them.
• Store endpoints used only by the RPC: ideally, responses of these endpoints would be just "forwarded" by the RPC to the user and so, minimizing serialization/deserialization work for this would be important which may be incompatible with making type-safe wrappers for them.
• Block producer endpoints an store endpoints used by the block producer: I think there is a clear case for these to have type-safe wrappers.