Deserializing dynamic types?

[rogeralsing]

In Proto.Actor, we have this bit of code for deserializing any proto message from our envelope type like so:

func deserialize(message *MessageEnvelope) proto.Message {
	t1 := proto.MessageType(message.TypeName) //get the go type from the proto type name
        t := t1.Elem() 
	intPtr := reflect.New(t) //create an instance of the type
	instance := intPtr.Interface().(proto.Message)  //get the message data from the envelope
	proto.Unmarshal(message.MessageData, instance) //fill the instance with the data 

	return instance
}

This works, but it feels somewhat hacky to have to resort to reflection here.

In C# using Protobuf, we only need to do:

public static object Deserialize(string typeName, ByteString bytes)
{
    var parser = TypeLookup[typeName];
    var o = parser.ParseFrom(bytes);
    return o;
}

I can however not see any such infrastructure in Go.
Is there a better way of solving the above or is that as good as it gets?

I do know about protobuf "Any" types, but AFAIK, not all languages have support for that yet.

[awalterschulze]

Yes this almost exactly what the Any type does.
Hmmm
If golang/protobuf rather registers a new function

proto.RegisterMessage(func () proto.Message {
   return &generatedType{}
})

Then you would simply be able to call the function.
Maybe you can take this issue up with them.
If they change it, then we can merge it into gogoprotobuf.

[bcmills]

I do know about protobuf "Any" types, but AFAIK, not all languages have support for that yet.

Any is a regular proto3 message type (defined in a .proto file): any language which supports proto3 at all should be compatible with it.

The only thing special about Any is that the proto runtime library for most languages has extra decoding support. (For Go, that's the ptypes package.)

[rogeralsing]

Ok, I finally had some luck using Any type.
It is however far slower than what we have now. I would assume its the size of the DynamicAny or something that cost allocation time.

Our benchmarks dropped from 1.85 mil msg/sec to 1.1 mil msg/sec (nonsense benchmarks but still)

That being said, I also tried the

proto.RegisterMessage(func () proto.Message {
   return &generatedType{}
})

Approach mentioned by @awalterschulze
This had a fairly decent impact in a good way.
Up from 1.85 mil msg/sec to 2.0 mil msg/sec
So 150k diff.

Right now I manually registered this for my types.
It would have been neat if that could be generated though

[awalterschulze]

This is very interesting. Could you also mention this to golang protobuf?

…

On Tue, 24 Jan 2017, 22:52 Roger Johansson, ***@***.***> wrote: Ok, I finally had some luck using Any type. It is however far slower than what we have now. I would assume its the size of the DynamicAny or something that cost allocation time. Our benchmarks dropped from 1.85 mil msg/sec to 1.1 mil msg/sec (nonsense benchmarks but still) That being said, I also tried the proto.RegisterMessage(func () proto.Message { return &generatedType{} }) Approach mentioned by @awalterschulze <https://github.com/awalterschulze> This had a fairly decent impact in a good way. Up from 1.85 mil msg/sec to 2.0 mil msg/sec So 150k diff. Right now I manually registered this for my types. It would have been neat if that could be generated though — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#255 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABvsLbk2w-6qqHKKgByUTCqUJxIjaJHnks5rVnKtgaJpZM4LrodY> .

[awalterschulze]

Thank you

…

On Wed, 25 Jan 2017, 07:27 Walter Schulze, ***@***.***> wrote: This is very interesting. Could you also mention this to golang protobuf? On Tue, 24 Jan 2017, 22:52 Roger Johansson, ***@***.***> wrote: Ok, I finally had some luck using Any type. It is however far slower than what we have now. I would assume its the size of the DynamicAny or something that cost allocation time. Our benchmarks dropped from 1.85 mil msg/sec to 1.1 mil msg/sec (nonsense benchmarks but still) That being said, I also tried the proto.RegisterMessage(func () proto.Message { return &generatedType{} }) Approach mentioned by @awalterschulze <https://github.com/awalterschulze> This had a fairly decent impact in a good way. Up from 1.85 mil msg/sec to 2.0 mil msg/sec So 150k diff. Right now I manually registered this for my types. It would have been neat if that could be generated though — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#255 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABvsLbk2w-6qqHKKgByUTCqUJxIjaJHnks5rVnKtgaJpZM4LrodY> .