Actors uses different serializers for different purposes, not all of them are pluggable

[rynowak]

Expected Behavior

Actors uses a single serializer by default, and allows flexiblity for different choices for compatibility or user freedom.

The problem with not providing a consistent default is that different serializers have non-overlapping features sets, and different configuration settings. The choice of serializer usually ends up being tightly coupled with the users' data model through attributes and other extensibility.

For instance DataContract serializer requires types and properties to "opt-in" to serialization - System.Text.Json assumes that types and public properties should be serialized - and you should "opt-out" for anything you want to exclude. It takes non-obvious work to migrate between these models, and many users are not familiar with the nuances.

Actual Behavior

Actors uses DataContract serializer (XML) for "remoting" method invocation (the choice of DCS is not configurable).

Actors uses System.Text.Json (JSON) for "non-remoting" method invocation (neither the choice of S.T.J nor the options passed to it are configurable).

Actors uses System.Text.Json (JSON) for state storage (the serializer and options have a replaceable abstraction).

Sidenote: Actors currently uses BinaryFormatter for exceptions in "remoting" method invocation, but that is planned for removal.

Steps to Reproduce the Problem

Build an Actor system that uses "remoting". Try to share data models between the state store and the actor method invocation. You'll find that you need to annotate data models for both DataContract serializer and System.Text.Json.

Release Note

We need to surface a design proposal and assess the impact of a change here. This is not a good users experience, but no simple change will fix it.

[rynowak]

We're not going to make any changes here. This has some rough edges (using the same types with state and actor proxy invocation) but users that are using Actors today are overall happy with the experience. We'll revisit this in the future based on feedback.

[tomkerkhove]

We are actually having issues with this. Would be good to standardize on one serializer.

[techgeek03]

This is a also an issue for us. @rynowak I started conversation on discord in the dotnet-sdk channel. I want to make a proposal for the change. Can you please comment on that and let me know if you think I should create new proposal.

[onionhammer]

Having a ton of issues with this using records and STJ's polymorphism.

Can anyone explain what the OP means about 'non-remoting' actor invocations?

[virtualcca]

Is there an update to this issue?

Now we encounter that if some newer types (such as dateonly) are returned in the interface method of the actor, DCS(DataContract serializer)related errors will appear

Under Net7, STJ can support dateonly these types
Now you can only temporarily use DateOnly to be converted to DateTime as an Actor's interface to external calls

[jpiquot]

Is there a documentation on unsupported actor method parameters serialization features? How can we make unit tests for our actors that validates the method parameter serialization?

[onionhammer]

Using Data Contract serialization is utterly excruciating in 2023. I would love to be able to transmit JsonElements without hacky workarounds.

What advantage does DCS have?

Method	Mean	Error	StdDev	Gen0	Allocated
DataContractSerialization	1,102.4 ns	1.39 ns	1.30 ns	0.3719	3120 B
SystemTextJsonSerialization	297.7 ns	0.31 ns	0.28 ns	0.0658	552 B

[WolfspiritM]

We're also having this issue.

We expect a base class as an actor method parameter and want to store that in the actor state.

Currently on the base type we have to set "KnownTypes" for the client -> actor communication to get it serialized correctly and then we need to also need to add "JsonDerivedType" to get actor -> state communication working.

Is that really needed?

[robbss]

Any updates on this issue?

It is causing my team quite a bit of grief currently, making it very hard to use Dapr Actors for complex data types (Polymorphism).

[onionhammer]

@RobertConanMcMillan you can try using the experimental JSON serializer, unfortunately it's undocumented

on server:

on client:

[darraghjones]

Does the UseJsonSerialization property apply to actor state storage as well as actor invocation?

[robbss]

@onionhammer Thanks for the tip, unfortunately this doesn't seem to work correctly, do I need to provide some special JsonSerializerOptions?

All methods with parameters fail with this exception:

Dapr.Actors.ActorMethodInvocationException: Remote Actor Method Exception,  DETAILS: Exception: JsonException, Method Name: Read, Line Number: 0, Exception uuid: 6e524ebf-2f91-4af7-9816-a17b9b427cad
 ---> Dapr.Actors.ActorInvokeException: The JSON value could not be converted to WrappedMessageBody. Path: $ | LineNumber: 0 | BytePositionInLine: 16.
   --- End of inner exception stack trace ---
   at Dapr.Actors.DaprHttpInteractor.InvokeActorMethodWithRemotingAsync(ActorMessageSerializersManager serializersManager, IActorRequestMessage remotingRequestRequestMessage, CancellationToken cancellationToken)
   at Dapr.Actors.Communication.Client.ActorRemotingClient.InvokeAsync(IActorRequestMessage remotingRequestMessage, CancellationToken cancellationToken)
   at Dapr.Actors.Client.ActorProxy.InvokeMethodAsync(Int32 interfaceId, Int32 methodId, String methodName, IActorRequestMessageBody requestMsgBodyValue, CancellationToken cancellationToken)

@darraghjones It does, storing complex polymorphic models works perfectly with it enabled 👍

Edit: I was mistaken, it does not, by coincidence the model I was using could be serialized fine, but deserialization does not work.

[paule96]

It seems like the current experimental implementation is broken, because a recursion is happening that shouldn't happen:

Dapr.Actors.dll!Dapr.Actors.Communication.ActorMessageBodyJsonConverter<WrappedMessageBody>.Read(ref System.Text.Json.Utf8JsonReader reader, System.Type typeToConvert, System.Text.Json.JsonSerializerOptions options) Line 48
	at /_/src/Dapr.Actors/Communication/ActorMessageBodyJsonConverter.cs(48)
System.Text.Json.dll!System.Text.Json.Serialization.JsonConverter<WrappedMessageBody>.TryRead(ref System.Text.Json.Utf8JsonReader reader, System.Type typeToConvert, System.Text.Json.JsonSerializerOptions options, ref System.Text.Json.ReadStack state, out WrappedMessageBody value, out bool isPopulatedValue) Line 197
	at /_/src/libraries/System.Text.Json/src/System/Text/Json/Serialization/JsonConverterOfT.cs(197)
System.Text.Json.dll!System.Text.Json.Serialization.JsonConverter<WrappedMessageBody>.ReadCore(ref System.Text.Json.Utf8JsonReader reader, System.Text.Json.JsonSerializerOptions options, ref System.Text.Json.ReadStack state) Line 52
	at /_/src/libraries/System.Text.Json/src/System/Text/Json/Serialization/JsonConverterOfT.ReadCore.cs(52)
System.Text.Json.dll!System.Text.Json.JsonSerializer.Read<System.__Canon>(ref System.Text.Json.Utf8JsonReader reader, System.Text.Json.Serialization.Metadata.JsonTypeInfo<System.__Canon> jsonTypeInfo) Line 301
	at /_/src/libraries/System.Text.Json/src/System/Text/Json/Serialization/JsonSerializer.Read.Utf8JsonReader.cs(301)
Dapr.Actors.dll!Dapr.Actors.Communication.ActorMessageBodyJsonConverter<WrappedMessageBody>.Read(ref System.Text.Json.Utf8JsonReader reader, System.Type typeToConvert, System.Text.Json.JsonSerializerOptions options) Line 78
	at /_/src/Dapr.Actors/Communication/ActorMessageBodyJsonConverter.cs(78)

In the Stacktrace above you can see the ActorMessageBodyJsonConverter<WrappedMessageBody>.Read is called twice.
The problem for me is I don't understand why.
The JSON that was tried to be converted was:

{
  "value": {
    "loadData": {
      "someStringProperty1": " a nice string",
      "someStringProperty2": "a nice string",
      "someStringProperty3": "a nice string",
      "someArrayProperty": []
    }
  }
}

The value and loadData are probably from WrappedMessageBody.
The problem is the reading of the JSON currently happens as follows:

• Start reading json and detect StartObject
• System.Text.Json calls the ActorMessageBodyJsonConverter to convert that object
• ActorMessageBodyJsonConverter detects the property value
• ActorMessageBodyJsonConverter calls JsonSerializer.Deserialize<T>(ref reader, options) with the content of value
• JsonSerializer does some stuff until it reaches again the StartObject token (the value of value)
• JsonSerializer calls the ActorMessageBodyJsonConverter to convert that object
• ActorMessageBodyJsonConverter tires to detect the property value
• ActorMessageBodyJsonConverter fails because there is only the property loadData in that object

So I guess currently this experiment can't be used.

@onionhammer are you aware of this issue or do you have any tips?

P.S.:

The problematic line is:

if (reader.TokenType != JsonTokenType.PropertyName || reader.GetString() != "value") throw new JsonException();

[paule96]

@onionhammer, I debug this behavior with a colleague a bit more. And actually, the issue is more the constructor than the if statement.
In the constructor you check that the wrappedRequestMessageTypes only contains one type. But usually it contains for my project up to 3 generated types. (generated by the dapr actor framework)
In my case it looks like that:

As you can see there are 3 types defined.
The next line is:

if (this.wrappedRequestMessageTypes != null && this.wrappedRequestMessageTypes.Count == 1)
{
    this.wrapperMessageType = this.wrappedRequestMessageTypes[0];
}

This line checks if the count is exactly 1.
So wrapperMessageType is never assigned.

So the method Read will never start the actually deserialization of the type, because the following if statement will always be false:

if (this.wrapperMessageType != null)
{
   /// deserialization logic
}

[onionhammer]

Hi @paule96, if you're able to create a PR with a failing test (such as in E2ETests.CustomSerializerTests.cs) I will take a look

[paule96]

@onionhammer would a sample project also help?
I'm now pretty sure what does break the converter. As far as my local tests show, it's always broken, if the Interface that contains the method that is called contains more than 1 method.

I tried to adjust the tests, but the test seems to not work in a DevContainer. You can find the adjusted test setup here.

But as mentioned I don't know if the existing test will fail because I can't run them.