-
Notifications
You must be signed in to change notification settings - Fork 50
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Custom Polymorphic Serialization #30
Comments
Hey @Vampire, I suspect the issue is the use of
|
Hm, yeah, that works indeed. @Serializable
data class GitHubAction(
/* ... */
val outputs: Map<String, @Serializable(with = Output.Companion::class) Output>? = null,
/* ... */
) {
/* ... */
} does not work, so I'd say this is definitely a kaml bug. |
This is what I understand them to mean:
I believe kaml is behaving correctly, but if someone from the kotlinx.serialization team disagrees with that interpretation, I'd be happy to change it. |
I'm new to this, so my interpretation might be wrong, but as far as I understood it, Indications by the kotlinx.serialization team that kaml is misbehaving:
|
OK, it seems like there are two issues here.
Do you have a code snippet that shows this behaviour? kaml isn't involved in deciding which serializer to use (it just reports information to kotlinx.serialization which then picks the right one), so that sounds like a bug to me.
Interesting, based on looking at the JSON format code I'd expect it to be behaving in a similar way to kaml and throwing an exception. If you use your custom serializer with |
I showed you here: #30 (comment)
Yes, I just tried it, works fine. |
Digging into this further, I think this is actually one issue... kotlinx.serialization's JSON decoder does some special handling when polymorphic types are involved, and invokes a completely different code path depending on the serializer type - if the serializer is a This feels weird - I feel that the encoding shouldn't care what serializer is being used exactly, but I'll open an issue with kotlinx.serialization to see what their opinion is. |
Issue created: Kotlin/kotlinx.serialization#1009 |
Hello!! I found this issue and I don't know where else to ask about this but this seems like the most relevant place. I have a YAML data type that may be defined as two different types but depend on the value of another field in the type. For example:
I am attempting to create a custom So far I have this:object CustomSerializer : KSerializer<List<CustomDefinition>> {
@OptIn(InternalSerializationApi::class)
private val typeDescriptor: SerialDescriptor by lazy {
buildSerialDescriptor("type", PrimitiveKind.STRING)
}
@OptIn(InternalSerializationApi::class)
override val descriptor: SerialDescriptor by lazy {
buildSerialDescriptor("CustomDefinition", SerialKind.CONTEXTUAL) {
val oneDescriptor: SerialDescriptor by lazy {
buildClassSerialDescriptor("TypeOneDescriptor")
}
val twoDescriptor: SerialDescriptor by lazy {
buildClassSerialDescriptor("TypeTwoDescriptor")
}
element("oneDefinition", oneDescriptor)
element("twoDefinition", twoDescriptor)
}
}
override fun deserialize(decoder: Decoder): List<CustomDefinition> {
if (decoder !is YamlInput) {
throw UnsupportedOperationException("Can only deserialize from YAML source.")
}
val input = decoder.beginStructure(descriptor) as YamlInput
if(input.node is YamlScalar && input.node.path.endLocation.toString() == "customDefinitionList") {
// We want to read the "type" key and parse into the correct type.
decoder.beginStructure(descriptor) as YamlInput
// No idea on the index here?
when(Type.valueOf(decoder.decodeStringElement(typeDescriptor, 1))) {
Type.Something -> {
decoder.beginStructure(descriptor)
}
Type.OtherThing -> {
decoder.beginStructure(descriptor)
}
}
}
input.endStructure(descriptor)
}
} I honestly have no idea what I am doing :) |
@QuinnBast, if the https://github.com/Kotlin/kotlinx.serialization/blob/master/docs/polymorphism.md describes the polymorphism support available in kotlinx.serialization, and uses the built-in JSON format for its examples. For kaml, you can configure the YAML parser to use a property for polymorphism by setting |
@charleskorn I have the same problem. kaml does not use For JSON: {
"name": "",
"age": 1
} // Successfully printed ->>
// JsonData(name=[], age=1)
prinln(Json.decodeFromString<Data>(json)) For kaml: name: ""
age: 1 // Errors printed ->>
// Value for 'name' is invalid: Expected a list, but got a scalar value
prinln(Yaml.default.decodeFromString<Data>(json)) @Serializable
data class Data(
@Serializable(with = StringOrListSerializer::class)
val name: List<String>,
val age: Int
)
object StringOrListSerializer : KSerializer<List<String>> {
private val listSerializer: KSerializer<List<String>> = ListSerializer(String.serializer())
override val descriptor: SerialDescriptor = listSerialDescriptor<String>()
override fun serialize(encoder: Encoder, value: List<String>) = listSerializer.serialize(encoder, value)
override fun deserialize(decoder: Decoder): List<String> = try {
listOf(decoder.decodeString())
} catch (e: Throwable) {
decoder.decodeSerializableValue(listSerializer)
}
} |
@rinorz, I suspect the issue is that the Something like this should work: val descriptor: SerialDescriptor by lazy {
buildSerialDescriptor(serialName, SerialKind.CONTEXTUAL) {
element("list", listSerialDescriptor<String>())
element("string", PrimitiveSerialDescriptor("value", PrimitiveKind.STRING))
}
} If not, could you please create a sample project that reproduces the issue so I can investigate further? |
@charleskorn Thank you. It successfully called java.lang.IllegalStateException: Must call beginStructure() and use returned Decoder
at com.charleskorn.kaml.YamlContextualInput.decodeValue(YamlInput.kt:235)
at kotlinx.serialization.encoding.AbstractDecoder.decodeString(AbstractDecoder.kt:34) object StringOrListSerializer : KSerializer<List<String>> {
private val listSerializer: KSerializer<List<String>> = ListSerializer(String.serializer())
override val descriptor: SerialDescriptor = buildSerialDescriptor(
StringOrListSerializer::class.java.name,
SerialKind.CONTEXTUAL
) {
element("list", listSerialDescriptor<String>())
element<String>("string")
}
override fun serialize(encoder: Encoder, value: List<String>) = listSerializer.serialize(encoder, value)
override fun deserialize(d: Decoder): List<String> {
val decoder = d.beginStructure(descriptor) as Decoder
return try {
listOf(decoder.decodeString())
} catch (e: Throwable) {
decoder.decodeSerializableValue(listSerializer)
}
}
} |
It's very difficult to diagnose the problem without a full sample project and a complete stack trace - without these, I can't see where the problem is coming from. Could you please create a sample project that demonstrates the issue and share a link to it? |
@charleskorn In fact, the above code is a complete example, but for convenience, I created a minimal project to reproduce the problem: https://github.com/RinOrz/kaml-bug30/blob/master/src/test/kotlin/Tests.kt |
The issue is that you need to call For example, the following appears to work in the sample project you provided: @OptIn(InternalSerializationApi::class, ExperimentalSerializationApi::class)
object StringOrListSerializer : KSerializer<List<String>> {
private val listSerializer: KSerializer<List<String>> = ListSerializer(String.serializer())
+ private val listDescriptor = listSerializer.descriptor
+ private val stringDescriptor = String.serializer().descriptor
override val descriptor: SerialDescriptor = buildSerialDescriptor(
StringOrListSerializer::class.java.name,
SerialKind.CONTEXTUAL
) {
- element("list", listSerialDescriptor<String>())
+ element("list", listDescriptor)
element<String>("string")
}
override fun serialize(encoder: Encoder, value: List<String>) = listSerializer.serialize(encoder, value)
override fun deserialize(decoder: Decoder): List<String> = decoder.fromJson() ?: decoder.fromYaml()
private fun Decoder.fromJson(): List<String>? {
val decoder = this as? JsonDecoder ?: return null
return try {
listOf(decoder.decodeString())
} catch (e: Throwable) {
decoder.decodeSerializableValue(listSerializer)
}
}
private fun Decoder.fromYaml(): List<String> {
- val decoder = (this as YamlInput).beginStructure(descriptor) as Decoder
- return try {
- listOf(decoder.decodeString())
- } catch (e: IncorrectTypeException) {
- decoder.decodeSerializableValue(listSerializer)
+ val decoder = this.beginStructure(descriptor) as YamlInput
+
+ return when (decoder.node) {
+ is YamlScalar -> decodeString(decoder)
+ is YamlList -> decodeList(decoder)
+ else -> throw IllegalArgumentException("Value is neither a list nor a string.")
}
}
-}
+
+ private fun decodeString(input: YamlInput): List<String> {
+ val decoder = input.beginStructure(stringDescriptor) as YamlInput
+
+ return listOf(decoder.decodeString()).also { decoder.endStructure(stringDescriptor) }
+ }
+
+ private fun decodeList(input: YamlInput): List<String> {
+ return input.decodeSerializableValue(listSerializer)
+ }
+} |
Just a comment on this since I've seen it has some activity recently. Thank you for the information about the |
Didn't want to pollute the #29 feature request, so I'm opening a new question (or bug) issue. :-)
I tried to make a custom serializer as you suggested, following the official example https://github.com/Kotlin/kotlinx.serialization/blob/v0.20.0/runtime/commonTest/src/kotlinx/serialization/json/JsonTreeAndMapperTest.kt#L35 that is linked to from the docs about custom serializers.
In the GitHub action yml file you can have
or
Meaning the output has a mandatory
value
property if it is acomposite
action but novalue
property if it is a normal action.Previously I didn't support composite, so I had
I now changed it to
The problem is, that it seems to not being used.
When I had a
TODO()
in thedescriptor
it failed accordingly.But when I now try to deserialize an action file, I get complaint from kaml about missing type tag instead of the custom serializer being used.
Do I do something wrong or is there a bug in kaml?
The text was updated successfully, but these errors were encountered: