Serializable implementation

[BigBoyBarney]

Hi!

Thanks for the shard! I just came across KDL recently, and it looks fantastic! Unlike YAML, it also seems to support comments, which very cool and has been a big wish of mine.

Is there any chance we could get a Serializable implementation for KDL to read to and save from Crystal structs / classes? (I have 0 idea how feasible this is, but I figured I'd still ask :D)

Thanks in advance!

[danini-the-panini]

Hi @BigBoyBarney

I'd love to implement this. However, KDL is a bit more complex than JSON or YAML, and will require some work designing an adequate Serializable syntax. I'd love to hear any idea you have, perhaps show some examples as to how you would use it. I'm thinking of getting inspiration from knus which does a similar thing in Rust.

In the meantime, I've added a Builder class, like JSON, that allows you to construct KDL documents, along with comments on Nodes, arguments, and properties. I've also added comment preservation to the parser.

[BigBoyBarney]

First of all, thank you very much!! That is fantastic already.

Since creating this issue I've spent some more time with KDL and I've come to realise that it's more akin to XML than YAML or JSON, so full serializable compatibility might be difficult 🤔

I'll play around with the builder and report back!

Thanks a lot!!

[danini-the-panini]

I've started working on an implementation here: #6

Struggling a bit to get it to work. Any assistance would be much appreciated 🙏🏻

[BigBoyBarney]

I looked into it a bit more in the past days, and I think there are a few things to consider regarding the possible API.

Some of the things to note:

• Document can have a comment and nodes
• Nodes can have arguments, properties, children nodes, a type and a comment
• Arguments are Value types and can have comments and a type
• Properties are Hash(String => Value type) and can have comments and a type.

Currently, the builder can output comments for arguments and properties, but the parser cannot read them.
However, putting that aside and assuming that we can read comments everywhere, the following could potentially work:

• Define a class for everything with at least comment, type and value equivalent instance variables. Example:

  • A Node would have [name, comment, type, arguments, properties]
  • An Argument would have [value, comment, type]
  • A Property would have [key, value, comment, type]

• By default, deserialise all root nodes into instance variables with matching names, if the instance variable type is KDL::Node.

  • Annotations can be used to control the name matching and deseralisation type.

• Classes can be defined by the user inheriting from Node to give finer control over the instance variables.

  • By default, all arguments and properties are grouped into the arguments and properties instance variable arrays.
  • The user annotates an instance variable with the desired annotation, for example @[KDL::Field(key: "level")] to parse selected parts into it.
  • Example:

KDL document

Collected-Pokemon {
  Snorlax "The Sleepy" status=asleep pokemon-level=10
  Jigglypuff "The Cute" status=awake pokemon-level=1
}

trainers {
  Sylphrena
  DaniniThePanini
}

and the corresponding Crystal code

class Info
  include KDL::Serializable

  @[KDL::Field(key: "Collected-Pokemon")]
  property pokemon : CollectedPokemon
  property trainers : KDL::Node
end

class CollectedPokemon < KDL::Node
  include KDL::Serializable

  @[KDL::Field(key: "Snorlax")]
  property snorlax : Pokemon

  # It could also be possible to define a new  `children` instance variable to serialize as a custom type
  property list : Array(Pokemon)
end

class Pokemon < KDL::Node
  include KDL::Serializable

  @[KDL::Field(key: "pokemon-level")]
  property level : KDL::Property
  property status : KDL::Property
end

# Then, one could call
info = Info.from_kdl (parsed_kdl_doc_here)
info.pokemon.snorlax.status # => "asleep"
info.pokemon.children       # => Array(KDL::Node) 
info.pokemon.list           # => Array(Pokemon)

This would be relatively close to how JSON, DB and YAML::Serializable work.

However, there's another issue with regards to deserialisation, in addition to property and argument comments not being parsed. Namely, unconventional usage of node names.

Take the following excerpt of the Niri compositor config:

trackpoint {
  off
  natural-scroll
  accel-speed 0.2
  accel-profile "flat"
  scroll-method "on-button-down"
  scroll-button 273
  middle-emulation
}

This uses node names themselves as the actual information. https://kdl.dev/ mentions that for lists like this, it's convention to name the nodes -, so:

trackpoint {
  - off
  - natural-scroll
  - accel-speed 0.2
  - accel-profile "flat"
  - scroll-method "on-button-down"
  - scroll-button 273
  - middle-emulation
}

What do you think?

[danini-the-panini]

I've got an initial working solution now on #6

One thing I haven't done yet is using - for arrays. Not sure how to do that just yet, perhaps it should be the default for an un-annotated property, either we ignore it, or we store it as a child node, and depending on the type it is one of the following:

1. single argument (if primitive)
2. arguments (if array of primitives)
3. properties (if hash of primitives)
4. the actual value as a kdl node (if not primitive)
5. array of values as children (if array of non-primitives)
6. array of key-values as children (if hash of non-primitives)