Serialization and protobuf net

Serialization

Serialization is the automatic process of transforming data structures or objects into a format that Fusee can store and reconstruct. For this Fusee uses Google's Protocol Buffers (protobuf). This is a data format for serialization which utilizes an interface description language that describes the structure of data. This description is generated automatically, through class annotations, from protobuf-net, with the help of annotations; for more information see engine developer section.

protobuf-net

Fusee uses protobuf-net in a modified version where all dependencies to System.ServiceModel and System.ServiceModel.Primitives are removed. The fork can be found here: https://github.com/FUSEEProjectTeam/protobuf-net-fusee. This is necessary as we use mono linker for our Webassembly builds which throws an exception when these references are being traversed.

All serializable classes can be found within Fusee.Serialization and Fusee.Math.

Conversion from FusFile to Fusee.Core.Scene and vice versa

All data within the V1 serialization needs to be converted from the "low-level" FusFile to the "high-level" Fusee.Core.Scene before being usable within Fusee.

To convert a serialized FusFile to a Fusee.Core.Scene.SceneContainer one can use the static method:

FusSceneConverter.ConvertFrom(FusFile fus)

For converting a "high-level" SceneContainer to a serializable FusFile the method:

FusFile FusSceneConverter.ConvertTo(SceneContainer sc)

exists.

Read files generated by the Fusee Blender exporter

Files generated with the help of Fusee's Blender exporter are composed of "low-level" classes which needs to be converted first. This is implemented within the AssetStorage.GetAsync logic. For example:

// open file
var fs = File.OpenRead("myFusFile.fus");
var lowLevelScene = ProtoBuf.Serializer.Deserialize<FusFile>(fs);
var scene = FusSceneConverter.ConvertFrom(lowLevelScene);

👷 Engine Developer

Adding a new serializable class to Fusee.Serialization

• Add class to Fusee.Serialization
• Let the class inherit from FusComponent
• Annotate the class with the [ProtoContract] keyword
• Annotate each class member which should be serializable with [ProtoMember(NUMBER)], make sure NUMBER is a unique number, start with 1, e. g.: [ProtoMember(1)].
• Go to FusComponent and let this class know of the new inheritance by adding [ProtoInclude(1xx, typeof(Name_of_your_class))]. Increase the counting accordingly.
• Add and implement both new visitors inside FusSceneConverter
• Add "high-level" class to Fusee.Core.Scene
• Write tests for your conversion inside Fusee.Test.Serialization.V?

Example

    /// Fusee.Serialization.V?.MyNewClass.cs
    [ProtoContract]
    public class MyNewClass : FusComponent
    {
        [ProtoMember(1)]
        public int Counter;
    }

    /// Inside FusComponent.cs
    [ProtoContract]
    [ProtoInclude(100, typeof(FusTransform))]
    /// [...]
    [ProtoInclude(197, typeof(MyNewClass))]

    public class FusComponent : IComponent
    {
       /// [...]
    }

    /// Inside FusSceneConverter.cs
    [VisitMethod]
    public void ConvMyNewClass(MyNewClass mnc)
    {
        /// conversion logic
    }

For inheritance and other advanced concepts study the existing classes and/or visit the protobuf-net wiki.

Conceptual changes from v0.8.0 onward

This paragraph describes the transition from the "old" (no version) FUSEE serialization scheme to the new one (V1).

The "old" serialization utilizes "container" classes and derivatives decorated for serialization with Protobuf .NET's [ProtoContract] etc. attributes while at the same time using these classes directly at run-time as scene graph building blocks.

Since Protobuf .NET version 3.* protobuf does not support references within serializable properties. Allowing components within multiple nodes requires this feature. Besides at several places, it was necessary to implement between scene graph building blocks used in serialization different from those used during traversal.

From V1 on, FUSEE's serialization is completely separated from scene graph building blocks in Fusee.Engine. As a result, Fusee.Xene was completely freed from all references and thus neither references Fusee.Serialization nor Fusee.Engine.

New/modified Files

Files added to Engine.Core.Scene

• SceneNode.cs
• SceneComponent.cs

"Old" Serialization	V1 Serialization	Engine.Core.Scene
SceneContainer	FusScene	SceneContainer
SceneNodeContainer	FusNode	SceneNode
SceneComponentContainer	FusComponent	SceneComponent
TransformComponent	FusTransform	Transform
Mesh	FusMesh	Mesh
LightComponent	FusLight	Light
AnimationComponent	FusAnimation	Animation
BoneComponent	FusBone	Bone
CanvasTransformComponent	FusCanvasTransform	CanvasTransform
RectTransformComponent	FusRectTransform	RectTransform
XFormComponent	FusXForm	XForm
XFormTextComponent	FusXFormText	XFormText
PtOctantComponent (File: OctantComponent.cs)	FusOctant (dummy implementation)	Octant
AnimationComponent	FusAnimation	Animation
AnimationTrackComponent	FusAnimationTrack	AnimationTrack
MaterialComponent	FusMaterialComponent	Material
MaterialPBRComponent	FusMaterialPBRComponent	MaterialPBR