Scalar Aliases?

[rmharris157]

I'm trying to wrap an existing legacy C API in Flatbuffers to decouple logic.

Maybe I missed this, but is there anyway that I apply an alias (or typedef) to a internally-defined scalar to keep the naming of the objects consistent with the specification?

i.e. My existing C header files have a number of typedef's which define a contextually-named type e.g. "slotId" to a native size construct "unsigned long". I can't find a way to do that in FB without wrapping it in a struct, but now I have this pesky internal field I need to deal with.

[aardappel]

FlatBuffers has no way to do C/C++ style typedefs itself. Wrapping in a struct is indeed the only way to do something similar, and is a "zero cost" abstraction, other than the slightly clumsier syntax.

@mikkelfj who maintains the C implementation.

[mikkelfj]

I have tried to read this question several times, and I simply do not understand it. Can I please have a concrete example with a C type, a flatbuffer schema, and exactly which names should match?

You can do a lot with define but that is manual work. If you are willing to put in some effort, you can generate a binary schema (bfbs) and read that information and use it to generate defines or inline functions or similar adaptors, but since I do not understand the fundamental problem I'm not sure if it will help.

Another point: I'd actually like to have typedef's in the schema language itself because you say that this 'string' is actually a date format, and you can associate attributes with that which can be used, for example when reading binary schema. But there is no such thing for the time being.

[rmharris157]

Let's say I have a legacy C header file with the following declaration:

typedef unsigned long BL_ULONG
typedef      BL_ULONG BL_SESSION_HANDLE;

It would be great if you could define an alias called blSessionHandle that reference ulong32. Or, as mentioned above, a typedef mechanism.

Currently, the only option is to declare blSessionHandle as a struct which containing a single ulong32 member; but that member is named and adds an unnecessary level of indirection when calling the accessor and mutator.

This could work with just declaring an alias to ulong32 or create a typedef-like construct.

[mikkelfj]

I still don't understand. If you have a table field

table Foo {
    field1 : uint32;
}

Would you then like a) the generated code to return a value of type blSessionHandle, or would you like the schema syntax to use the name b) blSessionHandle?

typedef blSessionHandle uint32;
table Foo {
    field1 : blSessionHandle;
}

?
In case a) I don't see any great benefits because flatcc already generates uint32_t types which should not cause any compiler issues with compatible types.

In case b) this does not relate to the actual "legacy" C code, because it is just within the schema, but it does help maintain context between the two systems.

case b) is actually close to what I would like to add, if not identical to, but there it is work, and it requires coordination. Let's call b) for schema typedef.

My thoughts on handling the schema typedef is something along always using structs internally because it simplifies many things including unions and alignment. But the C code generator can handle single value typedefs specially and automatically lift the indirection you mention. This is something I have pondered in the past and something I might do even if only supported for flatcc because it annoys me sufficiently not to have it.

The typedef from the earlier example will internally, but not user visibly be translated into

struct blSessionHandle (typedef_primitive) {
    value: uint32;
}
table Foo {
    field1 : blSessionHandle;
}

The code generator then picks up on the typedef attribute and unwraps it because it is a primitive.

The struct style can be written in todays schema format as is and the typedef attribute could be declared.

A user would also be able to use the struct form directly in schema. This has the benefit of allowing new attributes to be introduced:

attribute "date";
struct DateString (typedef_primitive) {
    value : char[20] (date: "ISO-8601");
}
table Foo {
    timestamp : DateString;
}

You might also want typedef_composite for typedef'ing a multi-valued struct into another struct be predefing attributes for that struct, or support strings, vectors unions, etc. and this is where it gets complicated.

Another much more pragmatic option is to use the C preprocessor or the sed tool to expand predefined types.

[mikkelfj]

Note that you can also use enums if it is strictly for scalar types.

enum blSessionHandle : uint32 { _unused_ };

table Foo {
    field1 : blSessionHandle;
}

[github-actions]

This issue is stale because it has been open 6 months with no activity. Please comment or this will be closed in 14 days.

[skinkie]

I would like to reopen this topic again. I am trying to convert an internal binary format heavily reliant on mmap towards flatbuffers. While the concepts used are very alike, not having the ability to have 'type abstractions' (without structs) is clumsy. Maybe it is zero cost as @aardappel suggested, if this struct-shadow-type has to be added to a vector, it just looks awefull. In relationship to for example the Java generated code is plain overhead.

The use case is the following. At serialisation stage I would like to define the 'size' of my fields so they are recognizable. For example, for a list of stops never exceeding uint16_t, I rather use spidx_t in my code, so that a reference to that stop list also uses spidx_t. Sure I can do like below...

struct spidx { value: ushort; }
table jps_points {
        points: [spidx];
}

...but the code for the description below is far more elegant.

table jps_points {
        points: [ushort];
}

I can see a situation where preprocessing a fbs file is an option, but so is aliasing.

[aardappel]

Simple typedefs wouldn't be hard if its a schema-only feature.

type spidx = ushort.

Then a simple extra lookup wherever it parses a type.

[skinkie]

Exactly that. Could we reopen this issue?

[mikkelfj]

I support that.
I'd like to allow it for other types as well, at least for structs.

struct Vec3 {
    float x;
    float y;
    float z;
}

type position3d Vec3;
type velocity3d Vec3;

I'd also like to associate attributes with the type definition.

For example, you might want to add attributes for validation or for JSON print format specific to the type, or specify metadata such as crypto cipher. Or just to clarify what the semantics of the type is.

attribute hex;
attribute base64;
attribute cipher;
attribute unit;
attribute timezone;
attribute epoch;

struct Int128  {
    uint8 [value:16];
}

type utc64 = uint64 (timezone: "zulu", unit: "microsecond", epoch: "unix1970");
type utc32 = uint32 (timezone: "zulu", unit: "millisecond", epoch: "unix1970");

type secret = Int128 (hex, cipher: "AES");
type hires_timer = Int128 (base64, unit: "picosecond", epoch: "relative");

table track {
    key: secret;
    event_time: hires_timer;
    location: position3d;
    heading: velocity3d;
    log_time: utc32;
}

(EDIT: update type syntax)

[mikkelfj]

Also, BTW: I think attributes should accept list form. And there is formatting error on the schema syntax for grammar:
attribute_decl = attribute ident | "</tt>ident<tt>" ;
https://google.github.io/flatbuffers/flatbuffers_grammar.html

[aardappel]

Yes, I think aliases can work for any type, even more complex ones, e.g.

type EntVec = [Ent]

not sure how useful that is, but no reason to limit it, I think.

We can certainly allow attributes, though we currently have no specific use for them:

type EntVec = [Ent] (attr: value)

PRs welcome :)

[mikkelfj]

It is not exactly an alias due to metadata, and type_decl is already used in the grammar.

schema' = schema | typedef_decl .
typedef_decl = "type" ident "=" type metadata ";" .

FlatBuffers spec isn't clear about attribute semantics for repeated values. I suggest treating them as ordered lists and let backends decide further interpretations. That means that (foo:1, foo:2, foe, foo:1, foe:"bar") becomes two lists that cannot be expressed directly in the grammar: ([foo:1, foo:2, foo:1], [foe, foe:"bar"]).

This becomes significant when types are based on types that already have metadata. In that case attributes are appended rather than replaced, but a backend to choose the first or last value if it so desires.

We could define a new default value for the type, for some base types, but this probably stretching it a bit too far.

[mikkelfj]

Corner case:

type x = [uint8:16];

This type can be used in structs, but not in table fields.
And vice versa

type y = [uint8];

cannot be used in a struct.

All of this doesn't quite solve the missing scalars in Unions because even if structs can be typedef'ed, the access mechanism won't change. I think this is orthogonal and at some point we should consider adding scalar types to unions.
It is some something that occasionally pops up as an issue, and the solution is never straight forward nor particularly clean.

[github-actions]

This issue is stale because it has been open 6 months with no activity. Please comment or this will be closed in 14 days.

[skinkie]

Please keep it open.

[github-actions]

This issue is stale because it has been open 6 months with no activity. Please comment or this will be closed in 14 days.

[skinkie]

Please keep open.

[zvookin]

I'm writing a schema where I have fields that are indices into an array elsewhere in the schema. I'd like to document that these fields are indices into a specific array. In C++, I'd use a typedef to do this. Nothing fancy is expected from the schema compiler, it is just a way to make the code more self explanatory.

Is there an already existing way to do this? If not, consider it another use case for type aliases.