OpenApi Flags enum not handled correctly

[dnv-kimbell]

Is there an existing issue for this?

• I have searched the existing issues

Describe the bug

I have two operations that uses enums; one has [Flags] applied to it, the other does not. The one without flags is handled correctly, the one with the attribute is not.

[Flags]
public enum EnumWithFlags
{
    None = 0,
    Value2 = 1,
    Value3 = 2,
    Value4 = 4,
    Value5 = 8,
    Value6 = 16
}
public enum EnumWithNoFlags
{
    None = 0,
    Value2 = 1,
    Value3 = 2,
    Value4 = 4,
    Value5 = 8,
    Value6 = 16
}

"EnumWithFlags": {
	"type": "string"
},
"EnumWithNoFlags": {
	"enum": [
		"None",
		"Value2",
		"Value3",
		"Value4",
		"Value5",
		"Value6"
	]

Expected Behavior

Enums with flags should be handled the same as without flags.

[Flags] is a .NET specific thing and one could consider this an edge case since it doesn't have a counterpart in OpenApi. I work on a set of old apps that have been upgraded from asmx->WCF->WebApi. These use flag enums and we have .NET on both sides.

The STJ Enum converter has support for Flags
https://github.com/dotnet/runtime/blob/f96898084d7e4fadd8679f280daef979d60e10cf/src/libraries/System.Text.Json/src/System/Text/Json/Serialization/Converters/Value/EnumConverter.cs#L23

Steps To Reproduce

https://github.com/dnv-kimbell/openapi-inlineschema

Exceptions (if any)

No response

.NET Version

9.0 RC1

Anything else?

No response

[dnv-kimbell]

One workaround for this is to create your own schema transformer.
This might be good enough for most scenarios.

public class EnumFlagsTransformer : IOpenApiSchemaTransformer
{
    public Task TransformAsync(OpenApiSchema schema, OpenApiSchemaTransformerContext context, CancellationToken cancellationToken)
    {
        var type = context.JsonTypeInfo.Type;
        if (type.IsEnum)
        {
            var flags = type.GetCustomAttribute<FlagsAttribute>();
            if (flags is not null)
            {
                var values = Enum.GetValues(type);

                foreach (var e in values)
                {
                    schema.Enum.Add(new OpenApiString(e.ToString()));
                }
            }
        }

        return Task.CompletedTask;
    }
}

[captainsafia]

@dnv-kimbell Thanks for reporting this issue!

It looks like this is intentional behavior of the JsonSchemaExporter API's given the implementation defined here.

I suspect that this is because bitwise enums don't have a definition that maps cleanly into JSON schema since there's no way to explicitly label the combined values of a bitwise combination. The representation that you're modeling via the schema transformer above totally works, but for the purposes of .NET <-> .NET intro, the fact that FlagsWithEnums supports bitwise combination has been lost when reduced to this form.

For this particular case, I'm inclined to say that we wouldn't modify the default behavior in the implementation to support actually emitting the enum values given the constraints of the schema.

I'll poke around to see if there's any guidance at the schema level about the best way to model bitwise enums.

[dnv-kimbell]

System.Text..Json serializes bitwise enums as comma separated list of values. If the schema listed the possible values as it does with normal enums, wouldn't the use of bitwise become an implementation detail between client an server? Some consuming languages could potentially have limited support for bitwise values; only use it when you know the details of each end.