OpenAPI incorrect oneOf definition when used with required

[b4nst]

What version of CUE are you using (cue version)?

$ cue version
cue version (devel)

go version go1.20.7
      -buildmode exe
       -compiler gc
     CGO_ENABLED 1
          GOARCH arm64
            GOOS darwin

Does this issue reproduce with the latest stable release?

Yes

What did you do?

exec cue import spec.yml -o -
 
-- cue.mod/module.cue --
module: "loftorbital.com/crdcue"
-- spec.cue --
// OneOf test

info: {
	title:   *"OneOf test" | string
	version: *"1.0.0" | string
}
// Oneof test
#main: ({
	opt1: _
	...
} | {
	opt2: _
	...
}) & {
	// option 1
	opt1?: string

	// option 2
	opt2?: string
	...
}

What did you expect to see?

// OneOf test
info: {
    title:   *"OneOf test" | string
    version: *"1.0.0" | string
}
// Oneof test
#main: {
    { opt1: string } | { opt2: string}
}

or something akin.

What did you see instead?

// OneOf test

info: {
	title:   *"OneOf test" | string
	version: *"1.0.0" | string
}
// Oneof test
#main: ({
	opt1: _
	...
} | {
	opt2: _
	...
}) & {
	// option 1
	opt1?: string

	// option 2
	opt2?: string
	...
}

which fails to vet correct values like

obj: #main & {
    opt1: "foo"
}

(incomplete)

[myitcv]

This is one of the situations where the current "encoding" of one-ofs fails/is just plain wrong. Whilst #943 will address this with a builtin, we could consider switching our notion of one-of "encoding" to be something like the following:

exec cue export x.cue working.cue
cmp stdout stdout.golden

! exec cue export x.cue broken.cue
cmp stderr stderr.golden

-- x.cue --
package x

#main: {
	{
		opt1?: string
		opt2?: _|_
	} | {
		opt1?: _|_
		opt2:  string
	}

	// Not sure whether you wanted the definition open or not.
	// Comment out if not.
	...
}

-- working.cue --
package x

examples: [string]: #main

examples: obj1: #main & {
	opt1: "foo"
}

examples: obj2: #main & {
	opt1: "foo"
	opt3: "test"
}

-- broken.cue --
package x

examples: bad: #main & {
	opt1: "foo"
	opt2: "test"
}
-- stdout.golden --
{
    "examples": {
        "obj1": {
            "opt1": "foo"
        },
        "obj2": {
            "opt1": "foo",
            "opt3": "test"
        }
    }
}
-- stderr.golden --
examples.bad: 2 errors in empty disjunction:
explicit error (_|_ literal) in source:
    ./x.cue:6:10
explicit error (_|_ literal) in source:
    ./x.cue:8:10

https://tip.cuelang.org/play/?id=GmmKjdTk_h9#cue@export@json contains a variation of that.

[b4nst]

Yup that would work. The error message would not the best in term of UX, but in term of functionality that would fix the issue.

[myitcv]

Yup that would work. The error message would not the best in term of UX, but in term of functionality that would fix the issue.

Absolutely. A builtin would facilitate a much more precise error message.

[rogpeppe]

ISTM there are a few of issues going on here. For reference, here's a full testscript example that includes the source OpenAPI spec that wasn't included above:

exec cue import -p x spec.yml
exec cat spec.cue
exec cue vet -c
-- spec.yml --
openapi: 3.1.0
info:
  title: OneOf test
  version: 1.0.0
components:
  schemas:
    main:
      description: Oneof test
      oneOf:
        - required:
            - opt1
        - required:
            - opt2
      properties:
        opt1:
          description: option 1
          type: string
        opt2:
          description: option 2
          type: string
      type: object
-- cue.mod/module.cue --
module: "example.com"

-- test.cue --
package x

examples: [string]: #main

examples: obj1: opt1: "foo"
examples: obj2: opt2: "foo"

This currently fails:

> exec cue import -p x spec.yml
> exec cat spec.cue
[stdout]
// OneOf test
package x

info: {
	title:   *"OneOf test" | string
	version: *"1.0.0" | string
}
// Oneof test
#main: ({
	opt1: _
	...
} | {
	opt2: _
	...
}) & {
	// option 1
	opt1?: string

	// option 2
	opt2?: string
	...
}
> exec cue vet -c
[stderr]
examples.obj1: incomplete value {opt1:"foo",opt2?:string} | {opt1:"foo",opt2:string}
examples.obj2: incomplete value {opt2:"foo",opt1:string} | {opt2:"foo",opt1?:string}
[exit status 1]
FAIL: /tmp/testscript2972845514/x.txtar/script.txtar:3: unexpected command failure

Specific issues:

• since both arms of the disjunction are valid for the input, we get the incomplete value error. This is the issue that @myitcv points out above.
• the #main definition is using regular fields where it should be using required fields
• the disjunction is unified with a struct that contains all the (optional) fields; instead, the fields should only be defined in the arms of the disjunction.

There's another issue: in the above example, additional fields are allowed, but if we explicitly try to disallow them (by specifying additionalProperties: false), the resulting CUE allows them anyway. Here's a repro for that:

exec cue import -p x spec.yml
exec cat spec.cue
exec cue vet -c
-- spec.yml --
openapi: 3.1.0
info:
  title: OneOf test
  version: 1.0.0
components:
  schemas:
    main:
      description: Oneof test
      oneOf:
        - required:
            - opt1
        - required:
            - opt2
      properties:
        opt1:
          description: option 1
          type: string
        opt2:
          description: option 2
          type: string
      additionalProperties: false
      type: object
-- cue.mod/module.cue --
module: "example.com"

-- test.cue --
package x

examples: [string]: #main

examples: obj1: opt1: "foo"
examples: obj2: opt2: "foo"

Result:

> exec cue import -p x spec.yml
> exec cat spec.cue
[stdout]
// OneOf test
package x

info: {
	title:   *"OneOf test" | string
	version: *"1.0.0" | string
}
// Oneof test
#main: ({
	opt1: _
	...
} | {
	opt2: _
	...
}) & {
	// option 1
	opt1?: string

	// option 2
	opt2?: string
}
> exec cue vet -c
[stderr]
examples.obj1: incomplete value {opt1:"foo",opt2?:string} | {opt1:"foo",opt2:string}
examples.obj2: incomplete value {opt2:"foo",opt1:string} | {opt2:"foo",opt1?:string}
[exit status 1]
FAIL: /tmp/testscript1805129990/x.txtar/script.txtar:3: unexpected command failure

Even without an explicit new builtin for this purpose, the above should work: it should probably produce something like the following definition for #main, which then validates OK:

#main: {
	opt1!: _
} | {
	opt2!: _
}

[myitcv]

There's another issue: in the above example, additional fields are allowed, but if we explicitly try to disallow them (by specifying additionalProperties: false), the resulting CUE allows them anyway.

I'm not 100% on the detail but I think this is an artefact of OpenAPI's usage of JSON Schema. I have a vague recollection of @mpvl telling me that this "encoding" within CUE is deliberate, because OpenAPI is intentionally "open".

[rogpeppe]

FWIW I looked for any indication of this kind of difference and was unable to find one. To the best of my knowledge, OpenAPI's JSON Schema is exactly the same as regular JSON Schema. Here's some evidence of that: https://apisyouwonthate.com/blog/openapi-v3-1-and-json-schema/
I'm fairly sure that if there was a distinction in the treatment of additionalProperties between the two, it would have been mentioned in that article or one of those that it points to.

[myitcv]

Very related to this I've just raised #3165 to capture discussion on how oneofs could/should be encoded in CUE. That new issue provides a JSON Schema example, but I don't think the question of how we encode oneofs is specific to JSON Schema necessarily. Protocol Buffers also has the concept - quite how the semantics and implementation lines up I'm not 100% sure.

On the basis OpenAPI uses an extended subset of JSON Schema, I'm tempted to close this issue and move discussion of how we should encode oneofs (perhaps generally) into #3165, consolidating this and a number of other issues.

Please do say though if I'm holding things wrong and we need to keep this open for OpenAPI-specific reasons. If we do, we need to evolve the conversation in #3165 in any case to make clear how specific encodings might carry with them specific oneof-semantics/details.