Enabling geometry variants in glTF

[sanjeetsuhag]

When using glTF models in a simulation or even on a e-commerce website, it may be useful to provide geometric variations inside a model. The KHR_materials_variants extension enables using variants in materials; this issue is for discussion on how to enable node/mesh variants in glTF.

The example I'm working with is of a car (and its components) that can have "damage states". These damage states may be:

enum DamageState {
   Pristine = 0,
   Damaged = 1,
   Destroyed = 2
}

Let's look at what could actually change between damage states, in increasing order of flexibility:

1. A mesh is morphed

• This could be achieved by KHR_material_variants and morph targets/animation/skinning (essentially pre-baking static poses).
• This approach requires the least additions to glTF, but it could be really tricky to work with when generating data.

2. A mesh is switched

• This could be achieved by an extension that enables variants of a mesh in a node.
• This approach allows using completely different geometry but the node graph remains unchanged.

3. A node is switched

• This could be achieved by an extension that enables variants of a node.
• This could get tricky if we want to enable switching node "deeper" in the scene graph, but if all the glTF extension does is provide the variant mappings, it may be feasible to represent something like this:

Here's a snippet of a glTF where we use "switch nodes" with variants - using a simple extension inspired by KHR_material_variants. A "switch node" cannot have a mesh and its children can only be other switch nodes. When a variant is not selected, the 0th variant is the default.

{
  "extensions": {
    "EXT_node_variants": {
      "variants": [
        { "name": "Pristine" },
        { "name": "Damaged" },
        { "name": "Destroyed" }
      ]
    }
  },
  "nodes": [
    {
      "name": "Car Body Switch Node",
      "children": [ 4 ], 
      "extensions": {
        "EXT_node_variants": {
          "mappings": [
            {
              "node": 1,
              "variants": [0]
            },
            {
              "node": 2,
              "variants": [1]
            },
            {
              "node": 3,
              "variants": [2]
            }
          ]
        }
      }
    },
    {
      "name": "Car Body Pristine Node",
      "mesh": 0
    },
    {
      "name": "Car Body Damaged Node",
      "mesh": 1
    },
    {
      "name": "Car Body Pristine Node",
      "mesh": 2
    },
    {
      "name": "Car Door Switch Node",
      "children": [ 8 ], 
      "extensions": {
        "EXT_node_variants": {
          "mappings": [
            {
              "node": 5,
              "variants": [0]
            },
            {
              "node": 6,
              "variants": [1]
            },
            {
              "node": 7,
              "variants": [2]
            }
          ]
        }
      }
    },
    {
      "name": "Car Door Pristine Node",
      "mesh": 3
    },
    {
      "name": "Car Door Damaged Node",
      "mesh": 4
    },
    {
      "name": "Car Door Destroyed Node",
      "mesh": 5
    },
    {
      "name": "Car Door Handle Node", 
      "extensions": {
        "EXT_node_variants": {
          "mappings": [
            {
              "node": 9,
              "variants": [0]
            },
            {
              "node": 10,
              "variants": [1, 2]
            }
          ]
        }
      }
    },
    {
      "name": "Car Door Handle Pristine Node",
      "mesh": 6
    },
    {
      "name": "Car Door Handle Damaged/Destroyed Node",
      "mesh": 7
    }
  ]
}

Here is a graphic for the schema above:

This is just an initial design and example. We're hoping to get some more insight on how to best achieve the functionality described. Also, discussions on other use cases for such functionality are welcome.

[pjcozzi]

Thanks for sharing @sanjeetsuhag

@bghgary is this potentially interesting to you as a progression of MSFT_lod?

[emackey]

Thanks for this proposal!

One of the lingering issues I have with KHR_materials_variants is that we didn't fully address the concept of the "default" material that appears outside the extension. In the StainedGlassLamp model for example, there are two variants, "Lamp On" and "Lamp Off", and most software shows a third variant, typically with some incongruent name like "Default" or "None".

• None
• Lamp On
• Lamp Off

Invariably, "None" ends up being the same as one of the other selections from the list, although which one is not immediately apparent to the user. It's guaranteed to be the one shown by viewers that don't support variants, however.

Ideally, the extension's design would not allow for a third-wheel un-named option like this. Instead, it would ensure that one particular option was the default option, and exactly matched the selection shown by viewers that don't handle the extension.

In the template glTF shown above, it looks like this proposal doesn't offer a fallback path at all. There's no default node to show when the viewer doesn't process the extension. I would think that the pristine car might be the one to show by default. If that can be added without duplicating my main issue with material variants, that would be great.

One more thought: Is it easier for most engines to swap out part of the node tree? Or would it be simpler to swap out just the mesh attached to one node, allowing the node tree to remain static? Each node could have a standard mesh index and a list of variant mesh indices, perhaps.

[emackey]

Related: There was a similar proposal for mesh variants last year in #1790. It needs a bit of work given what we know now about material variants, but is similar in spirit.

Also somewhat related: KHR_nodes_disable extension proposal, #1760