The KHR_materials_variants extension #1681
Conversation
zellski
changed the title
|
|
||
| (_the authors of this spec are not product designer, apologies for the dubious colour choices_) | ||
|
|
||
| The currently active `material` for a `mesh primitive` is found by stepping through this array of mappings, and selecting the first one which contains any one of the currently active tag. If none match, fall back on vanilla glTF behaviour. |
There was a problem hiding this comment.
This paragraph suggests that an application would look for a material containing "at least one" of several currently active tags, whereas the example below (['sneaker_red', 'laces_gold']) suggests an application would be looking for all active tags, or at least some sort of "best fit".
I wonder if the tags should simply be specified as having some application-defined semantic meaning, and left at that. Discussion of how an application should navigate among its tagged materials could be left to a non-normative section, as I could easily imagine different applications doing that differently. Or do you think that would be too loose to have models using this extension be broadly portable?
There was a problem hiding this comment.
I've been thinking about tags more and my feeling is that tags don't represent the "first match" nature of this design well. In fact when I wrote a prototype application around this I had forgotten the spec. I actually implemented "best match" implementation where the material with the most matching tags was applied.
I feel like they should be called selectors or something denoting that one the first matching material will be applied.
|
|
||
| Composition also works well with tags: if a scene were built from several different variational models, each with their own set of tags, then it's easy to imagine an API call on the entire scene that takes a set of tags and passes them on to each constituent model, recursively. | ||
|
|
||
| ## Interaction with existing glTF functionality |
There was a problem hiding this comment.
I think the intention is clear, but consider explicitly putting a non-normative note at the top of this section and the one below:
This section is non-normative.
|
|
||
| Beyond that, it's also a compression scheme: a way to compactly represent multiple assets that have a lot in common (e.g. most geometry, some textures). In this respect, it's most meaningful for the self-contained binary format GLB, where the asset file contains all its data up-front, both that which is shared between variants and that which differs. | ||
|
|
||
| If we also allow external references, i.e. images and buffers to source from URLs,then we can additionally come up with sophisticated mechanisms where geometry and textures are loaded separately, dynamically. There's pragmatic pros and cons to going down this route, all beyond the scope of this document. Each application will make its own decisions there; meawhile, the extension proposed works well regardless. |
There was a problem hiding this comment.
These two paragraphs (Beyond that, ... and If we also...) seem to contradict one another, and may be confusing as-is. I think you could effectively condense them to something like:
As a secondary effect, material variants allow multiple assets — with shared geometry but different materials — to be stored more compactly. When using external URIs as references to textures, applications may (optionally) process geometry only once and lazily request texture assets only when needed for a particular variant.
On this section...
If we also allow external references, i.e. images and buffers to source from URLs,then we can additionally come up with sophisticated mechanisms where geometry and textures are loaded separately, dynamically.
... I would consider this a certainty rather than an "if"; incrementally loading parts of a glTF asset is already possible and supported in engines like three.js. It's probably out of scope for this document, as you say, but I would strongly expect (though not require) that applications will take advantage of it.
There was a problem hiding this comment.
Thanks for the suggestions here @donmccurdy, I've gone ahead and condensed the language of the non-normative sections as per your proposal
|
|
||
| (_the authors of this spec are not product designer, apologies for the dubious colour choices_) | ||
|
|
||
| The currently active `material` for a `mesh primitive` is found by stepping through this array of mappings, and selecting the first one which contains any one of the currently active tag. If none match, fall back on vanilla glTF behaviour. |
There was a problem hiding this comment.
How does an app like Babylon.js Sandbox will know what is the active tag at load? Should we think about adding a "Default tag" information? Or do we consider that by default there is no selected tag so we pick the gltf vanilla material?
There was a problem hiding this comment.
I like the idea that the default tag is the gltf vanilla material. No need to specify it. That way authors are able to specify what the default should look like even on viewers that don't implement this extension.
| "mapping": [ | ||
| { | ||
| "tags": [ "sneaker_yellow", "sneaker_orange" ], | ||
| "material": 7, |
There was a problem hiding this comment.
Would you consider making adding visible as another property here and making all but tags optional?
In my work with customization we are frequently switching some small parts of the larger model based on user choice.
In terms of the example sneaker imagine that each of the variants was designed by a different designer and each or them have added a different volumetric brand mark at the back of the sole. Each of those logo geometries should be visible only when a particular tag is selected.
There was a problem hiding this comment.
I'm not sure if a visible flag is needed as the mapping is only used as a look up table that lists all of the materials that can be applied to the mesh. In the case of volumetric logos that have associated geometry, I'd say that would fall out of scope of the material variants extension as that would start changing the nodes a level up from the material.
|
|
||
| ## Tagged Variants | ||
|
|
||
| We introduce a simple tag-based extension scheme, that allows for high-level runtime swapping between which `material` is used to shade a given `mesh primitive`: the extension root contains a mandatory `mapping` property, which is an array of objects, each one associating some set of tags with a material reference. |
There was a problem hiding this comment.
Although in my customization work (athletic apparel and sports equipment) we are currently adding customization as a layer of logic around the model, I believe we should be able to replace a lot of the manipulation logic already with this declarative approach. Will try that out and report on the findings. But I have added some immediate conclusions below.
|
|
||
| Beyond that, it's also a compression scheme: a way to compactly represent multiple assets that have a lot in common (e.g. most geometry, some textures). In this respect, it's most meaningful for the self-contained binary format GLB, where the asset file contains all its data up-front, both that which is shared between variants and that which differs. | ||
|
|
||
| If we also allow external references, i.e. images and buffers to source from URLs,then we can additionally come up with sophisticated mechanisms where geometry and textures are loaded separately, dynamically. There's pragmatic pros and cons to going down this route, all beyond the scope of this document. Each application will make its own decisions there; meawhile, the extension proposed works well regardless. |
There was a problem hiding this comment.
As we are presenting custom-made products we are generating a lot of textures based on user input (text, logo files, patterns, colors, settings, placements, etc.). In our customizers those are primarily generated on client side but in a generalized scenario those could be generated by a server if the server receives all the relevant properties for a given texture request.
In a more generic example this would be relevant for a custom dog collar on Amazon (I have ordered one and wish the presentation was more accurate).
Would you consider including something like that?
I recognize that there are concerns about wastefully requesting all textures based on updates of properties that do not impact the textures. We could remedy that, for example, by specifying which properties need to be passed to that or (in worst case) allow server-side redirects to links that contain only the needed properties.
|
|
||
| ## Implications for Applications and APIs | ||
|
|
||
| How does an application communicate to a glTF engine what the initial variant state should be? How does it submit a runtime request for a different configuration? It's out of scope for this extension to constrain or mandate an engine's public API, but a useful implementation will require something of the sort. |
There was a problem hiding this comment.
Although we might not enforce users not making conflicting declarations I think we might need to cover the initial state by this spec in some way that helps avoid situations where the models by default look different on viewers that do not support this extension.
Maybe that could be solved by validation logic or a somewhat advanced json schema?
| | `sneaker_red` | `shoelace_hole_material_purple` | | ||
| | `sneaker_black` | `shoelace_hole_material_yellow` | | ||
|
|
||
| (_the authors of this spec are not product designer, apologies for the dubious colour choices_) |
There was a problem hiding this comment.
Would you consider exposing in metadata all the choices that the users can make and all the values that they can specify? There might be more than just a single set of variants.
Those could drive the UI. And even if the UI is defined by the database this could be a way to ensure that the model is capable of representing all the same options exposed to the user.
If this ends up being applied, I would say that the tagnames would benefit from a well-defined format that combines both the property name together with the selected value. For example underscores might not be allowed for property and value ids and be reserved only for concatenating those together.
There was a problem hiding this comment.
Would you consider exposing in metadata all the choices that the users can make and all the values that they can specify?
Yea I think we would need to expose multiple tag 'sets' through metadata. When we talk about driving UI with content from the glTF extension I get a bit wary as we should try to keep the data stored in the extension as business-logic free as possible.
If this ends up being applied, I would say that the tagnames would benefit from a well-defined format that combines both the property name together with the selected value.
I think having a clear and standardized way of tagging will make it easier for viewers to implement the spec so I can see this being useful regardless.
There was a problem hiding this comment.
This has been brought up several times and consensus so far is to keep business-logic out of this extension. It can definitely lead to a rabbit hole of complexity as soon as there are thousands of combinations or rules start being needed (e.g The red laces can go with the yellow or red sole, but not the blue sole unless the purple heel is chosen).
It also puts a big burden on the 3D authoring tools. We can't expect 3D modelling tools to support defining the metadata of all possible configurations.
|
@donmccurdy @pushmatrix I feel like this extension has converged into a form that a few different companies are comfortable with using. With the exception of clarifying some of the description for this PR, do you see anything preventing renaming this to be an EXT extension? Most of the development on this is coming from 3D commerce instead of just Facebook so it would seem reasonable to transition it into an EXT. If there is other criteria let me know and we can move forward on those points. |
|
@sleroux Sorry for the delay — the use of an EXT prefix would be reasonable, yes. There is no additional formal requirement beyond the adoption of multiple vendors. |
| @@ -0,0 +1,24 @@ | |||
| { | |||
| "$schema": "http://json-schema.org/draft-04/schema", | |||
There was a problem hiding this comment.
The filename for this file should be KHR_variant_materials.schema.json no?
There was a problem hiding this comment.
good catch, will fix with latest rename
zellski
changed the title
|
I think https://github.com/KhronosGroup/glTF/tree/master/extensions#naming |
|
@emackey The idea was to avoid confusing this extension with other PBR Next / material rendering extensions |
|
Oh, I see. I haven't been following this one closely but I didn't get that impression from the name being different. If it becomes a KHR extension, there may not be much need to separate it from PBR next, or there may be some better way to indicate that separation. I'd rather see the ecosystem's names be consistent, I think it looks cleaner and more organized. But that's just my $0.02. |
|
+1 for _materials_variants. That it affects materials in a different way than existing material extensions doesn't change the fact that it is primarily scoped for materials, and reordering the words doesn't particularly clarify how it's different from existing extensions. The same is true for In #1681 (comment) it was suggested that this move from FB_ to EXT_, but it seems to have gone directly to KHR_. Was there other discussion or rationale on this? |
|
@bghgary I haven't made a PR yet with the updated models, but here's a link to them in the meantime: https://www.dropbox.com/s/ynity52sut4t6x2/khr_materials_variants_models.zip?dl=0. @jercytryn I assume we can remove them from this PR now. |
extensions/2.0/Khronos/KHR_materials_variants/schema/mapping.schema.json
Outdated
Show resolved
Hide resolved
|
@jercytryn and @pushmatrix — would it be possible to get the spec and schema in this PR updated in the next day or so? Just want to be sure the PR is aligned with the plan described in #1681 (comment). Thanks! |
|
@donmccurdy Trying to! |
|
Schema and spec updates are applied. ✅ |
|
I see the sample models have moved to KhronosGroup/glTF-Sample-Models#271, but they should still be deleted from here. Also they need to be updated with the latest schema changes. |
...nos/KHR_materials_variants/schema/mesh.primitive.KHR_materials_variants.mappings.schema.json
Outdated
Show resolved
Hide resolved
extensions/2.0/Khronos/KHR_materials_variants/schema/glTF.KHR_materials_variants.schema.json
Outdated
Show resolved
Hide resolved
...nos/KHR_materials_variants/schema/mesh.primitive.KHR_materials_variants.mappings.schema.json
Outdated
Show resolved
Hide resolved
…th clarifications around some properties Signed-off-by: Jeremy Cytryn <jcytryn@fb.com>
extensions/2.0/Khronos/KHR_materials_variants/schema/glTF.KHR_materials_variants.schema.json
Outdated
Show resolved
Hide resolved
jercytryn
force-pushed
the
ext/zell-fb-asset-variants
branch
from
2439171
to
8543426
Compare
…roperties Also fixes dead image link
|
Made a last round of updates to clean up / simplify the spec's description of the schema and fixed broken image link (Good catch @emackey ) |
.../2.0/Khronos/KHR_materials_variants/schema/mesh.primitive.KHR_materials_variants.schema.json
Outdated
Show resolved
Hide resolved
EDIT: Since this original write-up, the extension has entered the pipeline to become
KHR_materials_variantsThis pull request introduces our vendor extension,
FB_material_variants, and it is our hope that in this discussion thread, we'll see opinion converge on promoting it to EXT level.I recommend anyone interested to catch up on the excellent & varied conversation in #1569, where much ground on the subject has already been covered.
In order to limit scope creep here, I also propose the likelihood of a future & more powerful
EXT_node_variants, which would use the same type of simple tag-based mechanism, except to switch the children of a node, rather than the material of a mesh primitive. That's a far more ambitious project, though, and doesn't obviate the need for a simple extension like this that covers the common use cases.Finally – there will shortly be a legally unencumbered tool on GitHub that produces files on this extension. It'll be a CLI tool at first, with a WebAssembly version later that runs entirely in-browser.