[gltfpack] Issue with the displacement of position in animated skinned meshes

[TimvanScherpenzeel]

Hi Arseny,

Thank you for all your hard work on gltfpack!

I've been testing various glTF models from the glTF-Sample-Models repo and came across a few that have issues after packing with gltfpack with default settings.

• BrainStem displaces the position of the mesh from the original position. Animations themselves appear to work correctly.

Correct

Incorrect

• CesiumMan has the same issue, displaces the position of the mesh from the original position. Animations themselves appear to work correctly.

Correct

Incorrect

• AnimatedCube does not spin anymore after conversion. There is an animation track but it does not appear to play correctly.

Incorrect

• BoxAnimated does not animate anymore after conversion. There is an animation track but it does not appear to play correctly.

• TriangleWithoutIndices does not render correctly, the viewer throws the following error: Cannot read property 'updateMatrixWorld' of undefined.

I've created a fork of three-gltf-viewer that uses the latest versions of lib/GLTFLoader.js and js/meshopt_decoder.js from this repo (using raw.githack.com) directly to test the implementation more easily, perhaps it can also be of use to you during development.

Kind regards,

Tim

[zeux]

Thanks! This is very helpful. I know there's currently one issue with transforms where if the node itself is animated and not skinned, gltfpack currently doesn't preserve this properly - this will be fixed soon-ish. I'll take a look at all these examples and make sure they work as well.

Note that gltfpack can be used with "-c" flag which compresses the buffers and requires meshopt_decoder, but also can be used without it which doesn't require meshopt_decoder (although it does require a couple of fixes in three.js and Babylon.JS that are all in master versions of these projects but not in the versioned releases yet). I've been alternating between gltf-viewer, Babylon.JS sandbox, and demo/index.html for testing, but the fork will help make this easier 👍

[zeux]

Animated objects should render correctly now. For the first two issues, as far as I can tell the meshes themselves are generated correctly, but it appears that three.js (and Babylon.JS maybe?) generate the bounding box in an odd way which causes the discrepancy in the view setup. I'm not sure atm how to best fix this, this will require a bit of thought - the issue seems to stem from the fact that node transform matrix with an attached skin mesh is ignored for the purposes of mesh transformation, but seems to be respected for the purposes of computing the bounding box...

[zeux]

Yeah, ok, aaaaaargh.

1. Three.JS computes bounding box for a mesh by transforming all positions by the matrix of the node mesh is attached to. This means that bounding box changes depending on the node a skinned mesh is attached to, even though visual results don't change since gltTF spec says that mesh node transform is ignored for skinned meshes.

2. Moving meshes under the original place in the hierarchy (where they were in the source file) fixes bounding box for the two meshes for Three.JS, but doesn't fix one of them (CesiumMan) for Babylon.JS.

3. This is because BabylonJS does the correct thing and computes bounding box by skinning all positions, but during this it assumes that position buffer has 3 elements for each position. gltfpack currently generates a position buffer with 4 elements (VEC4). Babylon.JS is correct in its assumption - positions must be VEC3 - however...

4. The reason gltfpack generates 4-element positions is to work around a three.js bug with interleaved buffers (GLTFLoader: Invalid interleaved buffer setup when multiple attribute accessors share the same buffer mrdoob/three.js#16802).

All of this is broken :( In theory these are just bugs in JS frameworks - if gltfpack used VEC3 positions then Babylon.JS behavior would be completely correct, but then this would hit two independent bugs in three.js.

[zeux]

The issues should be fixed now with the exception of TriangleWithoutIndices (which results in an empty .gltf file that three.js can't load), although the fix for the bounding box is a bit of a hack.

For TriangleWithoutIndices I'll need to add warnings when meshes are discarded but looking at the spec there's a lot of types of index data that I'm not sure are relevant so I will not fix this for now.

[TimvanScherpenzeel]

Awesome 👍!

[zeux]

Closing this since the problems have been addressed, feel free to open a new issue if you find anything else!