Add OMI_collider

[RangerMauve]

As per the discussion at the last GLTF working group meeting, here's an initial draft of the READMEs for the OMI_collider and OMI_physics_body extensions.

Based on the initial draft from this hackmd

These proposals add:

• A way to specify collision areas within a scene (different types)
• A way to attach colliders to physics bodies or trigger areas.

Some TODOs:

• Add proper JSON schema files to the respective folders
• Get more feedback on the spec
• Initial implementations in engines
• Talk more about authoring workflows (blender? Godot?)

[fire]

Hull meshes can be used to be used compound, multiples of 255 vertices compunded.

[fire]

compund -> compound

[RangerMauve]

BIG TODO:

• Should hulls be assumed to be pre-baked meshes? Should we expect engines to dunamically process them?
• Triggers: Body type? Property of colliders? Property of Physics Bodies?

[robertlong]

From our meeting: Is there any way to represent a cylinder with a capsule? Is that currently the right way to represent a cylinder in a physics engine that doesn't support cylinders?

[RangerMauve]

TODO:

• Remove physics body into its own PR
• Move trigger into collider
• Specify that by default colliders make static physics bodies (or Rigid if it's animated?)
• Note that we will have more detailed physics extensions in the future.

[mrmetaverse]

bump, whats our status on moving this forward to done @robertlong @RangerMauve :) ?

[robertlong]

Blocked on time needed to split these into two PRs and start working on an implementation of OMI_collider. We want to focus on the OMI_collider extension first and tackle physics bodies next.

I probably will have time to start implementing something in Third Room in the next couple weeks and can add to this when I start on that.

If anyone else is working on an implementation sooner, let us know.

[antpb]

thanks for the bump @mrmetaverse ! I split the physics body out to its own PR and now this PRcan remove the physics body folder and continue on toward a final Collider extension :D

#75

[antpb]

If given access to your fork @RangerMauve I can make the update to remove the physics body from this PR. (I think its your repo I would need access to?)

onward! :D

[aaronfranke]

Related GLTF proposal: KhronosGroup/glTF#1986

[memelotsqui]

As commented on tuesday meeting, here is the proposal I used in collider, I know this also includes physics bodies, but we can focus on the collider section if needed :)

{
    "name": "A Random Box",
    "translation": [-1, 1, 0],
    "extensions": {
        "OMI_collider": {
            "type": "box",          //type of collider
            "extents": [0.3531689, 0.751563, 0.3443799],  
            "center": [-0.0108851, 0.04044807, 0],  // offset from the center of this objects position (default [0,0,0])
            "sensor": true,         //sensor objects do not collide, but provide information when another collider is within (default false)
            "material": {           //the type of physics material (maybe it can be added by index instead?) (optional)
                "friction": 0.6,    
                "restitution": 0,
                "combineFriction": "average",   //options constants: [average, multiply, min, max]
                "combineRestitution": "average"
            },
            "rigidBody": {          //optional parameter, does this collider has rigidbody? 
                "gravity": 0.9,     //gravity multiplier modifier (default 1)
                "density": 2,       //density of the object (dedfault 1)     
                "damp": 1,          //dampening of the object (slows the movement of the objects position) (default 0)
                "angularDamp": 0.05,//angular dampening of the object (slows the rotation of the objects position) (defualt 0)
                "positionLock": {   //locks movement in any of the axis (default false in all) - maybe use array instead of object ? example: positionLock:[true,false,true]
                    "x": true,
                    "y": false,
                    "z": true
                },
                "rotationLock": {
                    "x": true,
                    "y": false,
                    "z": true
                },
                "ccd": true         //continous collision detection (default false)
            }
        }
    }
}


{
    "name": "A Random Sphere",
    "translation": [-1, 1, -1],
    "extensions": {
        "OMI_collider": {
            "type": "sphere",          
            "radius": 0.3531689,  
            "center": [-0.0108851, 0.04044807, 0],
            ...  
            }
        }
    }
},

{
    "name": "A Random Capsule",
    "translation": [-1, 1, -1],
    "extensions": {
        "OMI_collider": {
            "type": "sphere",          
            "radius": [0.3531689],  
            "height": [0.3531689],  
            "center": [-0.0108851, 0.04044807, 0],  
            ...
            }
        }
    }
},

{
    "name": "A Random Mesh",
    "translation": [-1, 1, -1],
    "extensions": {
        "OMI_collider": {
            "type": "hull",          
            "mesh": 32, // mesh accessor id
            "isHull": true, //default false
            "center": [-0.0108851, 0.04044807, 0],  
           ...
            }
        }
    }
},

[fire]

Comment by @lyuma was that center is duplicating the gltf translation property. When converting GameObjects from Unity the center can be emulated via adding a child node.

[fire]

@antpb mentioned that for a demo, we could implement this as blender extension as a button that can create a component creating the internal nodes.

Has an hack to show a visible presentation of the node.

https://github.com/omigroup/three-omi/tree/physics

[fire]

Add all contributors to the spec.

[fire]

@antpb mentioned that a next step can be an example:

1. three omi
2. Does not need to be only three omi but it is important to have an example.

Create an example asset as a sample for implementation without editing the gltf.

[fire]

There was multiple attempt to draft a collider spec for gltf.

1. Maxar
2. Gltf collider by @aaronfranke
3. Other implementations
4. O3DE implementation

Task to contact and to reconcile differences.

[aaronfranke]

Typo, gLTF should be either glTF or GLTF (I see both of these in the file).

[aaronfranke]

I highly recommend adding cylinders to this list. Cylinders are very useful, I would use them extensively. It's the perfect shape to use for barrels and circular platforms (which I'd like to store in GLTF files).

For engines that do not support cylinders, they can be approximated with capsules, and we can provide guidelines of what to do if this is needed. Maybe even allow providing extra data to hint at what the best way to approximate the shape is (like shrink/grow/average)? We could also discourage the use of cylinders in general, but I think it's very much worth having it in the standard since there are many use cases.

[aaronfranke]

We should consider making this size instead of extents, and making this be an array representing the "diameter" of the box. I think it's noticeably more intuitive this way, so much so that Godot switched from extents to size in Godot 4.0.

Also, like Sphere with a default radius of 0.5, we should specify that a box has a default size of [1, 1, 1].

[RangerMauve]

I think I came up with these names from looking at existing properties in different engines. Are any engines using a generic extents for everything?

[aaronfranke]

In the example above:

{
    "name": "A Random Capsule",
    "translation": [-1, 1, -1],
    "extensions": {
        "OMI_collider": {
            "type": "sphere",          
            "radius": [0.3531689],  
            "height": [0.3531689],  

"type" should be "capsule", and "radius" & "height" should be single values rather than arrays.

[aaronfranke]

isTrigger should be removed from OMI_collider, it's out-of-scope to define usage for the collision shapes in this extension, and it's already a part of OMI_physics_body.

EDIT: Hmm, good points below, see my latest comment.

[RangerMauve]

I think some engines combine whether a collider is a trigger or not with the physics body (I think it was unreal) so I'm not sure I agree with that. 🤔

[memelotsqui]

Hmm in my opinion I think "isTrigger" actually should be here in OMI_collider and not in OMI_physics, a collider may be a trigger without the need of having a rigidbody (physics), but a rigidbody -without- a collider, wont work with "isTrigger" flag.

Also, Im taking as reference from how its applied in Unity3D (https://docs.unity3d.com/ScriptReference/Collider.html) where isTrigger is applied from Collider component, and also im taking as reference how is applied in Rapier3D (https://rapier.rs/docs/user_guides/javascript/colliders) where sensor is set from Collider definition

[aaronfranke]

@memelotsqui Good points! In Godot, a trigger needs an Area node as a parent of the colliders.

How does it work in Unity if you want multiple collider shapes to trigger the same thing?

If we have this API on both the colliders and physics bodies, we can still import this in Godot, but we should specify how these are imported. Perhaps if a trigger collider has a parent physics object of type trigger, then they use that parent physics object Area node (and possibly multiple shapes could use it, not sure how this would work in Unity). Otherwise if trigger colliders are on their own without a trigger physics object parent, we generate 2 nodes in Godot, an Area and a collider shape child, or 1 GameObject with a component in Unity, and then this trigger is on its own.

If we move this API to only be on the colliders and not on the physics bodies, then this basically means that we would always have to generate 2 nodes in Godot, and always 1 GameObject with a component in Unity, and it wouldn't be possible to make a trigger have multiple shapes.

[lyuma]

My understanding for Godot is that you can add an Area node as a child of a physics body, and then put all of the "isTrigger" collision shapes as children of the area node, while adding the non-isTrigger shapes as direct children of the physics body itself.

I have seen multiple posts online that use this trick inside of Godot to detect collision events separate from the body itself, such as: https://godotengine.org/qa/92172/collision-detection-independent-of-a-kinematicbody

It would be good to make a demo project so we can show that this works, but I'm pretty sure it is possible.

[humbletim]

found these ammo binding docs earlier while looking at mozilla hubs. check out the fit option -- which seems to make using simple primitives that surround whole meshes more straightforward.

https://github.com/n5ro/aframe-physics-system/blob/master/AmmoDriver.md#shape-fit

Shape Fit

• fit: all – Requires a mesh to exist on the entity. The specified shape will be created to contain all the vertices of the mesh.
• fit: manual – Does not require a mesh, however you must specifiy either the halfExtents or sphereRadius manually. This is not supported for hull, hacd, vhacd and mesh types.

[RangerMauve]

found these ammo binding docs earlier while looking at mozilla hubs. check out the fit option -- which seems to make using simple primitives that surround whole meshes more straightforward.

I think this was brought up at a previous meeting, too. From what I remember "fit" is not a universal feature of runtimes in engines so it would make it a lot harder to incorporate such a feature everywhere. I think we also discussed how GLTF was likely going to get authored in an engine or editor somewhere ahead of time where colliders could be generated to fit before exporting the file.

[humbletim]

note that in the referenced docs "fit" refers to a feature of the binding itself (not of the underlying ammo physics engine).

considering how trimesh support would already require an engine to access mesh triangles as part of generating a collision hull, it's possible to see fit as an extension of that existing effort -- and one that engine developers might strongly prefer to have available given the performance advantage primitive colliders tend to offer.

so perhaps: fit == (trimesh + author's blessing for the engine to radically simplify the resulting hull)

[robertlong]

Outstanding questions for this extension and the OMI_physics_body extension:

• Where should physics materials belong in the spec? OMI_physics_body or OMI_collider or should it belong in it's own extension?
• How do we assign different physics materials per glTF primitive?
• How do we assign different physics materials per triangle? And how does this map to the different physics engines? https://gameworksdocs.nvidia.com/PhysX/4.1/documentation/physxguide/Manual/MigrationFrom28.html#material-indexes
• How do we want to specify hull collider types?
  • Perhaps we introduce hull decomposition as library-specific implementations like V-HACD https://github.com/kmammou/v-hacd

[aaronfranke]

@robertlong I think this would make sense to have in its own dedicated OMI_physics_material extension. OMI_collider is purely about specifying shapes, and OMI_physics_body is about specifying the purpose of physics objects (and likely also the mass - because that's very simple).

[memelotsqui]

These are my toughts on @robertlong questions:

I personally think physics material should go in collider, instead of being in the physics body: It seems that even if colliders don't have a physics body, they can have a physics material, and any other object with physics material will behave accordingly as they're in contact with them.

Also, as @aaronfranke mentioned I think it should have a separate extension and be connected in collider by an id: example:{physicsMaterial : id}.

Having it connected by id instead of fully defining it inside the OMI_collider (besides looking clean and compact :) ) might also be useful for programming section to modify all materials connected to the shared physics material (for example, change all the ice material friction to 0.1)

To assign different materials per mesh, maybe we could use an array instead of a single ID? ( example {physicsMaterials : [ids]} ), and Im not sure, but maybe this could also work in triangle definition if something like mesh groups are allowed/supported in mesh physics (https://threejs.org/docs/#api/en/core/BufferGeometry.groups), if its not supported we could separate the mesh into per material mesh, but this might create some compund mesh colliders and im not sure how good that would be.

[robertlong]

From the July 21st glTF Meeting:

• OMI_collider should likely not define a default behavior for physics materials. It would be up to the runtime to define it.
• However when the OMI_physics_body/OMI_physics_material extension is present there would be a default physics material that runtimes should use for all static bodies.
• There's still an outstanding question if OMI_physics_body and OMI_physics_material should be split
• There's still an outstanding question on how hulls should be specified
  • @robertlong and @fire are both in support of including V-HACD as a minimum required algorithm for hull decomposition
  • @lyuma notes that there still should be limits as to the maximum hull complexity. @robertlong and @fire also note that there may need to be limits on trimeshes but it's hard to set a limit since it is more about vertex density and relative collider size not as much about trimesh vertex count.

[robertlong]

Another outstanding question: How should a two different colliders under a single rigid body (compound collider) behave if the colliders have different physics materials?

[robertlong]

We should note the implementation detail that if using a mesh collision shape where the mesh has multiple primitives, all the primitives should be used in the resulting collider.

[robertlong]

During today's OMI glTF meeting @aaronfranke proposed renaming extents to size for box colliders. And making height for capsule colliders represent the full height of the collider, not subtracting the diameter of the capsule. There seemed to be consensus on this.

[robertlong]

@aaronfranke also suggested adding the cylinder collider type. It's not supported in all engines but could be approximated with a box collider, capsule collider, hull, or mesh collider.

[robertlong]

We should move from storing collider definitions on the node to storing them on a root level extension and storing an index to the collider on each node. Very similar to KHR_lights_punctual or KHR_audio etc.

[robertlong]

Here are our implementations of OMI_collider so far:
https://github.com/matrix-org/thirdroom-unity-exporter/blob/main/Runtime/Scripts/OMI_collider/OMI_ColliderExtension.cs
https://github.com/matrix-org/thirdroom/blob/main/src/asset-pipeline/extensions/OMIColliderExtension.ts
https://github.com/matrix-org/thirdroom/blob/main/src/engine/gltf/OMI_collider.ts

[robertlong]

Let's close this PR and continue in #118

At the next meeting lets make sure all of the feedback captured here is applied to the new PR.