Multiple active animations targeting the same properties?

[Alloyed]

The gltf spec seems deliberately sparse on how animations should be played back at runtime, but for now I'm assuming that animations can be grouped into clips, or not, and that it's worth having the ability to play multiple animations at the same time.

This brings up the question: how should my implementation handle the case where multiple animations attempt to animate the same node, or, in the case of #1301, the same arbitrary property? A user exporting clips that represent full animations, (eg idle/walk/run) will probably want to reuse the same set of nodes, and just provide different animation data to each node. Because I am supporting multiple animations at once, then, it's very possible that multiple clips would attempt to provide keyframe data to the same node at the same time.

[donmccurdy]

but for now I'm assuming that animations can be grouped into clips, or not...

If by "clip" you mean a distinct action, like idle/walk/run, then the spec does recommend that grouping, per the implementation note here. The meaning of an action is subjective and not consistent in all 3D animation tools, so perhaps that's not enforceable, but if there is a meaningful grouping possible it should really be used.

...how should my implementation handle the case where multiple animations attempt to animate the same node, or, in the case of #1301, the same arbitrary property?

Playback behavior is intentionally left out of the file format's scope, and will likely be dictated by the engine you're using. But if you were writing an implementation yourself, I would recommend understanding Unity's Mecanim system and perhaps the three.js animation system (source code available).

In either, there are concepts of transitions or blend trees. An animation could be "playing" but only partially applied, using some fractional weight. A common example would be transitioning from "walk" to "idle" state, by ramping each animation's weight from 100%–>0% and 0%–>100% over the course of ~500ms. Playing both animations at 100% weight would, in any implementation I'm aware of, give undesirable results.

[donmccurdy]

Here's a simple example: https://threejs.org/examples/#webgl_animation_skinning_morph

[lexaknyazev]

Animations that always play simultaneously should be written as a single animation object with multiple channels. In such a case different channels of the animation cannot have the same targets.

Well-defined support for playing several potentially conflicting animations at the same time sounds like useful feature to be added via a new extension.

[donmccurdy]

I don't think the engines I've contributed to (three.js, aframe) would be able to implement such an extension. For threejs and aframe this would mean taking over application control from the user, which we'd prefer not to do. Blending animations is a problem that users writing an engine from scratch will need to make decisions about, but is typically well-handled by mature engines, and glTF's leaving that out of scope is (IMO) pretty reasonable.

But certainly any application could do a vendor extension for their own needs, and see what ecosystem support evolves from that.

[prideout]

It's strange that an industry standard would relegate this issue to an "implementation note" or a "best practice". Could we simply add a sentence like this:

"An implementation must not apply more than one animation simultaneously."

This makes the conformance requirements clear, and future extensions would be free to loosen this restriction.

[donmccurdy]

For the purposes of animation, a standalone model viewer — playing one animation at a time — is a relatively minor concern in my opinion compared to VR, AR, film, and game applications that use glTF models and animations as the building blocks for complex behavior. We cannot prohibit playing more than one animation simultaneously without negatively affecting many users. Moreover, "don't play A and B together" is not a requirement that a file loader can reasonably pass along to a game engine or user-defined application.

I'm not sure what we can specify without really restricting engines' ability to compose behaviors, in the same way we expect engines to compose scenes from many files (glTF, FBX, or otherwise) flexibly. Consider:

1. Animations A and B might affect entirely different objects, and be reasonably played simultaneously.
2. Animations C, D, and E might be different skeleton states for character X, and intended to be played separately, but with blend trees handling the transition between animations.
3. Animation F might control a morph target affecting character X, which can be played regardless of C, D, and E.

This quickly becomes complicated, and none of that is information a DCC tool can provide. Simply getting a flat list of animations out of Blender is surprisingly difficult.

In summary, I think we've specified as much as can safely be specified with today's tools. For comparison, typical workflows with FBX often involve (a) writing each animation to an entirely separate file, or (b) writing a single animation, which the user slices into separate animations using keyframe offsets after loading.

[emackey]

Agreed with @donmccurdy. Among the glTF sample models we now have examples of both cases:

• The InterpolationTest model uses individual animations to demonstrate different interpolation methods, that can be run individually, in groups, or all at once. A more real-world example could be a vehicle or machine with multiple moving parts. One animation could make the wheels turn, another could run the windshield wipers, and these can be controlled independently. We wouldn't want the spec to state that you can't run the wipers while the wheels are rolling.

• The Fox model shows multiple animations targeting the same joints. In this case it would seem to make sense that only one of these animations should play at a time, as there are no other moving parts on this fox.

It's tempting to make the spec say something about not playing multiple animations that target the same nodes or joints. But glTF offers only storage for animations, the choice of when to play any of the available animations is considered an application-specific concern. The needs are different on a per-application (not just per-framework) basis. So it may be best for the glTF spec to stay hands-off on that.

[prideout]

Okay, sounds good.

[emackey]

Closing for now. Let me know if more discussion is needed here.