Making the dream Animation API for GDExtension

[GuilhermeGSousa]

So right now, there are no good ways to making motion matching as a GDExtension. There are some features missing that prevent us from extending certain engine functionalities. In this post I'll go into detail of what would be required on the engine side to do this. For each point I'll go over what we're trying to do and what the ideal engine API would look like to be able to support it.

Motion Matching Animation Node

What we want to do

From implementing it in an engine module, its has become clear that the best way of implementing motion matching in Godot is as an AnimationNode. Doing so comes with multiple advantages compared to other approaches we've tried:

• Blend the result of motion matching with other animations
• Use state machines to switch between motion matching and regular animation techniques
• Run motion matching queries on smaller animation libraries, and switch between them using state machines.

What we need from Godot

The _process method, arguments and return values

Probably the most important part of this section, to be able to extend a AnimationRootNode, both AnimationMixer::PlaybackInfo and AnimationNode::NodeTimeInfo need to be ref counted, so that the following function can have a GDVIRTUAL equivalent:

NodeTimeInfo _process(const PlaybackInfo p_playback_info, bool p_test_only = false);

Accessing the AnimationTree

During _process, AnimationNode::get_animation_tree() can be used to access the AnimationTree that owns the node. We'd need this same function exposed to extensions to be able to create a Motion Matching animation node.

At editor time (outside the _process function), that same method will always return null. In order to be able to implement a similar property validator as the one used by AnimationNodeAnimation, we'd need a way of accessing the animation tree from _validate_property. From a module, this can be done the following way:

#ifdef TOOLS_ENABLED
AnimationTree* tree = AnimationTreeEditor::get_singleton()->get_animation_tree();
#endif

We'd need that same API exposed to gd extensions as well.

Inertialization Blending Skeleton Modifier

What we want to do

Inertialization blending is a technique used to blend animations together, developed by the teams at Microsoft Studios for Gears of War 4. One of its characteristics is that the rate of change of the bones preserve some of their inertia when swapping animations, resulting in more fluid and realistic motion. It can also be more performant than "regular" track blending, as it is not required to evaluate two tracks simultaneously to compute the resulting blend.

Since this blending technique requires modifying the skeleton bone's position and rotations, implementing this as a SkeletonModifier3D is ideal. However, there are still a couple of missing pieces to the puzzle.

What we need from Godot

Signals from AnimationNodes

The inertialization blending algorithm needs to run specific calculations when animations change (i.e. when motion matching run a query and selects an animation). We therefore need a way for animation nodes to be able to signal outside systems when something happens. An idea of how to do it would be to have a similar method as AnimationNode::get_parameter_list that would expose signals instead:

Array[Dictionary] get_signal_list() const

(similar to the method with the same name of Object).

[fire]

Upcoming work that'll allow compiling godot-cpp gdextensions as a module. 💯

[TokageItLab]

If you are referring to the fact that in inertial blending, some transforms cannot be generated by interpolation of animation key values alone, then that is also probably beyond the role of AnimationNode/AnimationMixer.

AnimationNode only changes the time and the amount of blending. Also, the AnimationMixer only generates values by blending the animation key values.

Perhaps the hack of generating a dummy animation every frame using the AnimationProvider approach (https://github.com/Malcolmnixon/AnimationProviderDemo & https://github.com/Malcolmnixon/GodotXRAnimationRecorder) might be informative, but it raises performance concerns (And for this reason, finally XRAnimation was implemented as a SkeletonModifier).

[GuilhermeGSousa]

Generating a dummy animation is interesting, I will look into those implementations. The idea I initially had was:

• MMAnimationNode would switch from animation A to B without any blending.
• At the time of this switch, an inertial blending skeleton modifier would adjust bone position to maintain inertia and smoothly transition from animation A to B. The only thing preventing us from doing this right now is that the inertial blending skeleton modifier has no way of knowing when the MMAnimationNode switched animations (at least with the current API)

[momoluna444]

Maybe we could have an AnimationNode called “Cached Animation” that automatically caches the last N frames of the input subtree. Users can inherit this node and do whatever they want with the cached animation (motion matching, custom blending, IK, etc.). Since it's basically just a piece of temporary animation, AnimationMixer can mix it as usual. But I'm not sure how caching node subtree should be implemented, and the performance implications of editing animation frames at runtime. @TokageItLab what do you think?

[TokageItLab]

@tele4none As I have explained many times in many places, AnimationNode does not have values that can be applied directly to objects. It is the AnimationMixer or SkeletonModifier that applies values to objects, not the AnimationNode. So it is beyond the role of an AnimationNode.

All an AnimationNode can have is a reference to a specific Animation, which frame of that Animation to select, and the amount of blending.

In other words, as mentioned above, if you generate dummy animations in NodeAnimation, you can pass arbitrary values to AnimationMixer by make the AnimationNode refer that. However, editing the Animation key while the AnimationMixer is processing is not recommended currently from a safety or performance standpoint. Well, it would be possibly beneficial to consider an API specialized for dummy animations that solves these problems.

[momoluna444]

Yes, I see what you mean, and that's exactly how the idea came up while reading your previous comment.

Let me describe it in pseudocode:

NodeTimeInfo AnimationNodeCachedAnimation::_process(...) {
tree->cache_input_to_animation(animation); // push current "Pose" to animation
...
user_animation_modifier(animation);
...
blend_animation(animation);
return nti;
}

The caching part happens in AnimationTree, and the CachedAnimation node behaves like an AnimationNodeAnimation.

I also like the idea of AnimationMixerResultMixer. Anyway, it would be great to be able to blend and modify animations freely.