Skip to content

Action Unit to Shape Keys

Jump to bottom
Jambon edited this page Oct 9, 2023 · 1 revision

The main problem ACT is solving is the control of an avatar using Open Face's Action Units.

Shape Keys

We have been adding some modification to Lou's model. One of them is Shape Keys.

"Shape keys are used to deform objects into new shapes for animation. In other terminology, shape keys may be called “morph targets” or “blend shapes”. The most popular use cases for shape keys are in character facial animation and in tweaking and refining a skeletal rig. They are particularly useful for modeling organic soft parts and muscles where there is a need for more control over the resulting shape than what can be achieved with combination of rotation and scale." from the Blender Documentation

Blender Doc Image

We had a total of 82 Shape Keys to the model.

Action Units

Then, we use Open Face's action unit system to track the different expressions and we plug them into **ACT **with a conversion algorithm inspired by FACSvatar, made by Stef, to make Lou reproduce the same expression as in Open Face.

Conversion Algorithm

The algorithm works this way :

  1. Gathering the information, using Open Face, for the current frame
  2. Cleaning the data to save the Pose_* and AU**_R values
  3. Resetting the Shape Key values to 0
  4. Iterating through all AUs, we multiply the value by the conversion rate of the concerned Shape Key. Example, AU_2 corresponds to Shape Key 3 * 0.8 and Shape Key 14 * 0.1.
  5. Adding the obtained value to the concerned shape key.
  6. Finishing the iteration, we change the avatar's Shape Key values, called "Blend Shapes" in Unity
  7. Changing to the next frame

If you want to see the conversion rate of each action unit, browse through the files to Unity Side\Assets\Data\JSON\Action Units to Blend Shapes.

Clone this wiki locally