Assignment 3: Character Animation

Chris Tralie

Overview

The purpose of this assignment is to create a "thin slice" through a character animation pipeline completely from scratch. Students will implement the FABRIK algorithm to reorient sequences of joints to reach towards targets. They will then implement a simple rigging/skinning algorithm to move a surrounding triangle mesh of Homer Simpson along with the bones.

What To Submit

When you're finished, submit skin.js and skindebugplots.js to canvas.

If you're skinning your own mesh for the art contest, you should also submit the .off file for that mesh, as well as the .json file for the skeleton and binding weights (see this video and this video).

Learning Goals

  • Manipulate dynamic objects in Javascript
  • Implement the FABRIK algorithm
  • Implement skinning
  • Create coordinate frames to describe the orientation/position of objects

Getting Started

Click here to download the starter code for this assignment. The entry point for running things is the file skin.html. You'll be editing the files skin.js and skindebugplots.js

You can start by editing code in the main method of skin.js. Right now, the code plots the joints as a point cloud in the debugging area at the bottom of skin.html, using the plotJoints method in skindebugplots.js, as shown below:

This code uses the plotly library. You should study this example closely, because it will help you to debug as you go along.

I've provided a method plotSkin for another example. If you uncomment the line plotSkin(X); in main, it will plot the dense point cloud of all of the skin vertices on the homer mesh colored from front to back, as shown below

Eventually, you'll update the full surface mesh of homer by providing specific information to the mesh class:

but you should plot a lot of stuff in the debugging area as you go along to check incrementally to make sure each step is working as you expect (some tasks will require you to make methods to do this).

Part 1: Data Structure Initialization And FABRIK

For the first part, you'll implement the FABRIK algorithm that we discussed in class, treating leaf nodes of a skeleton tree as end effectors. Below is a video where I explain the FABRIK algorithm:

Task 1: Plot Skeleton Tree (10 Points)

I've provided a skeleton for the homer mesh that I made in blender in a file called rigs/homerskel.json, which is passed into the main via an object called skeleton. This has all of the information to build the skeleton.

Your Task

Create a method called plotSkeleton in skindebugplots.js that takes skeleton as a parameter and plots all of the bones. Below is an example of what such a plot might look like:

Hints/Suggestions

Study plotJoints to see how I plotted a scatter point of points using plotly. I show there how to plot points. To plot a line segment between points (x1, y1, z1) and (x2, y2, z2), you can use code like this:

Task 2: Create And Plot Chains (8 Points)

We're now going to do some preprocessing to identify chains that we can do FABRIK on

Your Task

Create a method getChain(skeleton, string) in skin.js that returns a list of joints, starting an end effector specified by a string, and ending at a subbase, or a joint with more than one child.

Also, for debugging, add a chain parameter to your plotSkeleton method in skindebugplots.js that draws chain in red on top of the skeleton, if it's provided. Below are a few examples:

getChain(skeleton, "Wrist.R") getChain(skeleton, "Ankle.L") getChain(skeleton, "Head")

Hints/Suggestions

  • It's probably helpful if you loop through the skeleton and add a parent field to each joint to make it easier to walk backwards.
  • To access the value associated to key in an object obj, you can say obj[key]

FABRIK (10 Points)

We're now ready to implement FABRIK to reach a limb towards a target!

Your Task

Create a method fabrik(chain, target, nIters) in skin.js which takes as a parameter a chain, as well as a target vec3 and a number of iterations to go through. The method should update the appropriate positions in the skeleton as a side effect.

Hints/Suggestions

  • It's probably easier to make a single method for both forward and backward iterations, and to pass it the reverse of the chain for the backwards iterations
  • You'll have to be very careful with references here. For instance, if you make the target the position of the subbase, you'll need to make a deep copy that position.

Wrist.R

Moved to a point 0.2 units above where it currently is

Ankle.L

Moved to a point 0.2 units to the right and 0.1 units above where it currently is (this is anatomically questionable...)

Part 2: Skinning And Animation

In this part, you will make the "skin" (mesh vertices) move along with the bones

Bones And Local Coordinate Systems (10 Points)

First, we're going to setup coordinate systems for each bone. These coordinates systems will change as the bones move around, and we'll eventually get the skin to move with them.

Your Task

Create a class for bones and initialize all the bones in the skeleton in a list of bones; there should be a bone between a joint and each of its children. The code below could get you started, but you'll also need to make a recursive method to initialize all of the bones in a list

During initialization, set up and store a glMatrix.mat4 matrix that transforms each bone from bone coordinates to world coordinates. To do this, let a and b be the positions of the the two joints on either end of the bone, and let wLast be the vector that comes into this bone from its parent bone. Then, create the following four vectors

  • c: The center of the bone (the average of a and b)
  • w: A normalized vector from a to b
  • v: The normalized perpendicular projection of wLast onto w
  • u: The cross product v x w

as shown in the picture below:

The transformation matrix T from bone coordinates to world coordinates should then be

\[ T = \left[ \begin{array}{cccc} u_x & v_x & w_x & c_x \\ u_y & v_y & w_y & c_y \\ u_z & v_z & w_z & c_z \\ 0 & 0 & 0 & 1 \end{array} \right] \]

Store c, w, u, v, and T as an instance variable for further use.

To check to make sure you've done this correctly, plot the coordinate systems centered at each point c. Below is what this looks like if you plot the w axis in blue, the v axis in green, and u axis in red, each with a length of 0.05:

Hints

  • The easiest way to implement this is with recursion starting from the root and branching out, passing along w as wLast to a child bone. You can make wLast be (0, 1, 0) at the root.
  • If you use the mat4.fromValues method, be mindful that it accepts parameters in column major order. This means that your first 5 elements should be ux, uy, uz, 0, vx ...

Setup Rig: Local Coordinates of Skin And Weights (10 Points)

You're now going to send the engine I made all of the information it needs to bind the skin to the bones. Before you do this, it might be helpful to watch this video about how I created the skeleton object in Blender:

Optionally, you might also want to watch this video about how I made the skeleton in the first place, especially if you want to skin your own mesh for the art contest.

TL;DR, the important thing to know is that the weights of the bones are stored in the joint objects in skeleton, and each joint stores the weights of the bone that has this joint as a tail (that is, the child of another bone). For example, the weights stored in Elbow.R correspond to vertices bound to the bone from Shoulder.R to Elbow.R:

The weights are a dictionary/object whose keys are indices int he mesh that are bound to this bone, and whose values are the weights of this bone. You can loop through them with a loop like this:

Your Task

First, gather the transformations of all of the bones in a list called transformations. Then, gather the coordinates of the numPoints points in the mesh:

Now, gather information about the top 3 weight bones for each vertex in the mesh. To do this, create three parallel lists:

  1. weights: Each element is a list that has numPoints weights, which are parallel to the mesh vertices X. The first list holds the top weights, the second list holds the second largest weights, and the third list holds the third largest weights. For example, weights[1][100] holds the second largest weight for the mesh vertex at index 100.

    Once you've determined what the three largest weights are for each vertex, you should normalize the weights for that vertex; that is, divide each weight for each vertex by the sum of the three weights for that vertex.

  2. boneIDs: Each element is a list that has numPoints indices into the bones, using indices corresponding to the order of the bones in your transformations list. These lists are parallel to weights. For instance, if the largest weight for vertex at index 476 is associated to the bone whose transform is at index 10, then bones[0][476] would be 10.

    As another example, if the third largest weight of vertex 123 is associated to the bone whose transform is at index 5, then bones[2][123] would be 5.

  3. Ys: Each element is a list that has nPoints points in local coordinates of the bones they're associated to. So, for instance, if the top weight bone for vertex 2024 is at index 6 in transforms, then Ys[0][2024] would contain X[2024] transformed by the inverse of transforms[6], since we're pulling the coordinates back from world coordinates to bone coordinates.

NOTE: If there are fewer than 3 bones bound to a particular vertex, then pad the weights, bone indices, and Y's with 0's up to 3. So, for instance, if only one bone is bound to vertex 1337, then

  • weights[1][1337] = 0
  • weights[2][1337] = 0
  • boneIDs[1][1337] = 0
  • boneIDs[2][1337] = 0
  • Ys[1][1337] = [0, 0, 0]
  • Ys[2][1337] = [0, 0, 0]

Once you have all of this data, pass it along to the mesh and plot the mesh with the following code:

If you've done everything properly, homer should show up in his original pose!

Hints/Suggestions

  • The easiest way to start is to fill in weights and boneIDs first, which you can do with a loop that goes through all of the bones, with a nested loop that goes through all of the weights of that bone. Then, you can normalize the weights, and you can apply the transforms to fill in Ys based on boneIDs.

  • If stuff isn't showing up right, it will be helpful to debug visually. For example, if you say Then you should get the following plots

    Ys[0]

    Ys[1]

    Ys[2]
    The plots look strange since we're in local coordinates.

Updating Transforms (10 Points)

Now that you have the rig loaded, you can move the bones around by applying FABRIK and changing the positions of the joints. To do this, you'll need to update the transformation matrices of each bone based on how its joints have moved.

Your Task

Implement a method updateFrame in the bone class that updates the transformation matrix based on the last coordinate frame (ubefore, vbefore, and wbefore) and the current position of the bone. To do this, first move c to the new center. Then, update w, and compare it to wbefore to determine an axis and an angle of rotation. You can determine the axis with the cross product wbefore x w, and you can determine the angle θ between them as you did in homework 1. Finally, use this axis and angle to rotate u and v. The picture below shows how this would be used to rotate u

Once this is finished, send an updated list of all transformations to the mesh.updateBoneTransforms method.

As an example, suppose you move the right wrist to the left by 0.8 units and up by 0.6 units:

Then I should get this:

As another example, suppose I move the left ankle forward in z by 0.4 and up in y by 0.2. Then I should get this:

Hints/Suggestions

  • Feel free to use the glMatrix.quat quaternion class to help you with axis/angle rotation, or use our code from class

Animation Art Contest (5 Points)

Finally, create an animation with at least two joint chains moving. The example below shows the right wrist changing as

\[ x(t) = x_0 - 0.5(0.5 + 0.5 \cos(t)) \]

\[ y(t) = y_0 + 0.5 + 0.5 \cos(t) \]

And the head changing as

\[ x(t) = x_0 + 0.1(0.5 + 0.5 \cos(t)) \]

As you can see, it's far from perfect, but not bad for doing this completely from scratch!

The final code you hand in should show the animation running, which proves that you've completed all of the tasks

To help get you started with the animation, here is a code snippet you can use that keeps track of elapsed time in an animation loop:

Other Ideas (For The Bored)

There's lots of directions this can go!