The face is the most expressive aspect of the body. Through the most subtle changes in our face we convey what we are thinking, how we are feeling, and what is going on within us whether we are talking or not. A slight eye twinge can convey that we are hesitant. A slight lip curl can convey that we are being sarcastic. These subtle changes are what really make us feel reel. It is also in their absence that we move toward things feeling dead, wooden, or just creepy as we will discuss in a bit.

Up until this point in animation we have largely ignored the face. In our exploration of body mechanics we had to focus our efforts on ensuring the body moved realistically and beleivably. The face is incredibly complicated and can quickly introduce a mountain of complexity very quickly. When we interact with people the face is what we largely interact with. Some of it is mechanics in order to make the proper sounds come out through speach. Some of it is to convey emotional modifications to our speach or even non-verbal queues. How do we even start to think about how to approach this in our animations? I hope this module will be a good start to that, but mostly it is through exploration, experimentation and observation that you will really start to get it.

To start off on facial animation please read through this blog on Animation Mentor that I felt to be a great overview to facial animation.

Facial Animation 101

Earlier I mentioned that the absence of subtlties in facial posing and animation can lead to dead, wooden or creepy outcomes. This creepiness is actually a well documented scientific observation and study called the uncanny valley. To quote wikipedia:

In aesthetics, the uncanny valley is the hypothesis that human replicas that appear almost, but not exactly, like real human beings elicit feelings of eeriness and revulsion among some observers.[2] Valley denotes a dip in the human observer's affinity for the replica, a relation that otherwise increases with the replica's human likeness.[3] Examples can be found in robotics, 3D computer animations, and life-like dolls among others. Read More

Notice the dip at the right side of the graph. This is the valley we are talking about. It is the point as characters become more lifelike that our demand on them actually being lifelike increases dramatically and therefore becomes more disasterous if we don't get it right.

This is why animated movies that aren't hyper realistic can look so much better and less weird than those that are. Think about the most realistic CG characters in visual effects. Which ones maybe come to mind? Gollum from Lord of the rings. He is human like, but different enough to not hit the uncanny valley. Take a look at this list of the top ten most realistic CGI characters in movies. Many of them are human-like characters, but none (except Grendel's Mother from Beowulf) is actually human. For a good example of the uncanny valley just watch The Polar Express. While it is a technological feat to be sure and the visuals overall are incredibly stunning, the renders of the people are so real that their inhuman movements are incredibly unnerving. Compare the motion of these characters (largely motion captured) with the list of the top 10 best above and see how it works well when the characters aren't human. Now you know why.

Technology is obviously improving and there is a real liklihood that we will be able to overcome this problem with time and technological/artistic advancements, but we aren't there yet. However, what does this have to do with facial animation? I am glad you asked. The advent of machine learning and the generative AI technologies is making this more and more likely every day. However, this has to do with facial animation because even if you don't have a realistic looking character you need to add the life into the face or the same effect can happen. Maybe it won't be unnerving, but the performance will definitely feel dead or wooden which is bad. In other words, the small details matter.

Let's get down to the technical side of the face. Speech!

The first job of the muscles and movable parts of the face when dealing with dialog is to move in such a way as to make the sounds come out in the right order and pattern to be heard as speech. While it is technically possible to have your mouth make certain sounds that otherwise don't match including the ability to speak without your lips moving your goal as an animator is to ensure the mouth moves correctly to produce the sounds we would expect to pre-recorded dialogue from a real person. While you can look at video reference and copy the movements and timing represented there are more efficient ways to prepare for this animation and approach it from a technical perspective. The two most common approaches are through phonemes and visimes.

Phonemes

Phonemes are the smallest individual units of sound that are combined to form speech. A phoneme shape refers to the position of the lips, teeth, tongue, and jaw required to utter a phoneme, or a specific sound of speech.

For example, the phoneme shape for the phoneme /m/ consists of lips that are pressed together, whereas the phoneme shape for /l/ consists of lips that are not round and that are slightly apart, with the tip of the tongue pressed lightly to the gums behind the upper teeth. -Reference

In other words phonemes are the various shapes your mouth takes in order to make the sounds for individual parts of speech. Here is a chart from the same reference above to describe what that could look like:

Did you notice that a lot of these shapes are really similar to others? Wouldn't it be prudent to combine these shapes into those that are close so you don't have to create so many different poses? The answer is yes and thus the use of what is known as visimes.

Visimes

Visimes are mouth poses that represent the main shapes the mouth takes throughout most of speech. This encompasses all of the phonemes and is designed to focus the facial pose on just the general shape rather than the specific shape required to hit every syllable.

The Cryengine (Yes the super fancy video game engine) documentation has a pretty good explanation of visimes. You should read it.

Another look at visimes and phonetics

The important thing to remember with facial animation is to look at the overall shape the face makes as the speech flows from word to word. Not every word is given the same inflection and exaggeration of shape and not every syllable is posed exactly as it is represented both with phonemes or visimes.

Remember, the human body wants to be as lazy as it can be. If it can get away with not having to hit every pose perfect and still produce the right sound then it will. Focus on the overall main shapes hit in the shot and you'll be a long way there.

Dialogue with Phonemes

Above all you need to remember that the principles of animation apply to the face just as much as they do to body motion. Every one of the principles needs to be a focus as you animate the face in order to ensure that quality motion and more importantly a quality performance is produced. Chesterton Sampson - 10 Tips for facial Animation

First, a quick review of how to setup a new maya project.

Next you are going to want to do your best to download and install the malcolm 2.0 rig provided by AnimSchool. This rig is awesome! and the plugin they provide to work with the rig is really awesome, but it can take some work to get it installed. I have provided an installation video below and the AnimSchool also has videos for installation too. I had some trouble installing in Maya 2024, so I will be using maya 2023 for this class. Hopefully you have better luck, but you can go and install the older version of maya from your license as well if you need.

This weeks assignment is pretty simple. You are going to get a bit of practice posing the Malcolm Rig's face. Please produce 2 poses of the character that evoke a strong emotion. You can use photo reference for comparison like this:

For each facial pose please take a screen shot or a still frame render in either .jpg or .png format and submit them to Assignment 2 on UNM Canvas before midnight next Friday.