A muscle based face rig

Innovations report

Jan Erik Olsen d1155694 BACVA3

Contents

Abstract Introduction Facial anatomy The rigging process - About cMuscle System - Building the rig - Jaw / Chin - The Nodes - The Lips - Nasolabial furrow, mouth angle furrow and the cheeks - Eyes and eyebrows - Interface Critical analysis Conclusion Images used in report References

Abstract
The report focuses on the creation of a facial rig based on a muscle system. A substantial part of it will discuss the features of the cMuscle System by Comet Digital, LLC.

Introduction
The human face must be one of the most challenging subjects to replicate in arts. This is because the face and the expressions it makes is such an important part of our communication with each other. Understanding what other people feel is highly important to us, so as a result we are able to detect very minute changes in a persons face. This also means that artists need to be very skilled to reproduce a face and the desired facial expressions. In computer graphics, most facial animation we see today is either based on shape animation or on joints. This can produce superb results, but many times it can also look a bit stiff. In those cases, a hint of jiggle and inertia could make the animation look more natural. The use of a muscle system could be one way of improving this, and this project aims at finding ways of using the features of one particular muscle system, the cMuscle system by Comet Digital, LLC, in a facial rig. For this project, I will use a model of a head that has been modelled for my major project.

Facial anatomy
In order to be able to rig a realistic cg human head it is necessary to understand the anatomy of a real life head. It is important to understand the how the muscles act in a face, but the study of muscles can also lead the modeller off the right track. On most of our body we have skeletal muscles, which are used for moving joints. Skeletal muscles roughly produce two visible results: The moving of a joint, and the changes of volume and firmness under the skin. For example the hamstrings can move the lower leg, and we can also see if the muscle is flexed or relaxed under the skin. Most facial muscles however, produce a quite different result. Most of them originate from bone on the scull, and insert into other facial muscles. Some connect directly into the skin, while others join up in nodes which are connected to the skin. The visible results from facial muscles, i.e. facial expressions are seen mostly as the skin is pulled in different directions. The flexing and relaxing of the muscles in the face is much less visible than for example the skeletal muscles on an arm. So to produce facial expressions in cg, it makes sense to concentrate on the wrinkles and folds of the skin and where facial muscles are connected to the skin rather than the muscles themselves.

The rigging process

The rigging or set up stage basically means to prepare a cg model for the animation process. For an animator to be able to do his or her job properly, the rig must provide good deformation and have an intuitive interface. There are many ways of deforming cg geometry, and as a result there are several approaches to rigging with their different strengths and weaknesses. Most rigs use a combination of several methods. As mentioned, this facial rig is based on a commercial muscle system plug in for Maya called cMuscle System by Comet Digital, LLC.

About cMuscle System:

The cMuscle System is a plug-in suite for Maya. Its not my intention to give a detailed explanation on how the system works, but I will outline the main features to help the reader to follow the rigging process. For detailed information about the cMuscle System, refer to: http://www.cometdigital.com/cMuscleSystem.php In my second year Specialist Project, I used the cMuscle System to build a rig with leg muscles. The cMuscle System uses a custom skinning solution that enables the objects supported by the plug-in to interact with the skinned object. The most important of these objects are called capsules and muscles. The capsules are in essence Maya locators with a length and a radius, and act similarly to regular Maya joints. These are rigid objects that can interact with the skinned object and are used for muscle attachment. (Fig 1)

Fig 1

A basic muscle attached to two capsules.

Fig 2

Moving the capsule causes the muscle to bulge.

The muscles are in short regular NURBS surfaces controlled by a cMuscle deformer. The deformer uses a hermite type spline to control the squashing and stretching of the muscle. (Fig 2) The muscle can also be deformed by moving the spline control points, giving more possible shapes for the muscles. (Fig 3)

Fig 3

Moving the control points of the hermite curve.

When a muscle is added to the system, it can interact with the skin in different ways. One way is to paint sticky weights. This means that the skin will move with the muscle as if it was glued to it. Another method is to pain sliding weights. This will cause the skin to only be moved outwards from muscle, The effect looks like the muscle is sliding under the skin. A very useful feature of the muscles is their ability to jiggle. This can help to make the rig appear more lifelike, simulating the inertia of the face.

Building the rig

The most interesting part, and also the most time consuming, has been to figure out how to make use of this muscle system in the face rig. As mentioned, to replicate real life facial muscles would only make sense in special cases. Instead I looked at the different qualities of the muscle objects, and how they could deform the face model. As a basis for the face rig, capsules are parented together to form a base root, three neck vertebras, a head capsule and a jaw. The skin of the model is weighted to the corresponding capsules by painting weights. At this stage, the set up is just like a basic joint driven rig, Muscles and other objects are later connected to it to add detail to the deformation. In the sections below, I will discuss my solutions to placement of muscles. These do not cover every possible face expression, but some of the issues where the muscle system can be helpful.

Fig 4

The polygon head with a basic capsule rig.

Jaw / Chin
One common problem with a basic joint based cg rig is that theres nothing to stop it from colliding with itself. This causes it to loose volume in certain areas, and the skin under the jaw is a typical example. As we open our jaw and/or bend our heads downwards, the space between the throat and chin will be smaller. This causes the skin, fat and muscles under the chin to be forced outwards and bulge. After looking at several ways of positioning and controlling muscles, I decided to place a muscle with origin under the chin and insertion in the upper parts of the throat area. The capsule at the origin is parent constrained to the jaw capsule, and the insertion capsule is parented to the third (top) neck vertebrae. As the jaw opens and closes, the length between origin and insertion will vary which is measured and used to control the squashing and stretching of the muscle. This worked nicely for the area directly under the chin, but the deformations on the side areas were not ideal. To fix this, I added two more muscles running along each side under the jaw. With this, I had better control of the bulging under the chin and jaw.

Fig 5

Three muscles acting on the skin as the head bends down.

Fig 6

The resulting deformation of the skin from the muscles under the jaw.

Another possible way of controlling this area would be to place a muscle like a band under the chin. This would have the advantage of using just one muscle, which means its easier to position and paint weights. Still, it would be harder to drive the bulging and flexing of the muscle based on the jaw and head movement since it would not directly cause any change in the length of its control spline.

The Nodes
A very important feature in facial anatomy is the nodes. The nodes are located just above the angle of the mouth, one on each side and acts as a central connection point for several muscles (nine in total). The most noticeable function they have is to allow muscles to pull the corners of the mouth to the sides, up and down from this point.

Fig 7

The nodes are highlighted in red.

Illustration from Human Anatomy for artists, Eliot Goldfinger, Oxford University Press, 1991.

The functionality of the nodes can be seen as a good example of that a muscle based cg rig does not have to be nearly as complicated as the real thing. It would not make much sense to rig eighteen muscles just to move two nodes. Most of them do not influence the skin directly, but the results of their actions are visible on the skin as they pull on the nodes. The most visible exception is the orbicularis oris which is the muscle shaping the lips. As I recognized the importance of the nodes in a real face, I decided to mimic this configuration in the cg rig. I thought that many aspects of a face expression could be driven by the positions of the nodes, and that it would in turn make the animation process easier. In the rig, the nodes are represented by capsules. Although they do not directly influence the skin, they are connected to most of the cg facial muscles which are in turn deforming the skin. In the illustration (Fig 8) we can see the nodes and the cg muscles they are connected to. These muscles and the nature of the connection will be explained in detail in later sections of the report. Note that these are not necessarily direct representations of anatomical muscles.

Fig 8

The cg nodes are highlighted in red. The connected cg muscles are called: 1: Upper lip, 2: Lower lip, 3: Mouth angle furrow, 4: Nasolabial furrow, 5: Cheek, 6: Cheek Fat.

The Lips
Real life lips are shaped by the complex muscle Orbicularis Oris. It can be divided into two major parts: The upper and the lower lip portions. These parts are furthermore divided into the outer and red lip portions. As a whole it runs all around the mouth. As we can see from the illustration (Fig 9), it is directly connected, or even a part of the nodes. The real muscle is able to produce a number of complex shapes, so to reproduce this in cg is a real challenge. I did a lot of experimenting with cg muscles before building the lips, and ended up with this conclusion: The best way of representing the red lip portions would be to place one cg muscle in the upper lip and one in the lower because this could be shaped very close to the lip geometry. However, this would only make sense if I could accurately control the muscles. Placing clusters on selected CVs on the NURBS muscle gave good local control, but moving them did not produce any jiggle. In contrast, moving the control points on the cMuscleSpline of the muscle does produce jiggle. The system also lets the user choose how many control points the muscle is controlled by. The control points can be rotated for even more fine control.

Fig 9

Orbicularis oris highlighted in red. The abbreviation oor (out) means orbicularis oris outer portion and oor (RL) means orbicularis oris red lip portion
Illustration from Human Anatomy for artists, Eliot Goldfinger, Oxford University Press, 1991.

Fig 10

The cg red lip portions seen from behind. Note the rotated control points.

In the final rig the red lip portions of the muscle is implemented with two cg muscles. (Fig 10) They have five control points, three of which are controlled directly from an animation interface. Moving and rotating the control points gives accurate control of the lips. A cluster is also added on two CVs on the bottom part of the upper lip to enable the animator to give it a flatter shape when needed. To control the region between the lips and the nose, three flat muscles was added. (Fig 11) Their lower endpoints are point constrained to the control points of the upper lip, making them expand and contract with the movements of the upper lip.

Fig 11

Frontal view: The red lip portions and three muscles representing the outer portion of the upper orbicularis oris.

Fig 12

The deformed cg lip.

Nasolabial furrow, mouth angle furrow and the cheeks

The nasolabial furrow is the skin crease that runs from the wing of the nose downwards and outwards and ends close to the node. It is a very distinct facial feature, and marks a clear border between the area around the mouth and the fat pads of the cheeks. (Fig 13) The mouth angle furrow is a short fold running downwards and outwards from the corners of the mouth. The tear bag is in short the tissue between the eyes and the cheeks. The area between the nasolabial furrow and the tear bag of the eyes is commonly called the cheek, and it holds a substantial amount of fat tissue. When we smile, the nodes and the lower parts of the nasolabial furrows are raised and pulled backwards leaving less space for the cheeks. The result is that the cheeks are compressed and bulge.

Fig 13

Red line: Nasolabial furrow Blue line: Mouth angle furrow Green line: Lower edge of the tear bag

For the cg model, I needed a way of reproducing the creases that appear as the nodes pull up, back or downwards. As I studied my own face, I noticed that the change in the nasolabial furrow, the mouth angle furrow and the cheek was caused by the position of the nodes. Also, that the creases seemed to appear because the nasolabial furrow and the cheek tissue traveled less distance than the nodes. To recreate the crease effect in cg, I placed long and thin muscles along the nasolabial furrow and the mouth angle furrow and connected their mid control points to the node with point constraints.

By also point constraining these control points to a non moving capsule (relative to the nodes), the movement of the control points could be weighted between moving 100% with the nodes, and not moving at all.

Fig 14

Default node position. The node is highlighted in red. Other muscles, 1: Upper lip, 2: Lower lip, 3: Mouth angle furrow, 4: Nasolabial furrow, 5: Cheek, 6: Cheek Fat.

Fig 15

Node is translated backwards and up. The node is highlighted in red. Other muscles, 1: Upper lip, 2: Lower lip, 3: Mouth angle furrow, 4: Nasolabial furrow, 5: Cheek, 6: Cheek Fat.

For the increasing volume in the cheeks as the node rises, I placed a large muscle (muscle nr 6 in fig 14 and 15) with origin under the eye and insertion at the node. The amount of translation in all axis received from the node to the midpoints of the mouth angle furrow (muscle nr 3) is 50%, the nasolabial furrow (muscle nr 4) receives 50% and the cheek fat (muscle nr 5) receives 40%. The cheek (muscle nr 6) receives 100% of the x- and yaxis translation, but 0% in the z-axis which is forwards and backwards. Disconnecting the zaxis fixed the problem of volume loss in the cheeks as the node translated backwards.

Eyes and eyebrows

The eyes and eyebrows are involved in most of the expressions we communicate. With the feeling that I should not limit the possibilities for the animator at the rigging stage, I made a set up with a lot of control left for the animator. With the set up for the lips in mind, the eyelids were rigged in a very similar way. (Fig 16) One muscle is used for each lid, and both muscles join up at the corners of the eye where they are connected to the same capsule. The muscles have five control points each. At the endpoints they are connected to static capsules, and the three in the middle are parented to capsules which are in turn parented to Maya joints. The joints are controlled from the interface using set driven keys.

Fig 16

The muscles used for the eyelids and the eyebrows.

Fig 17

To the left: The eyebrows and eyelids. Note the 6 control points in each brow. To the right: The deformed left brow.

For the eyebrow deformation, I made a muscle shaped as illustrated in Fig 16 and 17 with origin close to the middle of the forehead and insertion at the side of the eye. This shape developed from looking at how eyebrows move, what wrinkles they produce and where the muscles are connected to the skin. The bend from the origin to the more horizontal part of the muscle was chosen to make it easier to produce the shape the brows make when frowning. The horizontal part is roughly where the skin is attached to the muscles. This can be tested by frowning or raising ones brow, and with the fingers find what parts of it that can still be moved. The curvy part of the cg muscle at the insertion point is not indicating muscle attachment, but leads to the area where the brow influence fades. Thus it makes the skin in this area move in the right direction. The cg brow muscles have six control points which all receive direct control from an interface.

Interface
A cg model with good deformation is very important in the production of an animation, there also has to be an intuitive interface for the animator to use. It would be very cumbersome to move things around at the component level, so the components need to be connected to control objects.

Fig 18

Before looking at the control objects, Id like to name some important capsules in the rig. The numbers in fig 18 represent the following capsules: 1) 2) 3) 4) 5) 6) 7) Neck1 Neck2 Neck3 Head Jaw JawHinge Root

The Root capsule is the top parent of the whole rig. Selecting and moving this will move everything. The Head capsule is the parent of everything above the neck including the jaw and all the muscles. The Jaw capsule is parented to the JawHinge which in turn is parented to the Head. I will refer to the Jaw and JawHinge capsules simply as the Jaw in the following description.

Throughout the whole rig, I have used Parenting, parent constraints, point constraints and orient constraints in such a way that every control object and the objects they control will always go back to its default position when its values are set back to zero. This works even if for example the neck is twisted and bent.

Fig 19

The muscles to the left and the animation controllers to the right.

Fig 20

The controller objects numbered.

The controller objects are shown in Fig 20. The numbers represent the following objects: 1) Node controller: The node capsules are parent constrained to the node controllers. Translating and rotating this object moves the node capsules by the same amount. The nodes movements in turn control other objects as discussed previously. The node controllers are parent constrained to two objects called mc_left_node_parent and mc_right_node_parent. The reason for this set up is that the node parents receive translation to keep the corners of the mouth in the middle of the Head capsule and the tip of the Jaw. They are parent constrained to both the Jaw and the Head with 50% weight on each. Practically, this means that when the Jaw moves, the node parents move, the node controllers move, and the nodes themselves move. The advantage is that I can use the node controllers still have their original values and can be used to move the nodes from this new position.

2) Jaw controller: The Jaw is controlled by this object with set driven keys. Translation in the y-axis rotates the jaw, which is parented to the Head capsule, causing the chin to go up and down. Translation in x-axis and z-axis translates the jaw in the same axis respectively.

3) Lip controllers: The lip controllers consist of a cube and a circle. The cube translates the corresponding control point on the lip muscle spline curve via a parent constraint. The curve rotates the same points using an orient constraint. Having both rotation and translation on the cubes did not work properly with any of the available constraints or with parenting.

4) Upper lip raiser: This object is connected to a cluster on the upper lip with set driven keys. The cluster is the parent of two CVs under the lip. This enables the animator to make the upper lip flatter.

5 and 6) Eye lid controllers: These objects are opening and closing the eyelids by controlling the joints the eyelid muscles are connected to. Object nr 5 controls the lower lid, and nr 6 controls the upper. Translating these controls in y-axis rotates the connected joints through set driven keys.

7) Eyebrow controllers: The cubes that lie on this curve directly control corresponding control points on the brow muscle spline in all axis with parent constraints. This gives full control to the animator, but also the responsibility not to shape impossible brow shapes.

8) Head controller: The Head and the three Neck capsules are controlled by an IK spline handler. Each CV on the IK spline hander curve is parented to a cluster. The cluster at the Head capsule is point constrained to the Head controller making it able to bend the neck in all directions when it is translated. The z-rotation of the Head capsule is in addition orient constrained to the Head controller, in effect overriding the rotation in this axis from its parent (Neck3) This makes the character able to look up and down without bending the neck, and also to bend the neck without rotating the head. The twist attribute of the IK spline handler is directly connected to the y-rotation of the Head controller, letting the character turn his head.

Critical analysis
In the process of making this rig, I have used the muscle objects in every deformation in the face. For the purpose of this project I feel that was justified, because I wanted to know how well the system performed. In a commercial setting on a tight budget, I would have used blend shapes in areas where there were little need for jiggle and inertia. In some cases I thought the system worked very well, like in the lips. I was able to shape the muscles close to the geometry of the model and the controls gave a wide variety of possible lip shapes. In the example of the eyebrows, the muscles were placed at the areas they would influence, but with no restrictions. This made it easy to break the rig, and create shapes that did not look natural. Even if it is very possible to make unnatural poses with the cg lips too, it is easier to see when you are doing something wrong than with the brows. A similar problem occurs with the node control. For example when a person smiles, the real nodes are pulled to the sides, backwards and upwards. In the cg model, the nodes can be positioned anywhere while only a few positions are natural. There are ways of improving this. If the control objects in the interface lay on a surface similar to the face model, this would be a good guide for the animator when moving the points. If could be purely a visual aid, or a scripted set up that didnt allow the controllers to move outside a given area. Ideally, to get the best impression on the quality of the deformations using this system, I should have experimented more with other rigging methods. As it is highly time consuming to test and decide the best possible use of a muscle in a given area, I had to prioritise building the muscle rig. One area that I could not effectively rig just using this muscle approach was the forehead. When the brows are raised, the forehead receives roughly horizontal creases, while a frown produces vertical or radial wrinkles. The cMuscle System allows the user to paint wrinkle maps to control where wrinkles should occur. This could be a possible solution to this problem, but my initial tests using this method were not satisfactory so I left the idea. I further looked into the possibility to use blend shapes in conjunction with the muscle system, and discovered that it worked very well as long as the blend shapes were evaluated before the muscle system.

Conclusion
A major part of this project has been looking into out how to make use of a muscle system in a face rig. In many cases I did not mimic real life muscles, but rather used the cg muscles to preserve volume and simulate inertia in the face. In other cases the muscles proved to be a powerful tool in controlling difficult areas in the face, most notably the lips. Building a facial rig based on a muscle system does not seem to be something many people have attempted. Searching for examples on the Internet gave very few results. Well into the project, I discovered that many people with rigging experience actually do not recommend taking this approach to facial rigging. Michael B. Comet who wrote the muscle system of choice is one of them. Link to this discussion on cgsociety.org: http://forums.cgsociety.org/showthread.php?t=304707 One common argument is that the real life muscle system in the face is so complicated that cg technology still is not at the level required for building a cg muscle face simulation. As I gathered information about face anatomy, I discovered that even if the real life muscle setup is very complex, the results they produce do not have to be as complicated. Building a facial rig which exclusively uses muscle simulation is probably not the most effective approach, but using muscle systems together with other deformation methods gives a very powerful toolset for creating convincing animation.

Images used in this report

Front cover: Render by Jan Erik Olsen Fig. 1, 2, 3, 4, 5, 6, 8, 10, 11, 12, 14, 15, 16, 17, 18, 19, 20: Screen shots from Maya. Fig. 13: Picture by Jan Erik Olsen Fig. 7, 9: Illustration from Human Anatomy for artists, Eliot Goldfinger, Oxford University Press, 1991.

References
Human Anatomy for artists, Eliot Goldfinger, Oxford University Press, 1991. cMuscle System Documentation, included with the plug-in. http://www.cometdigital.com