Planar tracking a face
Video: Planar tracking a faceWe have imported our prepared image sequence into mocha. Now to open up that file or to work in mocha again, we do need to go through After Effects, at least with mocha for After Effects in CS6. The trick for doing that is to select a layer. It doesn't even matter which layer it is. Go to Animation, and go to Track in mocha AE. Mocha opens, but we don't necessarily want this. I can just cancel this New Project. I'll maximize it here. And then you can go and open the file that you want.
- Next steps
Viewers: in countries Watching now:
VFX Techniques introduces common yet critical visual effects techniques that are used in film and television shows on a regular basis. This installment shows how to build complex composites with Adobe After Effects and mocha, where motion graphics are mapped to live-action footage of an actor. Author Lee Lanier starts by combining rotoscoping and effects to digitally apply makeup to an actor to disguise motion tracking marks. Then discover how to transfer footage into mocha and generate planar tracking data that you can use to motion track graphics to the moving face of the actor. Plus, learn how to build and adjust motion graphics to create the look of a virtual tattoo and a pair of holographic heads-up glasses.
- Disguising tracking marks with rotoscoping and effects
- Preparing footage for motion tracking
- Planar motion tracking a face in mocha
- Applying tracking data to artwork in After Effects
- Fine-tuning motion tracking curves in After Effects
- Animating text and graphic elements
- Applying color and light effects
- Batch rendering the final comp
Planar tracking a face
We have imported our prepared image sequence into mocha. Now to open up that file or to work in mocha again, we do need to go through After Effects, at least with mocha for After Effects in CS6. The trick for doing that is to select a layer. It doesn't even matter which layer it is. Go to Animation, and go to Track in mocha AE. Mocha opens, but we don't necessarily want this. I can just cancel this New Project. I'll maximize it here. And then you can go and open the file that you want.
So here is the image sequence again. Now to make room to see our eventual layers, I want to close the Overlay Colors down here, View, turn off Overlay Colors. There we go, lots of room now. Mocha is a planar tracker, which means you identify a pattern surrounded with a spline shape, figure out where that goes over time, and then attach a planar surface to it, as if it was a flat piece of paper.
So let's get started with that. There are several ways to draw spline shapes inside mocha, which you have up here in the toolbar. I find the X-spline the easiest to use, so we're going to start with that. I want to zoom in nice and close though, so I'm going to use the zoom tool right here. Right-mouse click and just drag to get closer. It would be good to start in a frame where everything is very visible, in better resolution. So I'm going to go to a later frame, I think 200 will be good, I'll just type 200 in here, and there we go.
Now in terms of the X-spline, I'll just click this icon and then click in the viewer. Each time you click, you get a point, and it starts to form a closed spline shape. Now you don't have to have just four, you can go further, but I just want four in this case, it makes sense. In order to finish this shape without putting in additional points, right-mouse click, and there you go. As soon as you make this shape, you get a layer that appears up in the Layer Controls panel, and you can rename it, just click on it, backspace, and type a new name, like Cheek.
All right, I'm going to go back to my Pick Tool and click on that, so we can edit it. The goal here is to adjust the X-spline, so it surrounds these dots and so I can tweak it down the road. So a good way to do it here is to move these corner points close so they just touch the edge of the dots. As I move thess around, you'll notice that there's a little zoom window at the top left. This will come in handy once we start seeing keyframes to compare current frames with previous or next keyframes.
So for now I'm going to touch the edge of each dot. Okay, looks good. And notice that a keyframe gets placed, a little green one, right here when I do that. Now we're ready to start tracking. I'm closer to the end so I'm going to track forward, which is this button right here. Once it stops, you'll see a bluish, purplish bar here that indicates that tracking has occurred.
Now mocha will try its best to follow that pattern. But it will slip off based on resolution or grain or color, what have you. What you can do though is adjust this X-spline shape at key junctures to make it fit. Those become keyframes; it will help make the tracker more accurate. So I can just grab these corners again and click-drag, and you'll notice the zoom windows will show our previous keyframe and the current frame. This allows you to match up the keys. All right, that looks pretty good.
I'm going to go back to 200 now and track from this point to the beginning, and here is that button. Okay, looks good. All right, that looks pretty good. And there we go. Now it does slip off, the pattern does slip off, because one thing that interferes is her touching her face.
Two of the tracking marks were covered up and that confuses the tracker. It's consistent once the finger leaves, but it slipped off the dots. What we can do is apply additional keyframes to fix that. Before I go and do that, I want to mention one thing. I'm going to go back to 200 and show you the hidden planar surface. This is off by default, but you can show it by pressing this S button right here. Initially it's just rectangular and square to the camera, but this is the area where the thing you track in After Effects will eventually be placed, the four corners of the tracked layer in After Effects will be pulled towards the four corners of this planar shape.
What we can do at this point is zoom in a little bit and adjust this to match our tracked dots. I'm going to pull the corners, and just for now I want to place them at the corners of the X-spline right here, right here, so I'm overlapping the solid blue line of this shape or this planar surface with the dotted line of the X-spline. Now I'll have to adjust this as we go further in the tracking, but just for now I want to set the shape right here on Frame 200, then I'll hide it.
In any case, now I can go back and set more keyframes to get this planar surface to stick. Now keyframes disappear unless you have one of these points selected and then they appear again. So here I'm on Frame 1 or Frame 0, the first frame. I could try to get these in the correct position. Now the resolution is very small here for this section, but I'll try my best. Okay, looks good. Now you'd approach is like you would in After Effects. You'd place keys where you think they would produce the best curves.
In this case, it would be good to place additional keyframes where there are sudden changes in motion or where the X-spline suddenly slips off. Now it's going to be difficult for me to do this in real time, so I'm going to give you some frame numbers where it would be good to set additional keys. One is Frame 40. That's where it starts to slip off a little bit, even after your keyframe 0. There's Frame 70. Again, a point where it starts to slip off, and this actually is right before her finger touches her face.
77, when her finger is right over this dot here and the shadow starts to interfere. 88, the finger touches the face, we lose one dot completely. 103, where the finger presses deeply into the skin and starts to distort those four dots in terms of their planar quality. And 114, where the finger leaves, and once again, all four dots are visible.
So the idea would be to go to those keyframes and then adjust the X-spline shape to produce additional keyframes. If you do those minimal number of keyframes, then it should be sufficient to get pretty good tracking data. Now there still is a problem where the finger touches the face, but we'll talk about that further later on. We have drawn the X-spline shape and adjusted it over time based on the tracking. In turn the X-spline will determine where the planar surface goes, and that planar surface will turn into tracking data inside After Effects. So you can imagine that the artwork we track in After Effects will be where the X-spline shape is.
There are currently no FAQs about VFX Techniques: Tracking Objects onto a Face.