Ready to watch this entire course?
Become a member and get unlimited access to the entire skills library of over 4,900 courses, including more Video and personalized recommendations.Start Your Free Trial Now
- View Offline
The After Effects Apprentice videos on lynda.com were created by Trish and Chris Meyer and are designed to be used on their own and as a companion to their book After Effects Apprentice. We are honored to host these tutorials in the lynda.com library.
- Creating, erasing, and animating brush and clone strokes
- Using Paint to reveal a layer over time
- Creating animated distortions to flat artwork with the Puppet tools
- Replacing backgrounds with the Roto Brush
Skill Level Intermediate
Now the first step to using Roto Brush is to choose a frame where you can see the entire object that you need to cut out, or failing that, as much of it as possible. In this case these tulips don't move that much and they're always fully in the frame. So what frame I use for my starting point is not all of that important. But the frame you choose is what's going to be called a Base Frame. Roto Brush is going to take the information you create on that frame and propagate that information earlier and later in time. We'll just go to one second even now just for fun.
Next, you need to select the Roto Brush tool. It's this large brush painting out this little man. I'll select the tool and as I move over the Layer panel, you'll see I have a circle that indicates the size of our brush. It's also green in color with a plus sign in the middle, which indicates I'm going to add to my alpha channel. I'm going to be working on my foreground not my background. You can resize the Roto Brush, just as you would your Paint Brush, by holding Command on Mac or Ctrl on windows, then dragging. Unlike a normal painting where you might want to create a small brush to get the detail around the edges of this flower, Roto Brush works a little bit better if you make large strokes because it's going to be looking at the colors that you drag your stroke over.
So I'm actually going to make a fairly large brush here to take in a lot of these petals with a minimum amount of work. Once I've done that I'll click and drag through these petals that I want to be included in my Roto Brush. Don't go outside and don't cross over to the background, you'll get a false reading then. I'm going to Undo. Instead, just paint right through these petals. After Effects will draw a pink segmentation boundary around the area that it found from our colors that we stroked in searching outwards until it found a transition.
The area inside that propagation boundary is the foreground, the area outside is the background. You can add multiple areas to your foreground selection. For example, the butterfly flew behind this petal as well, or at least we want it to. So I'll click and drag through those petals and they will get a pink segmentation boundary as well. Down here in the Layer panel's timeline you'll see a little gold bar. The gold bar indicates the Base Frame that you created where you first drew these strokes.
You'll also see this gray bar with arrows pointing to the right and to the left. As you can see from the tooltip, this is called the Roto Brush Span. Roto Brush is going to take the information from the Base Frame and propagate it on successive frames earlier and later in time. It will look for any movements from frame to frame and build upon that information as it gets further way from your Base Frame. You can use Page Up or Page Down, or as you did with the Paint tool, you can use the 1 and 2 keys on the normal portion of your keyboard to step forward frames or to step backward frames.
You'll see the pink segmentation boundary is updating to take in to account these petals being blown around in the wind. If I'll just randomly jump to a frame later in time, there'll be some calculation time involved as Roto Brush propagates out to that frame. Roto Brush also has a number of tools that allows you to see the result of you creating your segmentation boundary, even including a masked overlay like you may be used to seeing in Photoshop. I'll go back to my segmentation boundary for now.
Okay. Let's see how this looks. I'm going to bring my Comp panel forward and I see the butterfly is indeed behind my flower petals. Cool! I'm going to press 0 on numeric keypad to cue up RAM preview. Roto Brush does take a while to calculate, because it has to do all that propagation work. So far, so good. Butterfly spans the petals, and the butterfly disappears for part of the flight. Well, what's going on there? Let's go to where it disappeared and go back into our Layer panel. You'll notice that the time marker is beyond my Roto Brush Span.
After Effects only assume it's good for so many frames, 20 to be precise, before it says, you know, I really would like to have some more information before I continue to use this old base frame to go this many frames away. If Roto Brush was having difficulty, you can train it to keep up with changes in your footage, and that's going to be covered in-depth in the next chapter. But in this case, the Roto Brush was working just fine up until the point we ran beyond the end of our span, and now you'll see the pink outline has enclosed the entire frame, you can go ahead and just drag out the span longer.
I'll jump here later in time. It'll take a few moments for it to propagate, and looks like it's still properly enclosing these tulips. No problem. I'll go back to my Comp panel, RAM preview, and now my butterfly stays in the picture during the span of when we have our Roto Brushed copy the layer in front. That's looking pretty good. But there's one other little issue here. You might have noticed when the butterfly is behind the tulip petals, there's a little bit of a black line.
Compositors often refer to this as a matte line and they're considered to be undesirable. You want those edges to blur together. I'm going to zoom in a little bit here, hold the Spacebar and pan down. You can see a little bit of that black fringing here where the alpha channel is not quite perfect. And a little bit in to here. Roto Brush has an entire section to refine the matte that it has created. You just need to turn it on. It slows things down. That's why it defaults to being off. But when you're at this stage, you now want a refined matte.
And you'll notice some of the black fringing went away. Before and after. By doing so it decontaminates any color that bled into those nice anti-aliased edges. It calculates motion blur for any movement in the mask outline. It's a very powerful section of the Roto Brush effect. You can even choke it a little bit more to get rid of more of those matte lines if you want to. So final step, I'm going to press T to reveal the opacity of my tulip in front. These tulip petals look like they are semi-translucent, so I'm going to back off the opacity of this copy in front until I just barely see the butterfly through that copy of the tulip.
Press Shift+/ to recenter my view, RAM preview, and again, it takes some time to calculate, particularly propagating the additional Roto Brush stroke information. But in the end I have a very nice composite and now my butterfly appears to be flying above the tulips even though the tulips are on their own plane. Now this was a particularly easy use of Roto Brush, trust me.
Almost never will it be this easy to use Roto Brush in real-life. Therefore, in the next chapter we're going to tackle a much more difficult real-world example of cutting out a person's arms as they move in front of the computer screen.