Join Mark Christiansen for an in-depth discussion in this video Making a quick selection with Roto Brush, part of After Effects CS5 New Features.
The very name Roto Brush implies a tool that will isolate an element by allowing you to use a brush to create a selection and that's pretty much what it does. Instead of looking at this tool as a wholesale replacement of rotoscoping, it may be helpful to think of it for situations where it might not have even occurred to you that an automatic selection would be helpful. Now in this shot the problem as I perceive it is that this bowl here in the foreground is so dominant in a shot that is actually establishing a woman arranging some flowers.
I could try to color correct it, but it's so similar to surrounding objects that I will be in for quite a challenge. Without Roto Brush I could consider tracking in a selection, I could throw a gradient across the bottom of shot to just sort of pull everything back down there, or I can even consider reframing the shot. But instead I'm going to see what Roto Brush can do to isolate that element and let me take control of it. I'll start by duplicating the background layer and I'll call my new layer roto brush.
To work in Roto Brush there are two things that you should probably make sure of. One is that you're at full resolution. It works best at full and two is that you're in the layer window, which you get to by double-clicking the layer. So I'll bring this window up full and use Option+/ or on Windows Alt+/ to bring the frame up full frame. Now I am going to look for the frame in which the bowl is most clearly visible. Of course, it's visible throughout the shot, although I do notice that it does dip down a little bit. The lip leaves the frame here.
So I'll go back to where it's clearly visible in the shot over here. The Roto Brush tool is selected here at the top of the layer window. I'll activate that. Now I'll scale it because it's looking rather small by holding down my Command key, on Windows this would be Control, and just drag the mouse to scale it up. Something medium-size will be fine for this nice big element. Now I am going to outline the basic shape of this bowl. Now watch carefully. I'm not going to be exact and I'm not going to go close to the boundaries.
I am just going to draw a circular shape kind of right around the middle of a bowl here. This is rather a big brush actually. And voila! The segmentation boundary created by Roto Brush is shown in pink, and now to remove this area down below that was also selected, I can make my brush little smaller using the Command key and dragging the mouse. Hold down Option on Mac or Alt on Windows and drag again, nowhere near the borders, and Roto Brush is pretty close to guessing what I wanted.
Now if I had made a mistake and say crossed into or out of my borders, just undo that. So once again, closer that time to the border. It's missing this little bit, so I can go pick that up. If I like, I can even zoom-in and get really precise by telling Roto Brush what is and is not part of shot. Looking down at a layer timeline I see a couple of new things.
There's a yellow anchor frame and these gray arrow shapes extending in both directions, which are known as the span of the Roto Brush. So the first thing you do is make sure that the anchor frame is as you like it. It doesn't have to be pixel perfect. Within a pixel or two of the edge is certainly good enough, and I am not going to worry so much for right now about things like this little stem. It's not such an important part of this color correction. Now work your way outward from the anchor frame for checking surrounding frames.
Pressing the 1 key at the top of your keyboard or Page Up as usual goes back one frame. Pressing 2 takes you forward. As you move further from the anchor frame, Roto Brush makes its prediction, frame by frame, of the selection area. If it's helpful, toggle Alpha Overlay, this rubylith view of the alpha channel. You can also use a matte view if you like. I can press the Spacebar and just watch Roto Brush draw this animation frame by frame.
Now notice if instead I skip to a frame several frames behind the anchor, the tool is still going to frame by frame draw all the intervening frames and that's just how Roto Brush works. It starts with the anchor point and works its way outward. So it's going to render those intervening frames whether they're the frames you want to look at or not. I can go to the very end of the span to see if that's looking good. Let Roto Brush render the intervening frames and if it is looking good, I even have the option of extending the span right back to the beginning of shot.
So that does still look good to me and I am fairly confident that there is not going to be a big change right back to the beginning frame of the shot. Moving to the other end of the shot, here where the bowl leaves the frame. I can confidently extend the span here, because again there's just not a whole lot changing. I'll take it as one frame longer and let it run out those frames as well. At that point, because the shape changes and leaves frame, I think I will take the option of creating another anchor frame and another span.
The reason for that is as we get out here, I'll show you what happens. More of the bowl is going to start coming into the shot out here. So I think I'll fix that on a separate span. So this looks like the last frame where that was not affected. Make that the end of the span. Then come out somewhere over here, draw the same shape again or something similar to it. Go back to the segmentation boundary just to see what I'm doing. Now carefully remove these areas that I don't want, work my way back, and this time I'm looking for any little hitch in this thing, for any error or any inaccuracy as it comes up while I work my way away from the anchor point.
There is a reason for that. If I move out here and fix the matte on this frame and then move back to the frame closer to the anchor point, it's not fixed. Whereas if I fix it closer, I have a much better chance of making an edit that will actually carry through to the end. Now getting each and every little last bit of detail here perfect may not be too necessary in this case. That's looking pretty good. When I'm happy with the matte, I can freeze it using this button and that will render the entire Roto Brush.
This prevents the brush from recalculating. So this tool is useful in production without pre-rendering it. The frozen Roto Brush frames are shown in purple along the timeline. Now I'll use the Tilde key or Accent key, that's the one above you Tab key at the upper left of your keyboard, and close the layer window. Back here in the comp all I have to do is apply Effect > Color Correction > Hue/Saturation and I'll just bring the Saturation and Lightness of that bowl down a little bit to blend it in more seamlessly with the rest of the scene.
If you don't need precise accuracy or your target is easy to isolate, Roto Brush needs little extra effort from you in order rotoscope a selection for you.
- Comparing After Effects CS4 and CS5
- Working in the CS5 Timeline
- Importing RED footage
- Selecting with the Roto Brush tool
- Optimizing performance and memory usage in a 64-bit application
- 3D image warping with Freeform AE
- Optimizing selections with the Refine Matte effect
- Changing color with Color Finesse and updated blending modes