Join Chris Meyer for an in-depth discussion in this video The Refine Edge tool, part of After Effects: Creative Cloud Updates .
Next we're going to look at a wonderful addition to After Effects called the Refine Edge tool. One of the hardest things to rotoscope or key are fine strands of hair. I'll RAM preview this original shot, and you see as this actress turns her head, her hair flies off to the sides, and also there's various strands sticking up and sticking out to either side. This is normally a great pain to attempt to key or roto. Now what I did try is using the ordinary Roto Brush Effect that came with prior versions of After Effects to create a hard mask around this person.
I am going to turn on the Effects, because Roto Brush is indeed an effect, and you'll see that I have a hard-edged matte. I'll turn off my backround for now, and you can see the result of Roto Brush. A shortcoming of Roto Brush is it creates what's called a Binary Matte, either you are outside or you are inside, your background, or your foreground. Roto Brush does have the ability to do some nice things with the edges between foreground and backgrounds, such as add Motion Blur and also decontaminate some Edge Color. This comes in particularly handy when this actress is turning her head, and her hair is flying off at an angle without Motion Blur, and width, but it still does not capture all of the strands of her hair.
Well this is where the brand- new Refine Edge Tool comes in. I am going to turn off Motion Blur and Decontaminate Edge Color for now, double-click the layer to open it up in its Layer Panel. You need to do this work in the Layer Panel, same as with Roto Brush, and return it to my base frame. Now before you use the Refine Edge Tool which is now part of Roto Brush, you must first have performed your normal Roto Brush work for the entire duration of the clip. You do not do one frame of Roto Brush, one frame of edge and keep going back and forth that way.
Do your entire Roto Brush work, tweak out your Propagation parameters, make corrective strokes, etcetera, before you use Refine Edge. Once you have done that you can go to the Roto Brush Tool and you'll see it now has a second option called the Refine Edge Tool. Opt or alt + w, switches between the two, and I'll be using that shortcut a lot during this lesson. With the Roto Brush Tool you've got your familiar green circle to add to the foreground, and if I hold opt or alt, red circle to subtract from the foreground or add to the background.
By selecting the Refine Edge Tool, you'll now see that I have a purplish brush outline for the edges. And I'll hold opt or alt, it'll turn to a very dark blue. So that's one visual clue as to which tool you are in. I am going to turn on my Propagation boundary so that I can see it around my entire actress, and more importantly see the hair that has escaped beyond my Propagation boundary. I have tried to make the Propagation boundary capture the general helmet of her head, gets some of the larger strands, but obviously I couldn't get every single small strand.
You can resize the Refine Edge Tool by holding cmd on Mac, ctrl on Windows and dragging, and you want to size it to be just enough to capture this little band here between where we have soft things outside of your Propagation boundary, and what's inside your Propagation boundary. Don't make it too large or you'll be capturing too many colors. The next thing you want to do is draw a stroke along your Propagation boundary that captures these fine hairs that are either flying off to the side or gaps created inside.
If you try to draw your first stroke completely in the foreground, you'll receive an error message, your first Refine Edge Stroke must straddle an off edge, I'll Undo, and you'll get the same thing if you drew a stroke in the background, and instead I'll draw along this edge and capture some of these hairs flying away. You don't need to do everything in one stroke but I'm going to grab a fair amount here, and when I'm done you are going to see what they refer to as an X-ray view, rather than seeing the color information you're going to see the resulting alpha channel in the area where you drew a Refine Edge stroke, and now you can see some of these really fine details in this woman's hair.
I am going to press the Tilde (~) key to maximize my Layer Panel, so you can see this as large as possible. Now I am going to add a couple of more strokes to capture bits of her hair that I've missed, little fly-away strand up there, these hairs coming down the side along here, some of these fly- away hairs going off to the side here. And through her shoulder to capture some of these holes that have appeared between her hair and the background. You'll see I'll get little areas of partial transparency there.
You have to be careful that Refine Edge doesn't go overboard and grab some of your fully opaque background and tries to make it semi-transparent. Such as what's happening in the shoulder here. The technique that I constantly use is that I click the Render switch on and off to make sure that it hasn't made some of my opaque foreground partially transparent, as it has right through here. If you have a problem where it's making something transparent that shouldn't be, hold opt or alt, you'll get the minus or subtraction version of this brush and brush through the area that's giving you trouble.
You'll see that Roto Brush with Refine Edge Tool has now subtracted that from the area it's calculating, and is now doing a much better calculation of exactly where the hair is. Now this actress, she has a lot of spill from her background onto her shoulders, so I really need to pay attention that I don't accidentally eat away part of her skin. I am going to turn the Render button back on and now work on the other side of her head. There is a little bit of strand that flew away there and capture this edge, as well as these gaps, in between the outer sheath or helmet of her hair and inside closer to her face.
I'll drag through here as well --lovely partial transparency there-- and down through her shoulder. And again be careful of going too far because if I do that you will see I'll start to get some gray areas on her shoulder, which should be fully opaque. I'll zoom in to 200%, so you can see this problem area. I am going to undo my stroke, and be careful just to grab the area where I have an edge with partial transparency, that's much better, and I'll go back to 100%.
And by the way an alternative to clicking the Render switch on and off all the time, is to go ahead and turn off the X-ray view. Now you'll see that the Propagation boundary is showing a fuzzy pink line instead of a hard pink line that helps to indicate the areas that are partially transparent and there is your new alpha channel. But I'm going to go back to X-ray view, because I am not done yet.
The September 2013 update brought the new Rigid Mask Tracker, as well as additional ways to scale up footage cleanly, while the highlight of the December 2013 update was the ability to convert parametric shape layers to Bézier paths, and Bézier paths into shape layers. The NAB 2014 update shows off important new integration with Adobe Premiere Pro and Typekit, as well major updates to effects. Smaller yet still important new and enhanced features in each release are also touched on throughout. As always, Chris doesn't just show you where these new features are, but how to apply them to your own projects, along with preferred working practices and potential gotchas.
Note: This course was created and produced by Chris and Trish Meyer. We are honored to host this content in our library.
- Integrating with CINEMA 4D
- Using the Refine Edge tool to fine-tune mattes
- Applying Reverse Stabilization
- Preserving scale while stabilizing
- Working with layer snapping
- Finding missing footage, fonts, and effects