Easy-to-follow video tutorials help you learn software, creative, and business skills.Become a member

Using the Roto Brush tool

From: After Effects CS5 Essential Training

Video: Using the Roto Brush tool

There is a powerful new type of brush in AfterEffects CS5 called the Roto Brush. And what this does this allows you to go in and select certain objects and remove them from the background. So, I have here this motion graphics back on, for example, with this snowboarder over it, but this is the same photo you were looking at before, which looks like this. So, what we're going to do in this movie is we're going to use the Roto Brush to take regular footage, and isolate the snowboarder so that he appears to jump off of our motion graphics background and then go back to the regular snowy environment to which he feels accustomed.

Using the Roto Brush tool

There is a powerful new type of brush in AfterEffects CS5 called the Roto Brush. And what this does this allows you to go in and select certain objects and remove them from the background. So, I have here this motion graphics back on, for example, with this snowboarder over it, but this is the same photo you were looking at before, which looks like this. So, what we're going to do in this movie is we're going to use the Roto Brush to take regular footage, and isolate the snowboarder so that he appears to jump off of our motion graphics background and then go back to the regular snowy environment to which he feels accustomed.

So, kind of a cool feature to be able to isolate objects like that. So, let's go over to the Start composition and see how this works. I've created some comp markers here for you so we know where to begin and where to end the Roto Brush. Go ahead and double-click this layer to open it up in the Layer panel. Just like every other brush, we need to be in the layer panel to use it, select the Roto Brush tool, which is merely to the right of the Brush tool, which we looked that in the last movie. Oh, yeah also, before you get started, make sure that the resolution is in Full resolution. The Roto Brush really needs to be in full resolution to work properly.

I am going to zoom in here and I am just going to do a rough drag around this object at one second and 12 frames in and actually I might want to be a little bit before then. We are kind of creating our master reference frame we can go back and change this if we want to-- I'll show you how do that momentarily, but what I'm doing here is I am clicking the areas that I want to be selected by the Roto Brush. Now you'll notice that I am not going exactly to the area, say for example, right here where we have this purple outline that tells us where the Roto Brush is seeing our selection.

You will notice that it's very much like the Quick Selection tool in Photoshop and right here at the edge of the snowboard is not being selected. So, I'm just going go ahead and click outside. You'll notice that as I zoom in here that my cursor is not going to the edge. Just a little part of my cursor is going that way. But if I click here, then it automatically just kind of guesses where I want that to go. So, you don't have to go right to the edge and it's probably better that you don't, so you better let AfterEffects kind of guess where we are going to go. So, I could click over here, for example, and then it kind of guesses where it wants it. So again best not to go all the way to the edge.

Now right here, I want to select the glove, but my cursor is a little bit too big. So, just as we saw in the last movie, hold the Command key on the Mac or the Ctrl key on a PC and drag downwards to reduce the size of the brush. Likewise you could drag upwards to increase the size of the brush and just click anywhere to be able to resize that. This is looking pretty good here. Now there are going to be some rough edges here. Do the best that you can to smooth those out, but there are some auto smoothing features built into this feature that we'll be looking at in just a moment.

So, I am just going to go around the edge, get a good selection, as good as I'm able to at the moment. It looks pretty good. Now one of the things that we want to do is we want to use the Page Down and Page Up keys one at a time so we can go to the next frame, and we could see if there's any problems. So, there are a few problems here. So, I am just going to click here to reselect those areas that kind of got lost for whatever reason. Usually these occur around the edges and so they're some of the shirt and back that didn't quite get covered.

That's looking pretty good. So, Page Down. At first, this will seem like a tedious process, although if you have ever done any rotoscoping, which is typically what they call when you have to mask out something frame by frame. That's extremely tedious, and this is a great timesaver for that. You don't have to do every single pixel by hand. It's a great start. And because this is such an intelligent tool, you really don't have to go and make that many changes frame by frame. It seems like after a while the Roto Brush starts to anticipate your very move and so it seems like you've to adjust less with every frame that you go to.

It's not going to be perfect. I'm going to show you some things in a just second and it will make things go even faster for you, but there this frame is almost perfect as is. There is a little bit of a problem around the edges, almost don't even need to worry about that, but with each frame we have less and less to correct. It seems like that frame looks pretty good and a little bit here on the snowboard needs to be fixed. So, what I am going to do is hit the Page Down key and keep advancing frame by frame until each frame looks pretty decent. Now, right here starting so that there is part of the helmet that's not covered so you could zoom in here and make sure that that's not getting covered.

Page Down. It's kind of getting the gist of it. So, I can just keep doing Page Down and I'll let you master this, but I want to pay attention to this. There is a kind of little mini timeline that's been created for us here in the Layer panel when you are working with the Roto Brush, and there's this first frame when we started Roto Brush, and that's where this little like orange dot is here. And we actually want to move the Current Time Indicator to where the Begin Roto Brush Marker was. And what am I actually going to do is trim this from the left edge, so that the Roto Brush starts at this frame.

Then I am going to move out to where our end Roto Brush is right about there and actually there are some pieces that I don't want here. And when you want to get rid of a piece of something that the Roto Brush is selected you hold the Alt key down the PC or the Option key on the Mac and just click, and it will remove stuff that the Roto Brush tool has selected. So, this looks pretty good, so I am going to trim the end of the Roto Brush selection, and now we have this little tiny Roto Brush area.

You can compare them with these icons down on the bottom left-hand corner. You could see our man little bit better so there is a white/black showing of our transparency where white is the opaque parts of our mask, and the black is stuff that will be removed. You could also see the original alpha boundary. We could also see it with like a quick mask type selection with this red overlay if we preferred that. Now I am going to go to the black-and -white selection, the alpha channel. Now what we once thought was a fairly decent mask is really a not very good mask at all.

So, what we could do is go over to the Effect Controls panel and notice that that there is a Roto Brush effect. And so we could do is we could check Refine Matte and just by checking that even without adjusting any other settings, we see a great improvement to what's going on. There's even been some motion blur automatically added for us, which is really getting clean things up a bit. We could also increase the smooth value if we want to smooth out the mask even more. We take it up to four or something. You want to be careful with this though because sharp corners will no longer be sharp if Smooth is high, so be careful about that.

If you want a softer edge, you could increase Feather to something like 40%. Increasing the Choke value will actually restrict the edge pixels bringing them in. I don't want to do that. I also want to show you this Reduce Chatter property, which is really cool. A lot of times when you are working with these automatic keying tools where you trying to remove a background, the noises will kind of jitter and shake from frame to frame. So, if you increase this value-- let's take all the way to 100 percent-- it's going to reduce the chatter. You might not see that much of a difference on the given frame that you are on, but from frame to frame, it's going to be a little bit more of a smooth edge.

Once you got everything the way you want it for the Roto Brush tool and you have the timeline correct here, what I am going to do is click the Freeze button that kind of lock this in place. So, now that we've frozen this selection, if we wanted to unfreeze it we could click this button to unfreeze it and basically when that's done it's cached our little segment of the Roto Brush time span. And so if we to adjust this what we want to do is unfreeze this. But if we go back to our composition here, we'll see that for the Roto Brush span the background has been removed. So here it is with the background and once we've started with the Roto Brush, then our snowboarder is now over nothing basically.

If I take off these layers, we could see that he's over a black background and these are the layers that I created as the background here. But this opens up a lot of possibilities. One, it's cool for motion graphics. We can put maybe some titles up here, something like that, maybe like outline this guy or create some of the motion graphic elements around him, which is a very cool effect. But it also is good for color correction and that type of thing where if we wanted to maybe just bring out this guy and maybe not necessarily isolate him and remove the background. Let me just go ahead and select this and hit Command+D to duplicate the layer, Control+D on the PC. On this back clip, what I'm going to do is I'm going to select it and then delete the Roto Brush effect.

So, basically, we have the snowboarder and then we have the regular background, but because he is isolated on the top layer we could do something like maybe invert. I'll apply the Invert effect to the top layer here and so we've inverted him for that time where he's flipping. So, we could add a color correction. Maybe we could just lighten him or darken him or something a little bit more subtle, but I just want to show you that if you wanted to perform some kind of extra color correction on just one isolated object, this is a great feature to use for that.

Now be careful that you don't over- estimate the power of the Roto Brush feature. For things like this like a little wispy hair or semi transparent objects, you're much better off using a keying tool to do that. So, don't think that the Roto Brush is like this be-all, end-all, cure-all problem for all things that you might run across with AfterEffects. But it can get you a majority of the way there and for times when the edge does not have to be perfect, like a lot of times when you are doing color correction, you really can't beat the new Roto Brush feature in AfterEffects CS5.

Show transcript

This video is part of

Image for After Effects CS5 Essential Training
After Effects CS5 Essential Training

111 video lessons · 67470 viewers

Chad Perkins
Author

 
Expand all | Collapse all
  1. 5m 40s
    1. Introduction
      1m 30s
    2. What is After Effects?
      3m 12s
    3. How to use the exercise files
      58s
  2. 28m 14s
    1. After Effects workflow overview
      2m 18s
    2. Bringing elements into After Effects
      2m 23s
    3. Adding elements to the Timeline
      1m 57s
    4. Working with layers
      3m 45s
    5. Creating animation with presets
      3m 24s
    6. Applying effects
      3m 34s
    7. Creating animation without presets
      5m 38s
    8. Previewing your work
      2m 46s
    9. Exporting content as a movie file
      2m 29s
  3. 27m 20s
    1. Touring the interface
      6m 2s
    2. How After Effects projects work
      4m 47s
    3. What is a composition?
      4m 52s
    4. Tips for adding content to compositions
      2m 49s
    5. Understanding the properties of video
      8m 50s
  4. 57m 8s
    1. Importing an Illustrator file
      4m 57s
    2. Animation basics
      7m 12s
    3. Animating opacity
      6m 40s
    4. Understanding anchor points
      4m 57s
    5. Animating position
      6m 8s
    6. Animating rotation
      4m 41s
    7. Animating scale
      7m 19s
    8. Using the Puppet tool
      7m 13s
    9. Copying and pasting keyframes
      3m 4s
    10. Animation shortcuts
      4m 57s
  5. 9m 42s
    1. Understanding precomposing
      6m 51s
    2. Navigating through compositions quickly
      2m 51s
  6. 1h 12m
    1. A showcase of effects
      2m 34s
    2. Creating a layer for effects
      3m 1s
    3. Applying effects
      4m 54s
    4. Animating effect properties
      4m 29s
    5. Using Glow
      5m 34s
    6. Creating patterns and textures
      6m 57s
    7. Creating a fireball
      7m 9s
    8. Using the Cycore effects
      5m 58s
    9. Adding blur
      5m 45s
    10. Creating a galaxy scene from scratch
      8m 38s
    11. Distorting objects with effects
      4m 7s
    12. Creating and using lens flares
      4m 21s
    13. Creating lightning bolts
      4m 3s
    14. Viewing random variations with Brainstorm
      4m 39s
  7. 30m 52s
    1. Shortening the duration of layers
      4m 23s
    2. Trimming in the Footage panel
      4m 14s
    3. Slowing and accelerating video speed
      7m 9s
    4. Applying video transitions between clips
      6m 7s
    5. Working with image sequences
      4m 47s
    6. Importing footage with an alpha channel
      4m 12s
  8. 36m 11s
    1. Brightening dark footage
      9m 12s
    2. Changing colors in footage
      6m 34s
    3. Creating cinematic color treatments
      8m 17s
    4. Creating a quick vignette
      3m 42s
    5. Colorizing black-and-white objects
      4m 50s
    6. Using adjustment layers
      3m 36s
  9. 21m 9s
    1. Creating and editing text
      7m 39s
    2. Applying text animation presets
      4m 41s
    3. Animating text manually
      4m 43s
    4. Applying layer styles to text
      4m 6s
  10. 28m 58s
    1. Let's get better
      37s
    2. Using work areas
      3m 37s
    3. Creating markers
      6m 17s
    4. Replacing layers
      2m 35s
    5. Mastering Timeline navigation
      3m 18s
    6. Aligning and distributing layers
      3m 4s
    7. Selecting layers quickly
      1m 56s
    8. Cropping layers
      3m 43s
    9. Adjusting comp resolution
      3m 51s
  11. 23m 53s
    1. Using the paint tools
      9m 35s
    2. Using the Roto Brush tool
      9m 25s
    3. Animating growing vines
      4m 53s
  12. 40m 29s
    1. Creating and using masks
      6m 42s
    2. Exploring mask options
      7m 57s
    3. Creating masks with Auto-trace
      6m 51s
    4. Masking objects with other objects
      5m 33s
    5. Making shape layers
      3m 43s
    6. Modifying shape layers
      9m 43s
  13. 30m 44s
    1. Turning 2D layers into 3D layers
      9m 22s
    2. Creating lights and cameras
      6m 14s
    3. Creating shadows
      4m 23s
    4. Using depth of field
      4m 42s
    5. Working with 3D effects
      6m 3s
  14. 18m 10s
    1. Removing a green screen background
      4m 37s
    2. Refining the matte
      4m 48s
    3. Compositing with color adjustments
      4m 50s
    4. Compositing with blend modes
      3m 55s
  15. 25m 44s
    1. Understanding spatial interpolation
      2m 5s
    2. Creating and adjusting motion paths
      3m 55s
    3. Orienting moving objects along a path
      1m 29s
    4. Drawing motion with Motion Sketch
      2m 51s
    5. Creating pauses in animation
      3m 6s
    6. Understanding temporal interpolation
      1m 56s
    7. Easing keyframes
      5m 57s
    8. About the Graph Editor
      4m 25s
  16. 12m 13s
    1. Stabilizing shaky footage
      7m 46s
    2. Tracking the motion in footage
      4m 27s
  17. 24m 58s
    1. Setting up parent layers
      5m 49s
    2. Working with null objects
      2m 31s
    3. What are expressions?
      7m 17s
    4. Modifying simple expressions
      2m 20s
    5. Using the wiggle expression
      7m 1s
  18. 6m 52s
    1. Understanding audio in motion graphics
      1m 22s
    2. Previewing and mixing audio
      3m 55s
    3. Enhancing audio tracks with effects
      1m 35s
  19. 11m 36s
    1. Adding comps to the Render Queue
      2m 30s
    2. Exploring key Render Queue settings
      4m 11s
    3. How should I export my video?
      4m 55s
  20. 7m 16s
    1. Using Photoshop with After Effects
      2m 10s
    2. Using Illustrator with After Effects
      3m 2s
    3. Using Flash with After Effects
      2m 4s
  21. 11s
    1. Goodbye
      11s

Start learning today

Get unlimited access to all courses for just $25/month.

Become a member
Sometimes @lynda teaches me how to use a program and sometimes Lynda.com changes my life forever. @JosefShutter
@lynda lynda.com is an absolute life saver when it comes to learning todays software. Definitely recommend it! #higherlearning @Michael_Caraway
@lynda The best thing online! Your database of courses is great! To the mark and very helpful. Thanks! @ru22more
Got to create something yesterday I never thought I could do. #thanks @lynda @Ngventurella
I really do love @lynda as a learning platform. Never stop learning and developing, it’s probably our greatest gift as a species! @soundslikedavid
@lynda just subscribed to lynda.com all I can say its brilliant join now trust me @ButchSamurai
@lynda is an awesome resource. The membership is priceless if you take advantage of it. @diabetic_techie
One of the best decision I made this year. Buy a 1yr subscription to @lynda @cybercaptive
guys lynda.com (@lynda) is the best. So far I’ve learned Java, principles of OO programming, and now learning about MS project @lucasmitchell
Signed back up to @lynda dot com. I’ve missed it!! Proper geeking out right now! #timetolearn #geek @JayGodbold
Share a link to this course

What are exercise files?

Exercise files are the same files the author uses in the course. Save time by downloading the author's files instead of setting up your own files, and learn by following along with the instructor.

Can I take this course without the exercise files?

Yes! If you decide you would like the exercise files later, you can upgrade to a premium account any time.

Become a member Download sample files See plans and pricing

Please wait... please wait ...
Upgrade to get access to exercise files.

Exercise files video

How to use exercise files.

Learn by watching, listening, and doing, Exercise files are the same files the author uses in the course, so you can download them and follow along Premium memberships include access to all exercise files in the library.


Exercise files

Exercise files video

How to use exercise files.

For additional information on downloading and using exercise files, watch our instructional video or read the instructions in the FAQ.

This course includes free exercise files, so you can practice while you watch the course. To access all the exercise files in our library, become a Premium Member.

Join now "Already a member? Log in

Are you sure you want to mark all the videos in this course as unwatched?

This will not affect your course history, your reports, or your certificates of completion for this course.


Mark all as unwatched Cancel

Congratulations

You have completed After Effects CS5 Essential Training.

Return to your organization's learning portal to continue training, or close this page.


OK
Become a member to add this course to a playlist

Join today and get unlimited access to the entire library of video courses—and create as many playlists as you like.

Get started

Already a member?

Become a member to like this course.

Join today and get unlimited access to the entire library of video courses.

Get started

Already a member?

Exercise files

Learn by watching, listening, and doing! Exercise files are the same files the author uses in the course, so you can download them and follow along. Exercise files are available with all Premium memberships. Learn more

Get started

Already a Premium member?

Exercise files video

How to use exercise files.

Ask a question

Thanks for contacting us.
You’ll hear from our Customer Service team within 24 hours.

Please enter the text shown below:

The classic layout automatically defaults to the latest Flash Player.

To choose a different player, hold the cursor over your name at the top right of any lynda.com page and choose Site preferencesfrom the dropdown menu.

Continue to classic layout Stay on new layout
Exercise files

Access exercise files from a button right under the course name.

Mark videos as unwatched

Remove icons showing you already watched videos if you want to start over.

Control your viewing experience

Make the video wide, narrow, full-screen, or pop the player out of the page into its own window.

Interactive transcripts

Click on text in the transcript to jump to that spot in the video. As the video plays, the relevant spot in the transcript will be highlighted.

Are you sure you want to delete this note?

No

Your file was successfully uploaded.

Thanks for signing up.

We’ll send you a confirmation email shortly.


Sign up and receive emails about lynda.com and our online training library:

Here’s our privacy policy with more details about how we handle your information.

Keep up with news, tips, and latest courses with emails from lynda.com.

Sign up and receive emails about lynda.com and our online training library:

Here’s our privacy policy with more details about how we handle your information.

   
submit Lightbox submit clicked
Terms and conditions of use

We've updated our terms and conditions (now called terms of service).Go
Review and accept our updated terms of service.