New Feature: Playlist Center! Pick a topic and let our playlists guide the way.

Start learning with our library of video tutorials taught by experts. Get started

After Effects Apprentice 05: Creating Transparency
Illustration by John Hersey

Masking with the Pen tool


From:

After Effects Apprentice 05: Creating Transparency

with Chris Meyer and Trish Meyer

Video: Masking with the Pen tool

In the previous chapter we used the mask shape tool to drag out some simple mask shapes to create transparency for a layer. However, if we need a more complex shape, for example to cut out a flower such as this, we need to use a different tool, the Pen tool. Its shortcut is G. I select the Pen tool and make sure the first option is currently selected. You can go ahead and switch to these other options in context while you're editing. So make sure that first one is the currently active one. And for now we're going to turn off the RotoBezier option.
Expand all | Collapse all
  1. 3m 17s
    1. Overview
      1m 17s
    2. Using the exercise files
      2m 0s
  2. 28m 53s
    1. Creating basic mask shapes
      5m 7s
    2. Using advanced parametric shapes
      3m 35s
    3. Basic mask shape editing
      4m 35s
    4. Masking in the Layer panel
      1m 55s
    5. Working with mask parameters
      4m 0s
    6. Animating masks
      5m 52s
    7. Creating vignettes
      3m 49s
  3. 25m 56s
    1. Masking with the Pen tool
      6m 44s
    2. Editing a mask path
      6m 19s
    3. Using RotoBezier masks
      4m 32s
    4. Targeting masks for animation
      3m 13s
    5. Setting the first vertex point
      5m 8s
  4. 19m 7s
    1. The basics of variable-mask feathering
      7m 33s
    2. Exploring variable-mask feathering options
      5m 56s
    3. Exploring visual effects applications
      5m 38s
  5. 7m 55s
    1. Exploring mask modes
      3m 58s
    2. Fading mask opacity
      3m 57s
  6. 11m 48s
    1. Using targeted treatments
      2m 35s
    2. Filling mask shapes
      3m 8s
    3. Following mask paths
      6m 5s
  7. 13m 31s
    1. Using alpha mattes
      3m 48s
    2. Grouping track matte pairs
      3m 40s
    3. Working with luma mattes
      3m 52s
    4. Animating mattes
      2m 11s
  8. 10m 0s
    1. Using Stencil Luma
      3m 5s
    2. Using Stencil Alpha
      2m 5s
    3. Using effects with stencils
      2m 20s
    4. Stacking stencils
      2m 30s
  9. 8m 23s
    1. Quizzler challenges
      1m 18s
    2. Quizzler solution one: One word at a time
      3m 53s
    3. Quizzler solution two: Stroke drawing direction
      3m 12s
  10. 11m 18s
    1. Idea corner one: More masks and effects
      2m 50s
    2. Idea corner two: Transition effects
      3m 44s
    3. Idea corner three: Sequenced layers as mattes
      4m 44s
  11. 16m 29s
    1. Understanding track matte rendering order
      5m 48s
    2. Exploring mask interpolation
      10m 41s

Watch this entire course now—plus get access to every course in the library. Each course includes high-quality videos taught by expert instructors.

Become a member
please wait ...
After Effects Apprentice 05: Creating Transparency
2h 36m Beginner Mar 23, 2011 Updated Nov 15, 2012

Viewers: in countries Watching now:

In this course, Chris Meyer demonstrates the most common techniques for adding selective transparency to layers in After Effects through the use of masks, track mattes, and stencils. In addition to explaining the tools and basic theory behind transparency, the course covers several practical applications for these techniques, including isolating objects, creating vignettes, and filling text with visual texture. Tutorials on crafting custom transitions and other treatments are also included. Exercise files accompany the course.

The After Effects Apprentice videos on lynda.com were created by Trish and Chris Meyer and are designed to be used on their own and as a companion to their book After Effects Apprentice. We are honored to host these tutorials in the lynda.com Online Training Library®.

Topics include:
  • Creating masks using parametric shapes or freeform with the Pen tool
  • Editing and animating masks
  • Combining multiple masks
  • Using one layer to define the transparency of others
  • Explaining the interaction between effects, masks, and mattes
  • Mastering mask animation
Subjects:
Video Motion Graphics Visual Effects
Software:
After Effects
Authors:
Chris Meyer Trish Meyer

Masking with the Pen tool

In the previous chapter we used the mask shape tool to drag out some simple mask shapes to create transparency for a layer. However, if we need a more complex shape, for example to cut out a flower such as this, we need to use a different tool, the Pen tool. Its shortcut is G. I select the Pen tool and make sure the first option is currently selected. You can go ahead and switch to these other options in context while you're editing. So make sure that first one is the currently active one. And for now we're going to turn off the RotoBezier option.

We'll demonstrate RotoBezier a little bit later on. If you have the exercise files that came with this project, open up the comp 02a-Bezier*starter. Bezier is the name of the type of curve you'll be creating with the Pen tool. And just as before make sure your layer is selected before you start drawing out a mask. Otherwise you'll just creating a brand-new shape layer rather than cutting out your desired layer. We're going to initially just click around to get familiar with using the Pen tool then we'll use it more carefully to try to cut out this flower.

If you just click the mouse, move it, and click again, you'll draw straight line segments for your mask shape. However, if you move to another location to press the mouse button and before you release start to dragging, you'll get curved mask shapes. Now you go ahead and drag on this handle to create its length, which is its influence and its angle. I'll click and drag on another point, then drag at another point, at another point. Now it's quite common while you're dragging out a mask shape that you realize may be you made a mistake with one of these previous points.

You can wait till you're done creating a mask. This little O by the Pen tool says you're about to close the mask. And then edit it later. That's what I am going to show you in the next movie. Or if you just can't stand looking at your mistakes you can go ahead and edit it before you've finished a mask. If you want to delete a previous point, one that's not currently selected, just hover your cursor over it. You'll see a little minus symbol appear next to your cursor. Click that and it will delete that point. If you want to add a point along an existing line segment place the cursor over that segment and now it'll turn to a plus symbol.

And now you have a brand-new point. If you want to move the location of a previous mask point or vertex or re-drag one of the handles, you need to hold down Command on Mac or Ctrl on Windows and watch the cursor change to this solid black arrow. Once you have that, you've got the Selection tool which means you can select and relocate your mask points as well as drag out the handles again. When you release the Command or Ctrl key, you get this upside down V cursor. That is the Change Direction tool.

If you were to drag on a handle in this condition you would break the handle and create a discontinuous curve through that segment. Now you don't always get that Change Direction tool just by hovering your cursor over it. If you do want to change direction of a vertex hold down the Option key on Mac or Alt key on Windows and then you'll get that inverted V. Clicking on a straight line segment point will convert it to a rounded segment. Clicking on a rounded segment point we will convert it back to a straight corner again. If you have discontinuous points like I have here where I've dragged out the handle, you'll need to convert it back to either a straight line segment or a curved segment.

Again you'll hold down Option or Alt, get the Change Direction cursor. Click once to get a straight segment; click again to go ahead and get now your Bezier handles. And if you want to edit those Bezier handles, hold down Command on Mac or Ctrl on Windows. In general hover your cursor over existing points or handles. Observe the cursor shape. For example, this inverted V is the Change Direction tool. And then on Mac, use Command and Option or on Windows use Ctrl and Alt to change the purpose of that tool. Solid arrow is the Selection tool.

Inverted V is the Change Direction tool. To continue drawing your mask, you need to reselect your last point that you drew along your mask shape, which means to holding down Command or Ctrl, clicking on it, and now you've selected it again. And now you can go ahead and continue clicking at your mask shape. To close your mask path hover the cursor over your first mask point. It's the one with 2 boxes around it instead of one. You'll see a little O appear next to the Pen tool and that indicates you're about to close your mask path.

Click and now you've got a closed mask shape that reveals just the portion of the layer that's inside that mask. By the way, you don't need to create closed mask shapes. We'll show you later on. A good reason is to create open that mask shapes. To do that, when you're almost ready to close, just switch back to the Selection tool and now you have an open mask path. So now that you have a general idea, I am going to give you a quick demonstration of how to do masking for real. And I'll go ahead and select the mask and delete the whole shape.

I am going to go ahead and zoom up to 200% so I can see my flower in greater detail. Hold down the spacebar to temporarily switch to the Hand tool and center it up. Select my Pen tool and start dragging. Okay. Let's say I want to start here. I've got a sharp corner so I only need to click once to create a sharp corner there. And I'll start dragging out some little curves right along this edge as I go.

If I have a sharper corner I'll have very short handles. If I have a more sweeping corner, I'll go ahead and drag out longer handles. If I have a discontinuous edge like a corner, I'll just do single click to add a point without any handles. Let's just keep dragging here out. That's kind of smooth. It has kind of a square edge there. Once I get these longer segments, I'll go ahead and drag out much longer handles for longer influence, like that.

Hold down spacebar to temporarily get the Hand tool and keep dragging along the edge of my tulip. Sharp corner there, sharp corner there, and continue to draw around the shape. I am not going to make you sit there and watch me draw all the rest of this mask shape. However, it would be educational to see how I correct this mask shape and that's what I am going to do in the next movie.

Find answers to the most frequently asked questions about After Effects Apprentice 05: Creating Transparency.


Expand all | Collapse all
please wait ...
Q: This course was updated on 11/15/2012. What changed?
A: We added a movie that shows our premium subscribers how to use the exercise files, We have also added new sets of exercise files designed for After Effects CS5.5 and After Effects CS6.
 
Share a link to this course

What are exercise files?

Exercise files are the same files the author uses in the course. Save time by downloading the author's files instead of setting up your own files, and learn by following along with the instructor.

Can I take this course without the exercise files?

Yes! If you decide you would like the exercise files later, you can upgrade to a premium account any time.

Become a member Download sample files See plans and pricing

Please wait... please wait ...
Upgrade to get access to exercise files.

Exercise files video

How to use exercise files.

Learn by watching, listening, and doing, Exercise files are the same files the author uses in the course, so you can download them and follow along Premium memberships include access to all exercise files in the library.
Upgrade now


Exercise files

Exercise files video

How to use exercise files.

For additional information on downloading and using exercise files, watch our instructional video or read the instructions in the FAQ.

This course includes free exercise files, so you can practice while you watch the course. To access all the exercise files in our library, become a Premium Member.

join now Upgrade now

Are you sure you want to mark all the videos in this course as unwatched?

This will not affect your course history, your reports, or your certificates of completion for this course.


Mark all as unwatched Cancel

Congratulations

You have completed After Effects Apprentice 05: Creating Transparency.

Return to your organization's learning portal to continue training, or close this page.


OK
Become a member to add this course to a playlist

Join today and get unlimited access to the entire library of video courses—and create as many playlists as you like.

Get started

Already a member?

Become a member to like this course.

Join today and get unlimited access to the entire library of video courses.

Get started

Already a member?

Exercise files

Learn by watching, listening, and doing! Exercise files are the same files the author uses in the course, so you can download them and follow along. Exercise files are available with all Premium memberships. Learn more

Get started

Already a Premium member?

Exercise files video

How to use exercise files.

Ask a question

Thanks for contacting us.
You’ll hear from our Customer Service team within 24 hours.

Please enter the text shown below:

The classic layout automatically defaults to the latest Flash Player.

To choose a different player, hold the cursor over your name at the top right of any lynda.com page and choose Site preferencesfrom the dropdown menu.

Continue to classic layout Stay on new layout
Exercise files

Access exercise files from a button right under the course name.

Mark videos as unwatched

Remove icons showing you already watched videos if you want to start over.

Control your viewing experience

Make the video wide, narrow, full-screen, or pop the player out of the page into its own window.

Interactive transcripts

Click on text in the transcript to jump to that spot in the video. As the video plays, the relevant spot in the transcript will be highlighted.

Are you sure you want to delete this note?

No

Thanks for signing up.

We’ll send you a confirmation email shortly.


Sign up and receive emails about lynda.com and our online training library:

Here’s our privacy policy with more details about how we handle your information.

Keep up with news, tips, and latest courses with emails from lynda.com.

Sign up and receive emails about lynda.com and our online training library:

Here’s our privacy policy with more details about how we handle your information.

   
submit Lightbox submit clicked
Terms and conditions of use

We've updated our terms and conditions (now called terms of service).Go
Review and accept our updated terms of service.