Easy-to-follow video tutorials help you learn software, creative, and business skills.Become a member

Perfecting a mask with overlay painting

From: Photoshop CS6 One-on-One: Advanced

Video: Perfecting a mask with overlay painting

All right! So, starting out from where we left off a couple of movies ago, our layer mask is in pretty darn good shape. However, it still needs some help, especially along the front of this guy's face. So we're going to be manually enhancing the mask using a special overlay painting trick that's really great for retouching masks in general, and I'll also pass along a few keyboard tricks for switching between the full color composition and the layer mask. Now, in my case, the layer mask is active here inside the Layers panel. I want to view it by itself, and you can do that by Alt+Clicking or Option+Clicking on the layer mask thumbnail.

Perfecting a mask with overlay painting

All right! So, starting out from where we left off a couple of movies ago, our layer mask is in pretty darn good shape. However, it still needs some help, especially along the front of this guy's face. So we're going to be manually enhancing the mask using a special overlay painting trick that's really great for retouching masks in general, and I'll also pass along a few keyboard tricks for switching between the full color composition and the layer mask. Now, in my case, the layer mask is active here inside the Layers panel. I want to view it by itself, and you can do that by Alt+Clicking or Option+Clicking on the layer mask thumbnail.

Then if you zoom in on this guy's nose, and lips, and chin, you can see that we've got a lot of gray pixels, and some rough edges remain as well. So I am going to firm those up by switching over to the Brush tool, which you can get by pressing the B key, and then I will right-click inside my image, and as opposed to painting with a Hardness of 100%, I'll crank that Hardness value down to 0%, and then I will press the Enter key, or the Return key on the Mac. Now let's say I want to make those dark edges nice and jet black. I'll press the X key so that my foreground color is black, and then if I start painting, you will see that I paint in a fuzzy brushstroke, which is no good.

I don't want that. So I will press Control+Z, or Command+Z on a Mac. What I want to do is limit my brushstroke to just the darkest details, and I can do that by changing the blend mode assigned to the stroke. The best blend mode for this kind of thing is Overlay. So I will go ahead and choose Overlay, and notice now, if I apply that same brushstroke -- I'm just brushing in along that edge, along the dark side of the edge -- notice that I'm darkening up the darkest pixels, and I'm leaving the white area alone. If I paint inside of white with black, I do nothing when working in the Overlay mode.

Then if I want to bolster the bright details on the other side, I'd press the X key to make my foreground color white, and I paint along the front of the nose, for example, in order to achieve this effect here. And I will paint down into the lip as well, and maybe into the chin too, maybe down here just a little bit; just give it a click. Now I will press the X key in order to switch back to black, and I will paint along the front of the glasses, and up into the forehead. Then I will press the X key to switch back to white, and I will paint down on the other side of the forehead. That gets us to about 50% of the way there.

Now I want to show you some tricks for working with layer masks. If you switch over to the Channels panel, you'll see that the layer mask turns up as a temporary alpha channel, and we can see that it's selected, and only that alpha channel is visible; the RGB image is hidden. We've got some keyboard shortcuts. So you've got Control+backslash or Command+backslash to switch to the layer mask. If I press Command+2 or Control+2, then I switch back to the full color composite. Notice that switches things in the Layers panel as well. So now the layer mask thumbnail is active, because I pressed Command+2 or Control+2.

If I press Command+backslash or Control+backslash, then I switch over to layer mask, so I'm editing it. You can also control the visibility of the mask by pressing the backslash key on its own. And notice now that I'm seeing the layer mask as a rubylith overlay, just as I do when working in the Quick Mask mode. If I want to change the color of that overlay, I'd switch over to the Channels panel, and double-click on that layer mask thumbnail there, and then I could click on the color swatch, and dial in something more complementary, such as a Hue value of 180 degrees, click OK, then click OK again, and that changes the color of that overlay.

If I now want to hide the image, and view the mask by itself, I press the Tilde key; that same keyboard trick that works in the Quick Mask mode. Then to switch back to the composite image, I'd press the Tilde key again to see both the image and the mask at the same time, and then I'd press the backslash key, so that I'm no longer seeing the mask, and you can see that that controls the visibility of the various channels here inside the Channels panel. So the layer mask is still active, but we're seeing the RGB image by itself. What that allows me to do is clean up some of these bad details right there.

So I am going to increase the Hardness of my brush to 100% by right-clicking inside the image window and cranking up that Hardness value. Then I will press the Enter key, or the Return key on a Mac, to hide the panel, and I'll switch the blend mode up here back to Normal by pressing Shift+Alt+N, or Shift+Option+N on the Mac. And then, with white as my foreground color, I'll just go ahead and paint like this, because this is a very ratty edge indeed that I want to paint away, and then I will press the X key to make black my foreground color, and I'll just go ahead and click inside this region here a few times in order to reinstate a better edge for that nose.

I might also press the L key to switch to Lasso tool, press and hold the Alt key, or the Option key on a Mac, and then dig my way into that corner inside the lips, and press Alt+Backspace or Option+Delete to fill it with black. And I might need to do something similar down here. I will press Control+D, or Command+D on a Mac, to deselect the image, then Alt+Click around this region to select it, press Alt+Backspace or Option+Delete, and then press Control+D, or Command+D on a Mac, to go ahead and add in a nice sharp corner. We've got a little weirdness going right there in that chin, so I am going to press the backslash key to view the layer mask, and press the Tilde key to hide the image, so I can see the mask by itself.

I will press the B key to switch back to my Brush tool, press the X key to switch the foreground color to white, and then click in that chin in order to get rid of that bad edge right there. We need to do some work on these glasses as well. That's what I am looking at in this region. And I might go ahead and press the X key to switch my foreground color to black, and paint into the glasses, like so. I am just going to paint up and over them. And now let's go ahead and switch back to the full color image by pressing the Tilde key to view the image, and then pressing the backslash key so that I can see the full color image by itself.

Actually, if I press the backslash key again, pressing it to turn the layer mask on and off, it looks like I'm doing a pretty good job, actually, of selecting the image at this point. That's because, if I switch back to Layers panel, I'd have to Shift+Click on the layer mask to see what the image looks like unmasked. So Shift+Clicking on that layer mask thumbnail turns it off. I know these are a bunch of keyboard shortcuts, but they're really darn useful if you can remember them. I am going to press the L key in order to switch to the Lasso tool, and then press and hold the Alt key, or the Option key on a Mac, and click around this area of the sunglasses like so in order to select them.

Then I'll go ahead and smooth that selection. I could do that using the Refine Edge command, but I am just going to go up to the Select menu, choose Modify, and choose Smooth, and then I will enter a Sample Radius value of 2 pixels here. And you can see, there's no preview; I just have to hope for the best, and click OK. That's probably going to end up working out nicely. I will Shift+Click on the layer mask to turn it back on, so we can see the results of that mask, and I will press Control+Backspace, or Command+ Delete on the Mac, to fill the selection with the background color, which is white for me. And then I will press Control+Shift+I or Command+Shift+I, as long as I am working with keyboard shortcuts, to reverse the selection.

So I've got the area outside of the marching ants selected. I will press and hold the Shift and Alt keys, or the Shift and Option keys on the Mac, and I will drag around this little area right there in order to select that region, and I will press Alt+Backspace, or Option+Delete on the Mac, to fill it with black, so that the sunglasses are in better shape. And I should probably just go ahead and select this region at the top of the nose as well, like so, and I am Alt+Clicking or Option+Clicking with the Lasso tool to, draw that polygonal selection outline and I will press Alt+Backspace, or Option+Delete on the Mac, to fill it with black, and we have a better edge there.

All right, let's check out what else we've got to work with here. I will Alt+Click or Option+Click on the layer mask thumbnail to view the layer mask independently of the image. It looks like we need to firm up that stuff. So I will press the B key in order to switch to my Brush tool, right-click inside the image, crank the Hardness value down to 0%, press the Enter or Return key to hide that panel. And this time, I'm going to switch to the Soft Light mode. So if you find the Overlay mode is too much, don't reduce the Opacity value; instead, switch to Soft Light, so you can make more subtle modifications.

Then I will go ahead and paint in that region in order to paint it away. Let's try out a few more adjustments. So I will go ahead and press Control+0, or Command+0 on a Mac, to zoom out from the image. I will Alt+Click or Option+Click on the layer mask thumbnail, so I can see the image at the same time as the mask. I am still editing the mask, however, and I am going to press the X key to switch my foreground color to white, and I am going to paint back in some of his hair. So I am just painting along the back of the hair, like so. Then finally, we've got these weird edges right there inside the hair, and I will Alt+Click or Option+Click on the layer mask, so you can see what I am talking about. A bunch of weird little sort of snivels that are showing up here in front of his 'do.

So I will right-click inside the image window. I am going to take that Hardness value up to 75%, press the Enter key, or the Return key on a Mac, to hide that panel. Then I will press Shift+Alt+N, or Shift+Option+N on the Mac, to switch back to the Normal blend mode. I will reduce the size of my cursor, and paint some of this stuff away, like so. Now, that may have kind of messed up the mask a little bit. So I will Alt+Click or Option+Click on that layer mask thumbnail again, and sure enough, I am bringing back some of that bad background. So I am going to press Control+Alt+R, or Command+Option+R on a Mac, to bring up the Refine Mask dialog box.

And this time, I am just going to paint in a little bit of edge detection using that Refine Radius tool, which is selected automatically, and I will just paint in this region, like so, and release, and that goes ahead and restores the blue background. Then I will click OK in order to accept that change. Now let's just take a tour of things. I will Alt+Click or Option+Click on the layer mask thumbnail, so that we can view the layer mask independently of the image, and then I'll just kind of tour around the edges to make sure we got everything. This looks pretty good.

We can probably make that edge a little better, but I'm not too worried about it. If you end up with some overly sharp edges, by the way, such as in this area, here is what you do; Alt+Click or Option+Click on the layer mask thumbnail, so you can see the image at the same time, and switch to the Blur tool. And you can get the Blur tool from this flyout menu right there. If you loaded my dekeKeys keyboard shortcuts, it's also available by pressing the N key. I will reduce the size of my cursor a little bit, and then I will go ahead and paint along the lips, like so, and I will paint down his chin as well.

We end up with what I consider to be some more naturalistic, organic transitions right there. Paint along the top of his nose as well. Otherwise, I think we're in pretty good shape. I might paint along the top of his forehead. It's really just the front of his face that was in kind of rough shape. And be sure, by the way, very important that you're painting inside the layer mask, not inside the image, otherwise you will end up blurring the details inside the photographic image, which is definitely not something you want. That's how you make manual modifications to a layer mask by overlay painting with the Brush tool; that is, setting the Brush tool to either the Overlay or Soft Light blend modes, as well as how you view the image and the layer mask, either independently, or at the same time here inside Photoshop.

Show transcript

This video is part of

Image for Photoshop CS6 One-on-One: Advanced
Photoshop CS6 One-on-One: Advanced

124 video lessons · 20115 viewers

Deke McClelland
Author

 
Expand all | Collapse all
  1. 30m 4s
    1. Welcome to One-on-One
      2m 19s
    2. Loading the dekeKeys keyboard shortcuts
      6m 5s
    3. Remapping your Macintosh OS shortcuts
      3m 4s
    4. Adjusting a few general preferences
      4m 3s
    5. Using the visual HUD color picker
      2m 2s
    6. The interface and performance settings
      5m 31s
    7. Adjusting the color settings in Photoshop
      7m 0s
  2. 47m 0s
    1. Smart Objects
      1m 36s
    2. Three ways to place a Smart Object
      3m 6s
    3. Copying and pasting from Adobe Illustrator
      4m 11s
    4. Transforming and warping a vector object
      4m 48s
    5. Blending a Smart Object into a photograph
      3m 10s
    6. Blurring with a nested Smart Filter
      4m 57s
    7. Editing a Smart Object in Illustrator
      3m 20s
    8. Creating "true clones"
      3m 50s
    9. Duplicating a group of clones
      2m 53s
    10. Breaking the Smart Object link
      2m 53s
    11. Styling and blending Smart Objects
      2m 44s
    12. Editing originals; updating clones
      3m 41s
    13. Removing people from a scene with Median
      5m 51s
  3. 29m 59s
    1. Luminance meets sharpening
      1m 2s
    2. Correcting for lens distortion
      4m 39s
    3. Introducing Shadows/Highlights
      3m 54s
    4. Mitigating halos with Radius values
      4m 19s
    5. Enhancing the effects of Midtone Contrast
      3m 18s
    6. Creating a "bounce" with Gaussian Blur
      3m 29s
    7. Sharpening on top of blur
      2m 47s
    8. Masking a group of Smart Filters
      2m 53s
    9. Reducing the density of a layer mask
      3m 38s
  4. 49m 10s
    1. Using Curves
      2m 40s
    2. Introducing the Curves adjustment
      7m 36s
    3. Adding and editing points on a curve
      6m 27s
    4. Winning Curves tips and tricks
      8m 12s
    5. Correcting a challenging image
      6m 33s
    6. Selecting and darkening highlights
      4m 39s
    7. Neutralizing colors and smoothing transitions
      6m 6s
    8. The new automatic Curves function
      6m 57s
  5. 1h 31m
    1. Camera Raw
      2m 11s
    2. Opening and editing multiple images
      8m 1s
    3. Correcting white balance
      4m 8s
    4. The revamped Exposure controls
      8m 8s
    5. Working with archival images
      7m 54s
    6. The Spot Removal and Graduated Filter tools
      6m 4s
    7. Painting edits with the Adjustment Brush
      7m 23s
    8. Tone Curves (and why you don't need them)
      5m 57s
    9. Straighten, crop, and geometric distortions
      5m 17s
    10. Applying manual lens corrections
      5m 14s
    11. Vignette, chromatic aberration, and fringe
      6m 49s
    12. Selective hue, saturation, and luminance
      6m 36s
    13. Working with JPEG and TIFF images
      6m 36s
    14. Camera Raw Smart Objects
      6m 48s
    15. Editing Camera Raw images from Bridge
      4m 24s
  6. 32m 30s
    1. Duotones
      1m 23s
    2. Creating a professional-quality sepia tone
      4m 18s
    3. Introducing the Gradient Map adjustment
      5m 42s
    4. Loading a library of custom gradients
      3m 48s
    5. Creating a custom quadtone
      5m 48s
    6. Colorizing with blend modes and Opacity
      4m 6s
    7. Creating a faux-color, high-key effect
      7m 25s
  7. 1h 6m
    1. Noise vs. Details
      1m 28s
    2. Introducing the Reduce Noise filter
      7m 29s
    3. Correcting a noisy photo
      5m 33s
    4. Smoothing over high-contrast noise
      5m 50s
    5. Protecting details with an edge mask
      4m 52s
    6. Adjusting overly saturated shadows
      3m 35s
    7. Correcting with High Pass and Lens Blur
      3m 45s
    8. Brushing away blur and sharpening
      6m 42s
    9. Creating texture by adding noise
      5m 28s
    10. The Camera Raw Detail panel
      7m 8s
    11. Correcting noise and detail in Camera Raw
      8m 10s
    12. Adding noise grain and vignetting effects
      6m 47s
  8. 44m 30s
    1. Blur Gallery
      1m 36s
    2. Creating depth-of-field effects in post
      5m 29s
    3. Modifying your Field Blur settings
      4m 57s
    4. Editing and exporting a Field Blur mask
      6m 15s
    5. Adding a synthetic light bokeh
      3m 52s
    6. Using the Selection Bleed option
      7m 29s
    7. Creating a radial blur with Iris Blur
      6m 59s
    8. Creating "fake miniatures" with Tilt-Shift
      4m 35s
    9. Combining multiple Blur Gallery effects
      3m 18s
  9. 1h 34m
    1. Blend Modes
      1m 16s
    2. Using the Dissolve mode
      9m 47s
    3. Multiply and the darken modes
      8m 30s
    4. Screen and the lighten modes
      8m 10s
    5. Cleaning up and integrating a bad photo
      6m 38s
    6. Blending inside blend modes
      6m 55s
    7. Overlay and the contrast modes
      6m 53s
    8. A few great uses for the contrast modes
      9m 7s
    9. Difference, Exclusion, Subtract, and Divide
      5m 5s
    10. Capturing the differences between images
      4m 18s
    11. Hue, Saturation, Color, and Luminosity
      4m 45s
    12. Blend mode shortcuts
      6m 21s
    13. The Fill Opacity Eight
      8m 57s
    14. Using the luminance-exclusion slider bars
      8m 8s
  10. 44m 20s
    1. Color Range
      1m 14s
    2. Introducing the Color Range command
      7m 24s
    3. Selecting a complex image with Color Range
      5m 49s
    4. Refining a selection in the Quick Mask mode
      7m 4s
    5. Viewing a mask with or without its image
      4m 24s
    6. Painting directly inside an alpha channel
      5m 39s
    7. Correcting fringes around a masked layer
      8m 5s
    8. Turning a layer into a knockout
      4m 41s
  11. 59m 43s
    1. Refine Edges
      1m 28s
    2. Laying down a base layer mask
      6m 49s
    3. Introducing the Refine Edge/Mask command
      7m 57s
    4. Edge detection and Smart Radius
      4m 42s
    5. Using the Refine Radius tool
      7m 31s
    6. The transformative power of Refine Edge
      3m 37s
    7. Perfecting a mask with overlay painting
      10m 58s
    8. Combining Quick Selection with Refine Mask
      10m 37s
    9. Bolstering and integrating hair
      6m 4s
  12. 1h 18m
    1. The Pen tool
      1m 50s
    2. Pixel-based masking versus the Pen tool
      6m 45s
    3. Drawing a straight-sided path outline
      6m 57s
    4. Moving, deleting, and adding anchor points
      6m 10s
    5. Dragging control handles to modify curves
      5m 27s
    6. Converting a path outline to a vector mask
      5m 36s
    7. Customizing a geometric shape
      5m 53s
    8. How to position points and control handles
      7m 7s
    9. Drawing smooth points with the Pen tool
      8m 7s
    10. Duplicating and scaling a vector mask
      5m 21s
    11. Cusp points and the Rubber Band option
      6m 21s
    12. Setting anchor points in the pasteboard
      6m 8s
    13. Using the Convert Point tool
      6m 43s

Start learning today

Get unlimited access to all courses for just $25/month.

Become a member
Sometimes @lynda teaches me how to use a program and sometimes Lynda.com changes my life forever. @JosefShutter
@lynda lynda.com is an absolute life saver when it comes to learning todays software. Definitely recommend it! #higherlearning @Michael_Caraway
@lynda The best thing online! Your database of courses is great! To the mark and very helpful. Thanks! @ru22more
Got to create something yesterday I never thought I could do. #thanks @lynda @Ngventurella
I really do love @lynda as a learning platform. Never stop learning and developing, it’s probably our greatest gift as a species! @soundslikedavid
@lynda just subscribed to lynda.com all I can say its brilliant join now trust me @ButchSamurai
@lynda is an awesome resource. The membership is priceless if you take advantage of it. @diabetic_techie
One of the best decision I made this year. Buy a 1yr subscription to @lynda @cybercaptive
guys lynda.com (@lynda) is the best. So far I’ve learned Java, principles of OO programming, and now learning about MS project @lucasmitchell
Signed back up to @lynda dot com. I’ve missed it!! Proper geeking out right now! #timetolearn #geek @JayGodbold
Share a link to this course

What are exercise files?

Exercise files are the same files the author uses in the course. Save time by downloading the author's files instead of setting up your own files, and learn by following along with the instructor.

Can I take this course without the exercise files?

Yes! If you decide you would like the exercise files later, you can upgrade to a premium account any time.

Become a member Download sample files See plans and pricing

Please wait... please wait ...
Upgrade to get access to exercise files.

Exercise files video

How to use exercise files.

Learn by watching, listening, and doing, Exercise files are the same files the author uses in the course, so you can download them and follow along Premium memberships include access to all exercise files in the library.


Exercise files

Exercise files video

How to use exercise files.

For additional information on downloading and using exercise files, watch our instructional video or read the instructions in the FAQ .

This course includes free exercise files, so you can practice while you watch the course. To access all the exercise files in our library, become a Premium Member.

Are you sure you want to mark all the videos in this course as unwatched?

This will not affect your course history, your reports, or your certificates of completion for this course.


Mark all as unwatched Cancel

Congratulations

You have completed Photoshop CS6 One-on-One: Advanced.

Return to your organization's learning portal to continue training, or close this page.


OK
Become a member to add this course to a playlist

Join today and get unlimited access to the entire library of video courses—and create as many playlists as you like.

Get started

Already a member ?

Become a member to like this course.

Join today and get unlimited access to the entire library of video courses.

Get started

Already a member?

Exercise files

Learn by watching, listening, and doing! Exercise files are the same files the author uses in the course, so you can download them and follow along. Exercise files are available with all Premium memberships. Learn more

Get started

Already a Premium member?

Exercise files video

How to use exercise files.

Ask a question

Thanks for contacting us.
You’ll hear from our Customer Service team within 24 hours.

Please enter the text shown below:

The classic layout automatically defaults to the latest Flash Player.

To choose a different player, hold the cursor over your name at the top right of any lynda.com page and choose Site preferences from the dropdown menu.

Continue to classic layout Stay on new layout
Exercise files

Access exercise files from a button right under the course name.

Mark videos as unwatched

Remove icons showing you already watched videos if you want to start over.

Control your viewing experience

Make the video wide, narrow, full-screen, or pop the player out of the page into its own window.

Interactive transcripts

Click on text in the transcript to jump to that spot in the video. As the video plays, the relevant spot in the transcript will be highlighted.

Learn more, save more. Upgrade today!

Get our Annual Premium Membership at our best savings yet.

Upgrade to our Annual Premium Membership today and get even more value from your lynda.com subscription:

“In a way, I feel like you are rooting for me. Like you are really invested in my experience, and want me to get as much out of these courses as possible this is the best place to start on your journey to learning new material.”— Nadine H.

Thanks for signing up.

We’ll send you a confirmation email shortly.


Sign up and receive emails about lynda.com and our online training library:

Here’s our privacy policy with more details about how we handle your information.

Keep up with news, tips, and latest courses with emails from lynda.com.

Sign up and receive emails about lynda.com and our online training library:

Here’s our privacy policy with more details about how we handle your information.

   
submit Lightbox submit clicked
Terms and conditions of use

We've updated our terms and conditions (now called terms of service).Go
Review and accept our updated terms of service.