New Feature: Playlist Center! Pick a topic and let our playlists guide the way.

Easy-to-follow video tutorials help you learn software, creative, and business skills.Become a member

Painting with the Overlay mode

From: Photoshop Masking and Compositing: Fundamentals

Video: Painting with the Overlay mode

In this exercise, we're going to take our mask so far, and we're going to selectively enhance its contrast using a combination of the Brush tool, along with a couple of different blend modes. I've saved my progress as Blue channel mask.tif. And the reason I used the TIFF format, by the way, is because we have an alpha channel, so JPEG is not going to work. We also have a flat image file. Notice all we've is the background here inside the Layers panel, which means that the TIFF format, combined with LZW compression, is going to produce a smaller, more efficient file than the native PSD format.

Painting with the Overlay mode

In this exercise, we're going to take our mask so far, and we're going to selectively enhance its contrast using a combination of the Brush tool, along with a couple of different blend modes. I've saved my progress as Blue channel mask.tif. And the reason I used the TIFF format, by the way, is because we have an alpha channel, so JPEG is not going to work. We also have a flat image file. Notice all we've is the background here inside the Layers panel, which means that the TIFF format, combined with LZW compression, is going to produce a smaller, more efficient file than the native PSD format.

I'm going to switch back to Channels. Notice that the alpha channel is selected. Now, as I say, we're going to be completing this mask by hand painting inside of it. So your best bet is to go ahead and duplicate the channel at this point. So I'm going to go ahead and drag it, and drop it onto the little Page icon at the bottom of the Channels panel. Because, after all, when you're working with layers, you can make nondestructive modifications, but when you're working with alpha channels, there are no layers, so every change you make is technically destructive, especially hand painting, as you're about to see.

I'm going to rename this channel overlay painting, because we're going to be comparing a couple of different techniques. Then I'll go ahead and switch to the Brush tool, which you can get by selecting it, or pressing the B key. Notice that my Brush is set to 13 pixels, as by default, so it's very tiny. What I want to do is paint away the regions of her face inside the white silhouette, so I need a much larger, harder brush. So I'm going to right-click inside the image window, and I'm going to take the Size value up to a whopping thousand pixels, and then I'll take the Hardness value up to 100%, and I'll press the Enter key, or the Return key on the Mac, a couple of times in order to accept that change.

I want you to notice something. See where my cross-shaped cursor is? That little bit of gray right there? That's an area of revealed background. If I press Control+2, or Command+2 on the Mac, to switch back to the RGB image, you can see that we have some white background between the hair and her cheek. So we want to make sure to retain that area. However, everything else falls inside of her face, and therefore it needs to be painted away. I'll press Control+7, or Command+7 on the Mac, in order to switch back to my alpha channel at hand, and you can see that's its keyboard shortcut, because it's listed there inside the Channels panel.

Press the D key in order to restore your default masking colors, which are white for the foreground color, and black for the background color. And then just go ahead and click where you see me click inside of the image window, and that paints away her face. Now, if you want to confirm that the entire interior of this white silhouette is absolutely white, then go ahead and grab the Magic Wand tool, and set the Tolerance value to 0, like so, and then make sure Anti-alias is turned off, and click inside the face. In my case, I just have this little bit of chin that I need to get rid of.

So I'll press the M key in order to switch to the Rectangular Marquee tool, and then I'll Shift+drag around that little bit of animated selection outline in order to select it. And because white is my foreground color, I'll press Alt+Backspace, or Option+Delete on the Mac, in order to fill that entire area with white. Now let's press Control+D, or Command+ D on a Mac, to deselect the image. I'm going to make yet another duplicate. Now that I've gotten rid of the face, which is a fine edit, by the way, no problem so far, I'll go ahead and duplicate this most recent channel by dragging it, and dropping it onto the Page icon. And I'll rename this guy soft light painting, this time around, because Soft Light and Overlay are different Blend modes and produce different effects, as you're about to see.

Let's start things off with overlay painting. Now, this is a pretty common technique. It's gained a lot of popularity over the years, and I'll show you why. I'm going to go ahead and switch back to the Brush tool. Let's say what we want to do at this point is we want to paint that background black, because it needs to be absolutely jet black in order for this alpha channel to serve as the mask. So I'll press the X key to switch my foreground color to black, and then if I were to paint, like so, well I guess I would paint my background black, of course, but I get these big chunky lumps around my brushstroke, and I have no way of actually painting into the details here, except for being very, very careful.

Then I would have to paint around the hair. That, of course, would take me forever. So I'm going to go ahead and undo those last brushstrokes by pressing Control+Alt+Z, or Command+Option+Z, a few times in a row. And what you want to do instead: two parts. First of all, right-click inside the image window in order to bring up the Brush panel, and take the Hardness value down to 0%. I'm also going to increase the Size value to, let's say, 500 pixels for now. Now, I have been warning you against painting with a soft brush inside of an alpha channel, because after all, that does give you certain amount of wiggle room and latitude, but you also introduce all this grayness, and this softness, and these artificially feathered details as you paint.

So obviously, that's not the way to go. Press Control+Z, Command+Z on the Mac, to undo that change. Here is what you do: go to the Blend mode menu, and choose the first of the contrast modes: Overlay. What that allows you to do is preserve the opposite luminance level. So in other words, if I choose Overlay, and I paint with black, then I will darken the dark stuff, but white, the opposite luminance level, will be absolutely protected. So let's go ahead and zoom out a click here.

I'm going to increase the size of my cursor as well, by pressing the right bracket key a few times, and then I'll just go ahead and paint fairly sloppily around the right-hand side of the image, and notice what a number that does. Not only does it make that background jet black, but Photoshop also goes ahead and protects the white areas inside the silhouette. Now all I have to do is paint along the left- hand side of the image, like so, and I'm done. That's it, because again, Photoshop is protecting that opposite color.

The question is, how good of a job did it do at protecting those details? I'm going to press the M key to switch back to my Rectangular Marquee tool, just so I don't have that big, huge Brush tool on screen. I'm going to go ahead and zoom in until I get to 100%, so we can see those hair details down here on the lower right region of the model's head. Notice that they're pretty crunchy at this point. Now, it would only get worse if I were to grab my Brush tool, and click again at this location. Let's go ahead and decrease the Size of the brush by pressing the left bracket key a few times.

If I were to click and drag inside this region, notice that I get even chunkier, more jagged transitions around those hair details. I'll go ahead and press Control+Z, or Command+ Z on the Mac, so you can see the difference. So it looks a lot better before. Every application, in other words, of the Brush tool set to the Overlay Blend mode ends up further exaggerating the contrast. Now, you might figure the trick then -- I'll go ahead and switch back to the Brush tool. you might figure the trick is to reduce the Opacity value.

That actually is not the trick. That does not work very well, because what ends up happening is you take those very, very dark grays, and you make them darker, but you never quite make them black. Instead, the solution is to switch to a different Blend mode entirely, and I'm going to show you how that works, and to compare the different results, inside the next exercise.

Show transcript

This video is part of

Image for Photoshop Masking and Compositing: Fundamentals
Photoshop Masking and Compositing: Fundamentals

128 video lessons · 28924 viewers

Deke McClelland

Expand all | Collapse all
  1. 15m 25s
    1. Welcome
      1m 12s
    2. Loading my custom dekeKeys shortcuts
      3m 45s
    3. Adjusting the color settings
      4m 29s
    4. Setting up a power workspace
      5m 59s
  2. 1h 0m
    1. The channel is the origin of masking
      1m 54s
    2. The Masks and Channels panels
      4m 48s
    3. How color channels work
      7m 7s
    4. Viewing channels in color
      3m 24s
    5. How RGB works
      4m 12s
    6. Single-channel grayscale
      5m 12s
    7. Mixing a custom "fourth" channel
      5m 15s
    8. The other three-channel mode: Lab
      5m 45s
    9. A practical application of Lab
      4m 55s
    10. The final color mode: CMYK
      7m 5s
    11. Introducing the Multichannel mode
      5m 56s
    12. Creating a unique multichannel effect
      5m 18s
  3. 44m 27s
    1. The alpha channel is home to the mask
      1m 40s
    2. The origins of the alpha channel
      3m 40s
    3. How a mask works
      7m 10s
    4. Making an alpha channel
      4m 2s
    5. Using the new channel icons
      6m 27s
    6. Saving an image with alpha channels
      4m 23s
    7. Loading a selection from a channel
      4m 7s
    8. Putting a mask into play
      3m 55s
    9. Loading a selection from a layer
      4m 27s
    10. Loading a selection from another image
      4m 36s
  4. 1h 0m
    1. The mask meets the composition
      1m 8s
    2. Viewing a mask as a rubylith overlay
      6m 13s
    3. Changing a mask's overlay color
      5m 34s
    4. Painting inside a mask
      6m 3s
    5. Cleaning up and confirming
      5m 18s
    6. Combining masks
      5m 10s
    7. Painting behind and inside a layer
      5m 27s
    8. Blending image elements
      6m 1s
    9. What to do when layers go wrong
      6m 3s
    10. Hiding layer effects with a mask
      4m 22s
    11. Introducing clipping masks
      5m 29s
    12. Unclipping and masking a shadow
      3m 50s
  5. 1h 35m
    1. The seven selection soldiers
    2. The marquee tools
      6m 31s
    3. The single-pixel tools (plus tool tricks)
      6m 48s
    4. Turning a destructive edit into a layer
      5m 34s
    5. Making shapes of specific sizes
      7m 7s
    6. The lasso tools
      5m 49s
    7. Working with the Magnetic Lasso tool
      7m 19s
    8. The Quick Selection tool
      8m 13s
    9. Combining Quick Selection and Smudge
      4m 52s
    10. The Magic Wand and the Tolerance value
      6m 55s
    11. Contiguous and Anti-aliased selections
      6m 58s
    12. Making a good selection with the Magic Wand
      6m 34s
    13. Selecting and replacing a background
      6m 55s
    14. Resolving edges with layer effects
      7m 52s
    15. Adding lines of brilliant gold type
      7m 28s
  6. 1h 11m
    1. Selections reign supreme
    2. Introducing "selection calculations"
      4m 19s
    3. Combining two different tools
      7m 29s
    4. Selections and transparency masks
      5m 17s
    5. Selecting an eye
      7m 1s
    6. Masking and blending a texture into skin
      5m 1s
    7. Painting a texture into an eye
      4m 19s
    8. Combining layers, masks, channels, and paths
      4m 54s
    9. Moving selection outlines vs. selected pixels
      5m 36s
    10. Transforming and warping a selection outline
      7m 45s
    11. Pasting an image inside a selection
      7m 26s
    12. Adding volumetric shadows and highlights
      6m 54s
    13. Converting an image into a mask
      4m 42s
  7. 1h 5m
    1. The best selection tools are commands
      1m 5s
    2. Introducing the Color Range command
      5m 59s
    3. Working in the Color Range dialog box
      7m 7s
    4. Primary colors and luminance ranges
      4m 12s
    5. A terrific use for Color Range
      4m 57s
    6. Introducing the Quick Mask mode
      7m 43s
    7. Moving a selection into a new background
      5m 43s
    8. Smoothing the mask, recreating the corners
      8m 43s
    9. Integrating foreground and background
      4m 44s
    10. Creating a cast shadow from a layer
      2m 51s
    11. Releasing and masking layer effects
      3m 11s
    12. Creating a synthetic rainbow effect
      4m 30s
    13. Masking and compositing your rainbow
      4m 46s
  8. 1h 17m
    1. The ultimate in masking automation
      1m 6s
    2. Introducing the Refine Mask command
      6m 58s
    3. Automated edge detection
      8m 23s
    4. Turning garbage into gold
      6m 19s
    5. Starting with an accurate selection
      7m 11s
    6. Selection outline in, layer mask out
      7m 48s
    7. Matching a scene with Smart Filters
      4m 29s
    8. Cooling a face, reflecting inside eyes
      4m 45s
    9. Creating a layer of ghoulish skin
      4m 28s
    10. Adding dark circles around the eyes
      5m 20s
    11. Creating a fake blood effect
      5m 38s
    12. Establishing trails of blood
      7m 40s
    13. Integrating the blood into the scene
      7m 3s
  9. 1h 48m
    1. Using the image to select itself
      1m 37s
    2. Choosing the ideal base channel
      5m 7s
    3. Converting a channel into a mask
      6m 34s
    4. Painting with the Overlay mode
      7m 27s
    5. Painting with the Soft Light mode
      5m 55s
    6. Mask, composite, refine, and blend
      4m 40s
    7. Creating a more aggressive mask
      7m 2s
    8. Blending differently masked layers
      7m 0s
    9. Creating a hair-only mask
      6m 0s
    10. Using history to regain a lost mask
      3m 42s
    11. Separating flesh tones from hair
      8m 28s
    12. Adjusting a model's color temperature
      4m 30s
    13. Introducing the Calculations command
      7m 22s
    14. Extracting a mask from a Smart Object
      6m 34s
    15. Integrating a bird into a new sky
      5m 40s
    16. Creating synthetic rays of light
      6m 4s
    17. Masking and compositing light
      7m 39s
    18. Introducing a brilliant light source
      7m 5s
  10. 1h 34m
    1. The synthesis of masking and compositing
      1m 36s
    2. White reveals, black conceals
      6m 45s
    3. Layer masking tips and tricks
      5m 8s
    4. Generating a layer mask with Color Range
      5m 38s
    5. The Masks panel's bad options
      5m 18s
    6. The Masks panel's good options
      3m 50s
    7. Creating and feathering a vector mask
      3m 42s
    8. Combining pixel and vector masks
      3m 50s
    9. Working with path outlines
      7m 10s
    10. Combining paths into a single vector mask
      7m 52s
    11. Sharpening detail, reducing color noise
      4m 27s
    12. Recreating missing details
      8m 49s
    13. Masking glass
      5m 50s
    14. Refining a jagged Magic Wand mask
      5m 53s
    15. Masking multiple layers at one time
      5m 15s
    16. Establishing a knockout layer
      6m 6s
    17. Clipping and compositing tricks
      7m 37s
  11. 1m 17s
    1. Next steps
      1m 17s

Start learning today

Get unlimited access to all courses for just $25/month.

Become a member
Sometimes @lynda teaches me how to use a program and sometimes changes my life forever. @JosefShutter
@lynda is an absolute life saver when it comes to learning todays software. Definitely recommend it! #higherlearning @Michael_Caraway
@lynda The best thing online! Your database of courses is great! To the mark and very helpful. Thanks! @ru22more
Got to create something yesterday I never thought I could do. #thanks @lynda @Ngventurella
I really do love @lynda as a learning platform. Never stop learning and developing, it’s probably our greatest gift as a species! @soundslikedavid
@lynda just subscribed to all I can say its brilliant join now trust me @ButchSamurai
@lynda is an awesome resource. The membership is priceless if you take advantage of it. @diabetic_techie
One of the best decision I made this year. Buy a 1yr subscription to @lynda @cybercaptive
guys (@lynda) is the best. So far I’ve learned Java, principles of OO programming, and now learning about MS project @lucasmitchell
Signed back up to @lynda dot com. I’ve missed it!! Proper geeking out right now! #timetolearn #geek @JayGodbold

Are you sure you want to delete this note?


Thanks for signing up.

We’ll send you a confirmation email shortly.

Sign up and receive emails about and our online training library:

Here’s our privacy policy with more details about how we handle your information.

Keep up with news, tips, and latest courses with emails from

Sign up and receive emails about and our online training library:

Here’s our privacy policy with more details about how we handle your information.

submit Lightbox submit clicked
Terms and conditions of use

We've updated our terms and conditions (now called terms of service).Go
Review and accept our updated terms of service.