Start your free trial now, and begin learning software, business and creative skills—anytime, anywhere—with video instruction from recognized industry experts.

Start Your Free Trial Now

Mixing layer and vector masks


Photoshop CS3 Channels & Masks: Advanced Techniques

with Deke McClelland

Video: Mixing layer and vector masks

The next step of my adventures is to add the blue waves on the right side of her head that go over her ear. In case you are like a little bit vague about what I am talking about, let's go ahead and switch to the final version of the image. This is fish face, it's actually the Fish face.psd image, the version with the fish in brown, instead of blue. You can see how we have got these wavy lines. They are defined using a shape layer, essentially, which is a vector mask assigned to a dynamic fill layer.
Expand all | Collapse all
  1. 2h 12m
    1. The Odyssey Continues
      2m 39s
    2. Mapping one image onto another
      7m 12s
    3. Making a custom contrast mode
      7m 9s
    4. Luminance blending
      8m 39s
    5. Forcing the visibility of underlying layers
      4m 3s
    6. Adjusting the appearance of clipped layers
      4m 33s
    7. Selecting a Blend If channel
      6m 11s
    8. Enhancing highlights by hiding them
      5m 8s
    9. Smart Object first, layer mask second
      7m 22s
    10. The Fill Opacity Eight
      4m 30s
    11. Blending Smart Filters
      7m 24s
    12. Cleaning up edges
      7m 38s
    13. More fun with luminance blending
      6m 22s
    14. A first peek at the Calculations command
      12m 11s
    15. Masking a softly focused model
      11m 46s
    16. Moving layers and masks between images
      7m 34s
    17. Matching colors
      9m 12s
    18. Building transitional blended layers
      6m 33s
    19. Restoring normal colors
      6m 49s
  2. 2h 32m
    1. Layer masks, clipping masks, and knockouts
      1m 20s
    2. Rotating and stretching
      7m 54s
    3. A jet of motion blur
      9m 17s
    4. Keyboard tricks
      5m 4s
    5. Merging sky and landscape
      6m 2s
    6. Masking an adjustment layer
      6m 37s
    7. Creating two windows into an image
      7m 42s
    8. Whitening teeth and adding other highlights
      3m 46s
    9. Mapping a texture onto an image
      4m 0s
    10. Isolating a texture with a layer mask
      6m 44s
    11. Welcome to the glass composition
      3m 18s
    12. Balancing shadows and highlights
      5m 51s
    13. Masking the glass
      7m 24s
    14. Masking the text
      9m 23s
    15. Adding and blending the goldfish
      8m 44s
    16. Assembling the perfect group photo
      5m 12s
    17. Aligning photographs automatically
      5m 26s
    18. Masking in each person's best shot
      5m 18s
    19. Masking densely packed people
      6m 17s
    20. Crafting the perfect final poster
      5m 15s
    21. From the improbable to the impossible
      1m 56s
    22. The fantastical "world of clones" effect
      10m 0s
    23. Upsampling and blurring a background
      8m 39s
    24. Adding a knockout mask
      8m 3s
    25. Choking edges with Gaussian Blur and Levels
      3m 46s
  3. 2h 26m
    1. The corrective power of masking
      1m 6s
    2. The amazing luminance mask
      7m 21s
    3. Brightening and neutralizing the eyes
      8m 21s
    4. Adjusting a nondestructive composition
      5m 52s
    5. Creating a corrective mask
      6m 3s
    6. Averaging away irregular flesh tones
      3m 52s
    7. Modifying specific colors
      7m 46s
    8. Initiating the color mask
      5m 59s
    9. Refining the color mask
      6m 39s
    10. Adjusting the edges around fabric
      7m 55s
    11. Perfecting hair
      9m 34s
    12. Sharpening with a High Pass layer
      10m 12s
    13. The also-amazing density mask
      4m 47s
    14. Adjusting the knockout depth
      8m 48s
    15. Fashioning a depth map
      6m 12s
    16. Invoking a depth mask from Lens Blur
      6m 38s
    17. The perfect depth-of-field effect
      6m 25s
    18. Sharpening an archival photograph
      7m 7s
    19. Creating an edge mask
      8m 29s
    20. Making a High Pass sandwich
      7m 45s
    21. Applying the edge mask
      6m 1s
    22. Customizing your sharpening effect
      4m 6s
  4. 2h 3m
    1. Channel Mixer, I am your father!
      1m 39s
    2. Three ways to gray
      7m 48s
    3. Meet the Channel Mixer
      8m 26s
    4. Mixing a custom black-and-white image
      7m 9s
    5. Creating a professional-level sepia tone
      5m 36s
    6. Employing the Black & White command
      8m 0s
    7. Extreme channel mixing
      4m 50s
    8. The infrared photography effect
      6m 42s
    9. Taking shadows to the brink of black
      3m 56s
    10. Elevating highlights, leeching saturation
      5m 58s
    11. Deepening a black-and-white sky
      5m 49s
    12. Infusing luminance levels with color
      5m 43s
    13. Creating an opposing colorization scheme
      4m 58s
    14. Bolstering contrast with the Green channel
      5m 37s
    15. A tiny improvement to a terrific technique
      7m 38s
    16. The simple (but wrong) approach to red-eye correction
      6m 39s
    17. Channel-mixing red pupils
      9m 17s
    18. The expert approach to red-eye correction
      5m 20s
    19. Fixing problem coronas (pupil edges)
      8m 8s
    20. Making pupils match
      4m 7s
  5. 2h 33m
    1. Chops are dead; long live maskops
      1m 36s
    2. The Calculations command
      8m 16s
    3. Blue Screen blending
      7m 40s
    4. Refining the Blue Screen mask
      5m 53s
    5. Brushing away color fringing
      7m 24s
    6. Locking the transparency of a layer
      6m 21s
    7. Nondestructive layer painting
      7m 36s
    8. How the Add blend mode works
      8m 40s
    9. How the Subtract blend mode works
      6m 43s
    10. Focus, noise, and other masking challenges
      5m 32s
    11. The Add mode in action
      7m 51s
    12. The Subtract mode in action
      8m 24s
    13. Comparing two channels with Difference
      5m 23s
    14. Enhancing the contrast with Curves
      9m 10s
    15. Gathering details with Apply Image
      9m 43s
    16. Dodge highlights, burn shadows
      6m 6s
    17. Dodge and Burn in action
      8m 24s
    18. Painting in the scalp
      10m 0s
    19. Painting away the face and chin
      4m 52s
    20. Compositing complementary images
      4m 13s
    21. Multiply, Minimum, Blur, and Apply Image
      6m 40s
    22. Crafting the final composition
      7m 7s
  6. 1h 57m
    1. Mark of the Pen tool
      1m 34s
    2. The big paths project overview
      6m 51s
    3. How to make a path
      8m 24s
    4. Corner points and freeform polygons
      8m 6s
    5. Editing paths with the arrow tools
      5m 2s
    6. Adding and deleting endpoints
      5m 14s
    7. Adding and deleting interior points
      6m 6s
    8. Converting a path to a selection
      3m 35s
    9. Converting a path to a mask
      6m 38s
    10. Smooth points and control handles
      8m 57s
    11. Making cusp points
      5m 59s
    12. Combining paths in a vector mask
      7m 54s
    13. Turning a path into a shape layer
      8m 57s
    14. Combining paths to make a layer mask
      7m 52s
    15. Mixing layer and vector masks
      10m 14s
    16. Editing character outlines as paths
      8m 39s
    17. Using the Convert Point tool
      7m 8s
  7. 3h 17m
    1. Where there's a will, there's a way
      1m 18s
    2. Masking natural cast shadows
      4m 9s
    3. Applying the cast show
      4m 2s
    4. Creating a difference mask
      3m 7s
    5. Applying an arbitrary map
      3m 50s
    6. Making the flesh mask
      7m 16s
    7. Roughing in an object mask
      6m 48s
    8. Drawing missing details with the Lasso tool
      4m 7s
    9. Combining flesh and object masks
      3m 52s
    10. Amplifying the cast shadow
      4m 10s
    11. Selectively choking edges
      3m 58s
    12. Power duplication in Photoshop
      7m 8s
    13. Masking blond hair
      5m 48s
    14. Using Levels to mask iterations
      3m 14s
    15. Drawing an iteration boundary
      4m 54s
    16. Merging the best of two Levels iterations
      4m 4s
    17. More fun with Dodge and Burn
      6m 14s
    18. Fixing edges with the Pen and Stamp tools
      7m 28s
    19. Pulling from another file with Apply Image
      4m 52s
    20. Blending clipped layers independently
      5m 42s
    21. Building the flame mask
      9m 21s
    22. Amplifying the flame
      3m 52s
    23. Masking an image against a busy background
      5m 15s
    24. The Freeform and Magnetic Pen tools
      6m 52s
    25. Masking with arbitrary maps
      9m 32s
    26. A more deliberate approach to arb maps
      10m 51s
    27. Combining arb maps with paths
      9m 28s
    28. Masking with the help of the History brush
      11m 38s
    29. Creating a High Pass mask
      7m 24s
    30. Coloring in the outlines
      8m 31s
    31. Mastering Calculations
      7m 28s
    32. Subtracting and merging the beak
      11m 5s
  8. 1h 33m
    1. The meaning of bit depth (and why you care)
      2m 49s
    2. Scanning line art in 8-bit and 16-bit
      5m 8s
    3. Measuring the 16-bit difference
      8m 8s
    4. Correcting 8-bit images in the 16-bit space
      9m 31s
    5. Opening a raw image directly in 16-bit
      6m 12s
    6. Editing in Camera Raw, opening in 16-bit
      8m 21s
    7. 16-Bit/channel vs. 32-bit/channel (HDR)
      8m 17s
    8. Working with auto-bracketed photographs
      5m 6s
    9. Using the Merge to HDR command
      5m 59s
    10. Adjusting the HDR preview
      6m 0s
    11. Building a 32-bit sky mask
      6m 29s
    12. Properly exposing land and sky
      4m 24s
    13. Modifying a layer mask in 32-bit
      4m 56s
    14. Converting to and correcting in 16-bit Lab
      12m 7s
  9. 2h 8m
    1. Photoshop flirts with the third dimension
      1m 13s
    2. The displacement map
      8m 24s
    3. Making custom waves
      7m 14s
    4. Creating a Gaussian distribution
      4m 31s
    5. Using a two-channel displacement map
      6m 27s
    6. Creating a rustic edge effect
      8m 21s
    7. Distorting and shading with a DMap
      6m 33s
    8. Moonlight reflecting off water
      8m 48s
    9. Mapping the reflection onto the water
      7m 3s
    10. Dipping the moon into the water
      6m 18s
    11. Turning flesh into stone
      7m 55s
    12. Wrapping the stone around the face
      7m 27s
    13. Softening a displacement map
      8m 4s
    14. Making a repeating watermark pattern
      9m 21s
    15. 3D embossing with Lighting Effects
      10m 47s
    16. The amazing credit card type effect
      6m 56s
    17. Lightening the credit card letters
      6m 16s
    18. Wrapping the background around the text
      6m 27s
  10. 1m 43s
    1. Goodbye
      1m 43s

please wait ...
Watch the Online Video Course Photoshop CS3 Channels & Masks: Advanced Techniques
20h 48m Advanced Nov 21, 2007

Viewers: in countries Watching now:

The elusive alpha channel remains one of the most misunderstood yet powerful tools in Photoshop. Alpha channels are collections of luminance data that control the transparency of an image, and they inform just about every aspect of Photoshop. As he builds transitional blended layers, fashions a depth map, makes edge adjustments, and takes on extreme channel mixing, Omni Award-winning expert Deke McClelland teaches Photoshop users that where there's a will, there's a way. Photoshop CS3 Channels and Masks: Advanced Techniques covers mapping texture on an image, turning flesh into stone, using vector masks, working with all different channels, creating a rustic edge effect, and much more. Exercise files accompany the tutorials.

Download Deke's customized keyboard layouts for Channels and Masks from the Exercise Files tab."

Topics include:
  • Distorting and shading with a DMap
  • Understanding bits and channels
  • Creating paths with the Pen tool
  • Using blend modes and the Dodge and Burn feature
  • Understanding channel mixing
  • Using layer masks, clipping masks, and knockouts
  • Applying Smart Filters
Design Photography
Deke McClelland

Mixing layer and vector masks

The next step of my adventures is to add the blue waves on the right side of her head that go over her ear. In case you are like a little bit vague about what I am talking about, let's go ahead and switch to the final version of the image. This is fish face, it's actually the Fish face.psd image, the version with the fish in brown, instead of blue. You can see how we have got these wavy lines. They are defined using a shape layer, essentially, which is a vector mask assigned to a dynamic fill layer.

Then I added a gradient layer mask on top of the whole kit and caboodle. It's an amazing thing that you can combine vector masks and layer masks inside of Photoshop. I think, it's just utterly and completely fantastic. So let's go ahead and switch to the image at hand and this version of the image, this catch-up document, is called Blue super suit.psd, found inside the 15_paths folder. I am going to bring up my Layers palette right here, and I want you to go ahead and select the blue 2 layer, if it's not already selected.

I want you to press and hold the Alt key or the Option key on the Mac, click the black/white icon, choose Solid Color because we are inventing another shape layer. Solid Color allows us to create a dynamic fill layer. Even though it's called Solid Color, the result of the Solid Color is a dynamic fill, meaning that we can edit that fill any time we like, just by double clicking on it. Here comes the New Layer dialog box and I am just going to call this whiteness because we are just filling this area with white. Use Previous Layer to Create Clipping Mask is now on, by default. Well, that is as it should be, so that's nice. These settings are fine for now, Normal, 100%, everything is okay. Click OK and let's go ahead and make this layer white by clicking and dragging up into the upper left corner of the big color field here; or you could set the R, G, and B values, all to 255 and then click OK.

Now we need to go ahead and mask the contents of this layer and we are going to do that using one of the paths in the Paths palette. So go over to the Paths palette. You will see this path called waves, which are actually made up of a bunch of sub-paths. Let me show you how I made it. I am not going to walk through the entire thing because it requires some patience, put it that way and I don't want to try your patience at this point. You can do it, if you want to. Here is how. I am going to first invent a new path by clicking on the little Create new path icon down here at the bottom of the Paths palette. Actually, I am going to Alt+Click or Option+Click and I will say, new waves or something like that; although, I am only going to make partial progress. I click OK and let's go back to the Layers palette and turn off whiteness for a second, so we can see through to the underlying image. See what portion of the image, we want to select.

Switch back to the Paths palette. We have got new waves selected. I am going to draw this as a custom shape because there is a custom shape that draws waves for you. So I will go ahead and switch to the Custom Shape tool. Notice, by the way, I have gone ahead -- if you loaded my Deke keys. I have gone ahead and given this tool, a keyboard shortcut of U. I got rid of U for all the other tools, so that you don't cycle between them because really custom shape is the tool of tools, where the shape tools are concerned. Notice by the way, inside the options bar when any of the Custom Shape tools are selected, you can also switch to the Freeform Pen tool, which is I have told you I don't like one little bit; or to the wonderful standard Pen tool.

Let's switch to a different shape here and if you click the down pointing arrowhead, you will see the small collection of shapes that are installed along with Photoshop and load with Photoshop, by default. Click the right pointing arrowhead and choose All, so that you can see all of the shapes that are installed with the program. You don't want Append, you want OK, just say OK because otherwise, you are going to duplicate a bunch of shapes. Then you will look like crazy for the waves and let's see, where in the world are the waves here? You can obviously make this drop down palette bigger by dragging the lower left corner, if you are so inclined, and there is waves right there. Go ahead and click on it in order to make it active, then you can hide the shapes of palette, if you want to. Then you just drag, like so. I went ahead and actually, I don't think, I pressed the Shift key; I went ahead and left it a little taller than it was wide, than it was originally drawn, that is to say.

I wanted more waves. You can see here my waves, they are actually more continuous, they continue on and on. So I went ahead and took these waves, I selected them using the Path Selection tool or using the black Arrow tool, click and you will see that you select all of these paths together. Then if you press and hold the Alt Key or the Option key on the Mac and then start dragging these paths, you will clone them. I also press the Shift key, so that I could drag them exactly horizontally here. Then I will release the mouse button and then release both keys. I will zoom in and you can see that they don't really snap in place, actually, as they should.

So you got to zoom way in and then nudge the paths around using the Arrow tool. Bear in mind, when you are zoomed in, you are going to move one screen pixel, when you press the arrow keys. So it does give you more and more control, the farther you zoom in. Then at this point, you can see that these edges don't really match these edges. So these guys are aligned but these guys over here are not aligned; the far left points are not aligned. So I went ahead and switched to the white Arrow tool, clicked off the shapes, and then this is a tedious part, Click, Shift+Click, Shift+Click, Shift+Click, Shift+Click. If that's not tedious, I don't know what is. Then press the Backspace key or the Delete key on the Mac.

Now here is the deal. Photoshop doesn't really give you a lot of options for joining paths together. That's really Illustrator's territory. If you want to be able to combine paths with each other by, sort of, joining end points and that kind of thing, the stuff that Illustrator excels at, then you are going to need to create your paths inside Illustrator in the first place, copy them, and then paste them into Photoshop as paths; but otherwise, you just have to do this, kind of, monkey work inside Photoshop itself.

Then I coil them up, and I had to create some additional paths over here, and so on, and so on to create these waves. I am going to spare you the details. If you want to investigate on your own, be my guest. Anyway, I have got these waves all ready to go. Let's go ahead and click on them here inside the Paths palette. Let's return to the Layers palette, turn the whiteness layer back on, and make sure it's active. Then go down here to the layer mask icon and Ctrl+Click on it or Command+Click on that icon on the Mac, in order to make these, in order to convert these guys to a vector mask or here is another way to work.

We have already seen that way of Ctrl+ Clicking or Command+Clicking on the Mac, here is another way to work. We know, we are going to need a layer mask, as well, in order to allow us to draw the gradient. So you could just click down here. Even though, the paths are selected and highlighted and you can see them in everything. If you just click on this icon, the Add a layer mask icon, and notice it says, Add layer mask, Photoshop will go ahead, if you click on it, it will go ahead and add an empty layer mask. So it won't do anything. It's not actually masking at this point because it's filled with white. Now hover your cursor over the Add layer mask icon and it becomes an Add vector mask icon because you have already got layer mask. You can only have one layer mask, but if you make another mask it's got to be a vector mask.

So now click on it and notice what happens. Photoshop is smart enough, you don't have to Ctrl+Click this time. Photoshop is smart enough to go ahead and assign those path outlines as the vector mask. Now then we need to cut off the vector mask. Actually, let's go ahead and switch to the Full Screen mode, so I can drag it down a little bit. We need to cut off the vector mask at the costume hem, so go to the blue 3 layer right there. In order to make it active, just click on its vector thumbnail and using the Arrow tool -- let's go ahead and press the A key again to get the black Arrow tool, go ahead and click on the path outline to select it, press Ctrl+C or Command+C on the Mac in order to copy it.

Let's go back to the vector mask associated with the whiteness layer and we are running out of room in order to see the name of the whiteness, layer but you know that's it right there. Go ahead and click on its vector mask thumbnail to make it active and then press Ctrl+V or Command+V on the Mac in order to paste that vector mask into place. Now it's currently set to the Add mode, as you can see up here in the options bar. Let's go ahead and switch it to the Subtract from shape area mode and you can also do that just by pressing the minus key, if you want to. Plus and Minus are the keyboard shortcuts for the Add and Subtract modes up here. All right, so that's a beautiful thing.

Now we can go ahead and click on the vector mask thumbnail in order to turn off those path outlines. Now what I am going to do with the layer mask active, you can see it's active, let's go ahead and Shift+Tab away my palettes, so we can hide them from view and I am going to grab my Gradient tool. I am actually going to zoom out, so we have some smoother transitions on screen here. So Photoshop is doing a better job of rendering the image. So I switch to the Gradient tool. Make sure the foreground color is black and the background color is white, as it should be when you first create the layer mask, even though it's not really the default settings. Otherwise, you can press D for default and then X for black and white, like so. Go up here, make sure we have got our default settings going. If you want to just make sure you have defaults, you can Right Click on the down pointing arrowhead and choose Reset tool, and you should see these settings, like so. So we are going to draw a black to white linear gradient, Normal, 100% Opacity, and so on.

Then I am just going to move my image over to the left, a little bit. I might, otherwise, end up getting an auto scroll and I would like to avoid that. Then I drag from here to about here in order to fade the waves in and that's actually not enough. So let's try that longer, like so. That's better, I think. We end up creating these fading waving lines that are getting more and more opaque, as they proceed over to the left, beautiful, wonderful. Now let's go ahead and bring back the Layers palette. Make sure that layer is active. This isn't always and it wouldn't be active. Press the M key to switch back to the Marquee tool and then press the 7 key to lower the Opacity of the layer to 70%. That is the result of combining a vector mask and a layer mask together here inside the mighty and powerful Photoshop.

There are currently no FAQs about Photoshop CS3 Channels & Masks: Advanced Techniques.

Share a link to this course

What are exercise files?

Exercise files are the same files the author uses in the course. Save time by downloading the author's files instead of setting up your own files, and learn by following along with the instructor.

Can I take this course without the exercise files?

Yes! If you decide you would like the exercise files later, you can upgrade to a premium account any time.

Become a member Download sample files See plans and pricing

Please wait... please wait ...
Upgrade to get access to exercise files.

Exercise files video

How to use exercise files.

Learn by watching, listening, and doing, Exercise files are the same files the author uses in the course, so you can download them and follow along Premium memberships include access to all exercise files in the library.

Exercise files

Exercise files video

How to use exercise files.

For additional information on downloading and using exercise files, watch our instructional video or read the instructions in the FAQ .

This course includes free exercise files, so you can practice while you watch the course. To access all the exercise files in our library, become a Premium Member.

* Estimated file size

Are you sure you want to mark all the videos in this course as unwatched?

This will not affect your course history, your reports, or your certificates of completion for this course.

Mark all as unwatched Cancel


You have completed Photoshop CS3 Channels & Masks: Advanced Techniques.

Return to your organization's learning portal to continue training, or close this page.


Upgrade to View Courses Offline


With our new Desktop App, Annual Premium Members can download courses for Internet-free viewing.

Upgrade Now

After upgrading, download Desktop App Here.

Become a member to add this course to a playlist

Join today and get unlimited access to the entire library of video courses—and create as many playlists as you like.

Get started

Already a member ?

Exercise files

Learn by watching, listening, and doing! Exercise files are the same files the author uses in the course, so you can download them and follow along. Exercise files are available with all Premium memberships. Learn more

Get started

Already a Premium member?

Exercise files video

How to use exercise files.

Ask a question

Thanks for contacting us.
You’ll hear from our Customer Service team within 24 hours.

Please enter the text shown below:

Exercise files

Access exercise files from a button right under the course name.

Mark videos as unwatched

Remove icons showing you already watched videos if you want to start over.

Control your viewing experience

Make the video wide, narrow, full-screen, or pop the player out of the page into its own window.

Interactive transcripts

Click on text in the transcript to jump to that spot in the video. As the video plays, the relevant spot in the transcript will be highlighted.

Learn more, save more. Upgrade today!

Get our Annual Premium Membership at our best savings yet.

Upgrade to our Annual Premium Membership today and get even more value from your subscription:

“In a way, I feel like you are rooting for me. Like you are really invested in my experience, and want me to get as much out of these courses as possible this is the best place to start on your journey to learning new material.”— Nadine H.

Start your FREE 10-day trial

Begin learning software, business, and creative skills—anytime,
anywhere—with video instruction from recognized industry experts. provides
Unlimited access to over 4,000 courses—more than 100,000 video tutorials
Expert-led instruction
On-the-go learning. Watch from your computer, tablet, or mobile device. Switch back and forth as you choose.
Start Your FREE Trial Now

A trusted source for knowledge.


We provide training to more than 4 million people, and our members tell us that helps them stay ahead of software updates, pick up brand-new skills, switch careers, land promotions, and explore new hobbies. What can we help you do?

Thanks for signing up.

We’ll send you a confirmation email shortly.

Sign up and receive emails about and our online training library:

Here’s our privacy policy with more details about how we handle your information.

Keep up with news, tips, and latest courses with emails from

Sign up and receive emails about and our online training library:

Here’s our privacy policy with more details about how we handle your information.

submit Lightbox submit clicked
Terms and conditions of use

We've updated our terms and conditions (now called terms of service).Go
Review and accept our updated terms of service.