Easy-to-follow video tutorials help you learn software, creative, and business skills.Become a member
You have heard me say several times in this course that a grayscale mask, whether in an alpha channel or as a layer mask, is another way to visualize a selection. If you do happen to have made an initial selection into a mask, you can refine the mask the same way you would have refined the selection. But in the case of a mask, you work in the Refine Mask dialog box. In the case of a selection, you'd work in Refine Edge dialog box, as I showed you earlier in this chapter. The two dialog boxes are basically replicas of one another.
In this movie, I will show you how to refine the edge of a layer mask using Refine Mask. I will start by making a rough selection with the Quick Selection tool. If you don't want to make that selection along with me, and you are using the exercise file for this course, you can go to the Select menu, choose Load Selection, go to the Channel menu, and choose Wall and OK. I am going to choose Cancel, and I'll make that selection by clicking with the Quick Selection tool inside of the orange climbing wall. Then I am going to hold down the Option key on the Mac or the Alt key on the PC, and I am going to travel along the edge of the orange climbing wall on the blue sky, and what I am doing is telling this tool that I don't want it to select the blue sky, as you will learn in a separate movie on the Quick Selection tool.
Now with the Quick Selection tool, I'll move over the orange wall, and I'll click and drag, and I'll also drag over the thatched roof. When I have this selection, I could go to Refine Edge in the Options bar and click there and refine the selection, but I am not going to do that; instead, right away, I am going to make a layer mask from this selection. You have seen me do this in other movies in this course. With this selection active, I will go to the bottom of the Layers panel and click this icon that looks like a circle inside a square, or I could go up to the Masks panel and click this icon to add a pixel mask.
By the way, if your Masks panel isn't open, you can open it from the Window menu at the top of the screen. So I'll click here, and that creates a layer mask on the selected climbing wall layer, which is another way to view and work with my initial selection. Why would I make a layer mask from my selection rather than go right into Refine Edge? Well, I might do this to make sure that I don't accidentally delete the selection, or to give myself the flexibility of using a modifiable mask from the get-go. Once I have this mask, notice that the marching ants of the selection no longer appear in the image.
Now I want to modify the layer mask on the climbing wall layer to get a more detailed selection along the edges of the thatched roof and to smooth out the selection along the edge of the climbing wall. I will make sure that I have the layer mask thumbnail selected on the climbing wall layer, and then I will go up into the Masks panel, and I'll click this button: Mask Edge. That opens the Refine Mask dialog box. All the controls that you see here, you are probably familiar with if you have been following along with this course, because they are the same as the controls in the Refine Edge dialog box that I have used in other movies to refine a selection edge.
First, I am going to try to smooth out the edge of the mask along the wall. So I am going to click the Radius slider and drag it slightly to the right, and right away I get that result that I want, here along the wall. I am also going to click Smart Radius, and now I am going to drag the Radius slider further to the right, and I am getting a result that I want up here in the thatched roof, which is that I am bringing more of the foliage back into the image along this detailed edge. As you may remember from the movies on the Refine Edge dialog box, what Smart Radius does is tell Photoshop to decide where the transition area should be along all the edges, and which is going to make decisions about what to select.
So if you'd like to see that transition area, I will check Show Radius, and you can see that with Smart radius enabled, Photoshop is giving me a larger, softer transition area around the thatched roof than down the more well-defined edge of the climbing wall. I will uncheck Show Radius. Now I am going to use one of the refinement tools. These are the same as the refinement tools in the Refine Edge dialog box. I will choose the Refine Radius tool, and I'll move into the image, make my brush tip slightly smaller by pressing Left Bracket key, and I am just going to drag along the thatched roof to refine the transition area there, which is giving me an even better mask in this area.
I might also come down to the Shift Edge slider and drag it to the right to expand the edge to bring back more of the foliage in the roof. And because I see some blue along the edge of the foliage, I am going to click Decontaminate Colors, and I will drag the Amount slider there a little bit to the right. Now I have a really good mask that is hiding the blue sky along the edge of the roof and along the edge of the climbing wall. Now I'll go to the Output To menu, and notice that I don't have a selection or layer mask option, and that's because I enabled decontaminate color, just as I showed with the Refine Edge controls in earlier movies.
So I will choose to output as a new layer with layer mask, and I will click OK. That gives me this new layer called climbing wall copy with its own layer mask, representing the refinements I just made in the Refine Mask dialog box. At this point, I could delete the climbing wall layer if I wanted to; I will just leave it here. There are a few additional controls available when I am working with a mask that I don't have when I am working with a selection, and those are here in the Masks panel. The most important of those is the Density control.
If I click and drag the Density control to the left, I am making the black parts of the mask on the climbing wall layer mask less dense. In other words, they are being changed from black to gray. I can show you that by holding the Option key on the Mac or the Alt key on the PC and clicking on that layer mask thumbnail. So you can see that this area is now gray, which means that it is only partially hiding the content of the climbing wall. If I click and drag the Density slider back to the right, that same area of the layer mask becomes black, so that it's completely hiding this portion of the climbing wall.
I will Option+Click or Alt+Click again on that layer mask thumbnail. There is also a Feather slider right here on the front of the Masks panel. This is just like the Feather slider inside the Refine Masks dialog box, and dragging this will just blur the edge of the mask. In this case, I don't really want to blur the edge of the mask, so I will leave that at zero. There is also a Color Range button. If I click this button, it opens the Color Range dialog box - another selection method that I will show you in another movie in this course - but to just get a taste of this, if I click on a color here in the Color Range dialog box, like this orange, it will limit where my layer mask applies.
So I click OK here, and you can see that the mask is now changed so that it's hiding everything except the orange on the climbing wall copy layer. I am going to Undo. Finally, there is an Invert button here, and if I use this, I am inverting my mask so that the black areas of the mask are now on the left side, and the white areas of the masks are now on the right side, and I get a completely opposite result in my image. I will click Invert again in the Masks panel to switch the layer mask on the climbing wall copy layer back to where it should be.
There are a few more controls here that are not really relevant in this case, and the main controls for refining a layer mask, like this one, or a mask you have saved in an Alpha channel in the Channels panel are just like the controls for refining a selection that are found in the Refine Edge dialog box.
Get unlimited access to all courses for just $25/month.Become a member
180 Video lessons · 77090 Viewers
64 Video lessons · 94881 Viewers
86 Video lessons · 62255 Viewers
103 Video lessons · 31698 Viewers
Access exercise files from a button right under the course name.
Search within course videos and transcripts, and jump right to the results.
Remove icons showing you already watched videos if you want to start over.
Make the video wide, narrow, full-screen, or pop the player out of the page into its own window.
Click on text in the transcript to jump to that spot in the video. As the video plays, the relevant spot in the transcript will be highlighted.