Easy-to-follow video tutorials help you learn software, creative, and business skills.Become a member

Splitting a scene into render layers

From: Lighting and Rendering with mental ray in Maya

Video: Splitting a scene into render layers

I would like to show you how to split up a scene into render layers for easy compositing. In this scene, I have my dining room geometry with some basic lighting texturing applied, and I also have a volumetric lighting effect established, and this is using this directional light and this volume surface. If I open up the Attribute Editor for the directional light, you'll see it has a physical light shader applied, and that's part of what's necessary to create the volumetric lighting effect.

Splitting a scene into render layers

I would like to show you how to split up a scene into render layers for easy compositing. In this scene, I have my dining room geometry with some basic lighting texturing applied, and I also have a volumetric lighting effect established, and this is using this directional light and this volume surface. If I open up the Attribute Editor for the directional light, you'll see it has a physical light shader applied, and that's part of what's necessary to create the volumetric lighting effect.

I am going to go back to my renderCam camera, so you can see from the point of view of how I am going to render. I am going to switch to the Channel Box, and the first thing I want to do is create a render layer that contains all my room geometry and lights. So, I will select the roomGeometry group, go to the Render Layer Editor and choose Layers > Create Layer from Selected. I am going to double-click on this and call it roomLit_RL.

The RL is just there to remind me that this is a render layer. So if I select on roomGeometry group and scroll down in the Channel box under Inputs, I can see roomLit_RL among the inputs. By having the RL there, I know that that's a render layer. The next thing I need to do is add my lights. I am going to switch to Perspective view and press 4 to switch to wireframe. If I go back to my defaultRenderlayer, you can see the lights there. I will select on the chandelier group, and this group is just made up of all the point lights that are in the chandelier.

I just created that group to help me organize it a little bit better. So if I switch back to roomLit_RL in the Render Layer Editor, right-click and choose Add Selected Objects, it's going to add that group. I will go back to my defaultRenderlayer, and I will select these four lights. To be consistent and neat, I will press Ctrl+G to group these, double-click on the group name and call these lampLights. So I know that that is the lampLight group.

Click back on roomLit_RL. With lampLights group selected, I will right-click over the render layer and choose Add Selected Objects, and now they appear. Finally, I do have this other directional light here, and this is used to simulate maybe light coming from a room from behind the camera, but really it just helps to boost the lighting in the room to make everything a bit more invisible. So now that I have this selected, I will go back to roomLit_RL, right-click and choose Add Selected Objects, so now this is added.

The last thing I would like to do is switch back to the renderCam camera, press 5 to switch to shaded view. You can see these skylights up here. What I want is this volumetric lighting effect is created with the volumeLight1, and this is just a directional light, and it's pointed down at a particular angle so that the beam's light will come in through right here. I am not going to render the beams of light on the roomLit_RL render layer, but I do need the effect of the light hitting the surfaces here.

So what I want to do is I want to add a light that is at the same angle and has the same properties as this light to the roomLit_RL render layer. So let's switch back to the Perspective view here. An easy way to do this is just to select volumeLight1 in the Outliner and press Ctrl+D to duplicate it, and I will just move it over to the side here. Now remember, this won't change the angle of the light, because the position of a direction layer has no bearing on how it shines; it's just through rotation.

So I am going to rename this light sunLight, so I know it's distinguished from this. If I open its Attribute Editor, I am going to select it, open the Attribute Editor, and switch to the sunLightShape tab. There are no Light shaders applied to this light, so these are both blank. If I select on volumeLight1, you will see physical_light1 is in the Light Shader, sunLight, no light shader. But I do need to turn on Raytrace Shadows. So I will just scroll to the top, go down to the Raytrace Shadows Attributes and turn on Use Ray Trace Shadows.

So that's good to go. And then of course, the last thing I need to do is with sunLight selected, I will switch to roomLit_RL render layer, right-click and choose Add Selected Objects. So now in this render layer, I have my room geometry, my room lighting, and a duplicate of the sun light. So I am going to switch to the renderCam camera. Now you will notice one thing. The only thing I did not add to these render layers is the renderCam camera. When you are using render layers, you should not have to add or subtract a camera from one render layer to another, unless you want to.

You can, if it helps you to remember that that camera is on a particular render layer, but it's not going to affect how the layer renders. Cameras sort of get a free pass when it comes to render layers. So let's open up the Render view and create a render from the renderCam. Okay, so here we have our rendered image. I will store this in the Render view. It will take about not quite four minutes. We can see that we have, obviously, the room geometry.

Everything is shaded correctly. We have our lights. You can also see that the sunlight is coming through the windows here, and it's actually shining down on the table, creating these white squares. It's because of the bars here in the skylight. You could also see the same light coming through the windows here. It looks a little strange because, of course, it is dark outside the window; there is a number of ways we can handle that after we set up all our render layers. For one thing, when we composite it in Photoshop or After Effects, I will take a look at the alpha channel here, just by clicking on this icon here in the top menu bar in the Render view, and you can see that the outside is not included in the alpha channel.

So we could easily composite a simple gradient or image for sky back here, and that will work as well. So let's store that, and let's go to setting up the next render layer. Now the next render layer is going to have the geometry on it and then all the necessary elements for creating the volumetric lighting effect, and this will be the volumelight1 directional light, as well as the volume geometry that I've created. Let's go to the defaultRenderlayer, which contains everything in the scene. I want to select the roomGeometry group, and again, I am going to go Create Layer from Selected.

Double-click on that layer in the Layer Editor, and I am going to rename this volumeLight_RL. The other two things that I am going to need, I am going to need my volume geometry, so I will select that in the Outliner and right-click and choose Add Selected Objects. So now this has been added. If I switch to the Perspective view, you can see now there it is. It's not on the roomLit_RL render layer, but it is on the volumeLight_RL render layer.

Now the last thing I need to add to this layer is the volumeLight1 directional light. So I will select that in the Outliner, right-click over volumeLight_RL and choose Add Selected Objects, and there it is. I am going to switch back to the renderCam camera. There's one other thing I'd like to do. I am going to switch to the Select tool here in the toolbar, and the first thing I would like to do is select the roomGeometry, and I am going to apply a special shader to this geometry, so that it doesn't appear in the render.

It will block the light coming through the top here in order to create the nice- looking rays that come through the skylight. But it actually won't show up in the render, and it won't show up in the alpha channel, as well. So with the roomGeometry group selected, and I am in the volumeLight_RL layer, I have that selected - those are very important - I am going to click on the black surface material shader and that applies the shader to the roomGeometry group. I will open up the Attribute Editor to the surfaceShader2 tab, and I am going to set the Out Matte Opacity to black, so this does not show up in the alpha channel.

Now what's interesting to note is since I applied the shader while I was in the volumeLight_RL render layer, if I switch back to roomLit_RL render layer, I will deselect it, and you can see the textures are correct; we have a brown wood texture here, white wall texture, and so on and so forth. So the shader, you can have different shaders applied to the same geometry on different render layers, and this is what makes them so useful. I'll turn on Wireframe on Shaded so we can see the geometry. So on this render layer, we have a black surface shader applied. On this render layer, we have a regular texturing applied.

So that's a very important element of render layers. It's what makes them so flexible. And there is one thing I like to point out is that since I applied the shader to the entire group, this included the window geometry here, these polygon planes that act as the glass in the windows. So I am going to select those, and once they are selected, I will do Ctrl+H to hide them. So you can see what's going on here, I am going to hide this to volume geometry and take a look what's going on.

So you can see through these windows because the geometry is not there; same with these two pieces of window, Ctrl+H to hide them. So this is another element of render layers. The visibility of an object can be different, depending on which layer you are on. So in other words, if I'm in the roomLit render layer, these pieces of glass are visible; their visibility is set to on. On the volumelight_RL, their visibility is turned off, so they are invisible. So that's also very helpful; you can this with lights to have lights on or off, depending on different render layers, or objects as well.

So at this point, I should be good to go. I will go to my renderCam camera, and I am going to make sure that the volumelight geometry, I am going to select that in the Outliner and do Shift+H to turn it back on, because we are going to need that to create the volumetric lighting effect. And I am going to open up the Render view, make sure this image is stored, and create a render. Okay, so we have finished our render of the volumetric lighting effect, so you can see it's completely isolated.

I am just seeing the volumetric effect. I am not seeing any of the lights in the room or the shading on the geometry, and this will make it much easier to composite. This is just one of the many users of render layers, but I think this is a good example of how they can be useful. So now that I have created that, I am going to keep this image. I will minimize the Render view, and I only have one more render layer that I would like to create, and this is the one in which I can use the red, green, and blue colors to create a geometry mask so the different elements can easily be selected based on their color channels and the compositing software.

So all I need for that is really just the room geometry. So I am going to select the room geometry. I will switch to the default render layer and make sure that everything is visible, and in the Layer palette, I will choose Create layer from Selected, and there we go, and I am going to select on this layer in the Render Layer Editor, double-click, and call it colorChannel_RL. Actually, I would like to call it colorMask. That makes more sense. There we go.

Now that I have this selected, I can start applying different shaders to it as well. The first thing I will do is, I am just going to, with the entire group selected, click on the black surface shader icon here on the rendering shelf. This applies it to everything that was selected, and I will open up, this is surfaceShader3. I will call this redShader, and the Out Color I will set to red. Now everything in the room is red. So now what I want to do is I want to select on just this globe geometry.

So I will switch to Perspective view, press 4 to switch to Wireframe, so I can see what I am selecting here. And really, an easy way to do this is to go on the Outliner, expand roomGeometry group, expand the room subgroup and expand the chandelier, and I have my globe geometry right here. You could just select the first one; this is globe8, Shift+select globe1. Now that they are selected, I will just click on that surfaceShader icon in the rendering shelf again, and this applies a new shader, and I am going to call this greenShader in the Attribute Editor.

Click on the Out Color in the Attribute Editor and pick green. Switch back to the renderCam camera. If I press the 6 button to switch to Shaded mode - and in the Shading palette, I will turn off Wireframe on Shaded - now you can clearly see red for the geometry of the room, green for the globes of the chandelier. So the last thing I want to do, I will press 4 to switch to Wireframe, and I am going to select this lampShade, carefully select this lampShade, carefully select this lampShade, and I will go to the other side of the room, Shift+select on this lampShade, Shift+ select on this lampShade.

So now I have all four lampshades selected, and I will zoom in on the candles, as well. Shift+select the little flame there, move over here to this candle, Shift+select that flame, so now I have the two flames and the four lampShades selected on the rendering shelf. I will click surface material to create a new surface material, and I will call this blueShader. And I will click on the Out Color of the blueShader and select a blue color.

And now, when I go to the renderCam camera and press 5 and deselect everything, I can clearly see that the geometry of the room is red, the geometry of the chandelier - the little glass globes - are green and the geometry of the lampShades as well as the two flames are blue. And I am just picking these somewhat arbitrarily, just as an example of why you might want to use this technique. The reason using red, green, and blue is helpful is because I can do a single render layer render, and with these three colors, create three different types of masks, easily, from my compositing software.

This would be - I mean, I could create one render layer for the glass globes of the chandelier, another render layer for the geometry and another render layer for the lampShades here and the candle flames, but this just combines it all in one single render pass, and this makes things a little bit faster. Of course, when I select this render layer and create a render, this is also a very fast render. It only takes about three seconds, and there we go. And since this has an alpha channel, I will click on alpha channel here.

I essentially have four different types of masks: red, green, blue, and alpha. I will store this image, and that's essentially how you would go about splitting a scene that already exists into different render layers.

Show transcript

This video is part of

Image for Lighting and Rendering with mental ray in Maya
Lighting and Rendering with mental ray in Maya

59 video lessons · 7872 viewers

Eric Keller
Author

 
Expand all | Collapse all
  1. 3m 46s
    1. Welcome
      1m 32s
    2. Using the exercise files
      2m 14s
  2. 19m 8s
    1. What is a CG light?
      1m 22s
    2. Types of CG lights
      10m 55s
    3. Direct lighting
      4m 48s
    4. Indirect lighting
      2m 3s
  3. 53m 20s
    1. Decay rate
      6m 30s
    2. Previewing lighting and shadows
      2m 37s
    3. Creating depth map shadows
      1m 57s
    4. Troubleshooting depth map shadows
      2m 38s
    5. Shadow map overrides
      5m 30s
    6. Using the shadow map camera
      5m 31s
    7. Saving and reusing shadow maps
      2m 48s
    8. Creating raytraced shadows
      1m 56s
    9. Adding softness to raytraced shadows
      3m 42s
    10. Creating area light shadows
      5m 11s
    11. Sample: mental ray area light
      4m 23s
    12. Setting area light visibility
      8m 7s
    13. Creating soft shadows with spot lights
      2m 30s
  4. 43m 35s
    1. Setting global illumination for interiors
      2m 33s
    2. Tuning global illumination
      5m 56s
    3. Global illumination photons
      1m 12s
    4. Activating caustic light effects
      3m 28s
    5. Tuning caustic settings
      3m 35s
    6. Setting caustic light effects on metal
      2m 35s
    7. Using final gathering for indirect lighting
      2m 9s
    8. Tuning final gathering
      4m 2s
    9. Reusing final gathering maps
      3m 21s
    10. Adding light with shaders
      5m 27s
    11. Creating final gathering maps for animation
      4m 26s
    12. Combining final gathering with global illumination
      4m 51s
  5. 1h 2m
    1. Activating the Physical Sun and Sky network
      2m 33s
    2. Tuning the Physical Sun and Sky settings
      7m 18s
    3. Applying physical light shaders
      8m 54s
    4. Applying image-based lighting
      8m 57s
    5. Tone mapping
      6m 23s
    6. Applying portal light shaders
      7m 45s
    7. Creating light beams with participating media
      10m 9s
    8. Adding depth of field with the Bokeh lens shader
      10m 39s
  6. 48m 21s
    1. Introducing render layers
      6m 13s
    2. Creating render layers
      4m 28s
    3. Splitting a scene into render layers
      15m 36s
    4. Applying render layer presets
      7m 47s
    5. Setting render layer overrides
      7m 7s
    6. Creating render layer composites
      3m 52s
    7. Organizing renders with tokens
      3m 18s
  7. 42m 24s
    1. Introducing render passes
      2m 56s
    2. Comparing render passes and render layers
      6m 44s
    3. Editing render passes
      10m 41s
    4. Using appropriate materials
      5m 51s
    5. Batch-rendering passes
      5m 56s
    6. Compositing in After Effects
      6m 41s
    7. Rendering the EXR image format
      3m 35s
  8. 23m 3s
    1. Anti-Aliasing Quality
      6m 44s
    2. Setting color profiles
      2m 53s
    3. Diagnosing raytracing
      5m 7s
    4. Adjusting motion blur
      6m 57s
    5. Finding mental ray help
      1m 22s
  9. 21s
    1. Goodbye
      21s

Start learning today

Get unlimited access to all courses for just $25/month.

Become a member
Sometimes @lynda teaches me how to use a program and sometimes Lynda.com changes my life forever. @JosefShutter
@lynda lynda.com is an absolute life saver when it comes to learning todays software. Definitely recommend it! #higherlearning @Michael_Caraway
@lynda The best thing online! Your database of courses is great! To the mark and very helpful. Thanks! @ru22more
Got to create something yesterday I never thought I could do. #thanks @lynda @Ngventurella
I really do love @lynda as a learning platform. Never stop learning and developing, it’s probably our greatest gift as a species! @soundslikedavid
@lynda just subscribed to lynda.com all I can say its brilliant join now trust me @ButchSamurai
@lynda is an awesome resource. The membership is priceless if you take advantage of it. @diabetic_techie
One of the best decision I made this year. Buy a 1yr subscription to @lynda @cybercaptive
guys lynda.com (@lynda) is the best. So far I’ve learned Java, principles of OO programming, and now learning about MS project @lucasmitchell
Signed back up to @lynda dot com. I’ve missed it!! Proper geeking out right now! #timetolearn #geek @JayGodbold
Share a link to this course

What are exercise files?

Exercise files are the same files the author uses in the course. Save time by downloading the author's files instead of setting up your own files, and learn by following along with the instructor.

Can I take this course without the exercise files?

Yes! If you decide you would like the exercise files later, you can upgrade to a premium account any time.

Become a member Download sample files See plans and pricing

Please wait... please wait ...
Upgrade to get access to exercise files.

Exercise files video

How to use exercise files.

Learn by watching, listening, and doing, Exercise files are the same files the author uses in the course, so you can download them and follow along Premium memberships include access to all exercise files in the library.


Exercise files

Exercise files video

How to use exercise files.

For additional information on downloading and using exercise files, watch our instructional video or read the instructions in the FAQ.

This course includes free exercise files, so you can practice while you watch the course. To access all the exercise files in our library, become a Premium Member.

Join now "Already a member? Log in

Are you sure you want to mark all the videos in this course as unwatched?

This will not affect your course history, your reports, or your certificates of completion for this course.


Mark all as unwatched Cancel

Congratulations

You have completed Lighting and Rendering with mental ray in Maya.

Return to your organization's learning portal to continue training, or close this page.


OK
Become a member to add this course to a playlist

Join today and get unlimited access to the entire library of video courses—and create as many playlists as you like.

Get started

Already a member?

Become a member to like this course.

Join today and get unlimited access to the entire library of video courses.

Get started

Already a member?

Exercise files

Learn by watching, listening, and doing! Exercise files are the same files the author uses in the course, so you can download them and follow along. Exercise files are available with all Premium memberships. Learn more

Get started

Already a Premium member?

Exercise files video

How to use exercise files.

Ask a question

Thanks for contacting us.
You’ll hear from our Customer Service team within 24 hours.

Please enter the text shown below:

The classic layout automatically defaults to the latest Flash Player.

To choose a different player, hold the cursor over your name at the top right of any lynda.com page and choose Site preferencesfrom the dropdown menu.

Continue to classic layout Stay on new layout
Exercise files

Access exercise files from a button right under the course name.

Mark videos as unwatched

Remove icons showing you already watched videos if you want to start over.

Control your viewing experience

Make the video wide, narrow, full-screen, or pop the player out of the page into its own window.

Interactive transcripts

Click on text in the transcript to jump to that spot in the video. As the video plays, the relevant spot in the transcript will be highlighted.

Are you sure you want to delete this note?

No

Your file was successfully uploaded.

Thanks for signing up.

We’ll send you a confirmation email shortly.


Sign up and receive emails about lynda.com and our online training library:

Here’s our privacy policy with more details about how we handle your information.

Keep up with news, tips, and latest courses with emails from lynda.com.

Sign up and receive emails about lynda.com and our online training library:

Here’s our privacy policy with more details about how we handle your information.

   
submit Lightbox submit clicked
Terms and conditions of use

We've updated our terms and conditions (now called terms of service).Go
Review and accept our updated terms of service.