TUTORIAL 7: Global Illumination & Ambient Occlusion

The goal of this short tutorial is to introduce a few select aspects of mental ray so that you may
consider incorporating them in your projects if appropriate. These include global illumination and
ambient occlusion. Well focus on the key attributes that are useful in optimizing each, and use
different scenes to simplify and facilitate the demonstration of each effect. Well also explore
another feature of mental ray the approximation editor that is unrelated to lighting.

Global Illumination
Global illumination (GI) is a method of indirect lighting photons are emitted from one or several
light sources and are bounced around your scene. As they hit surfaces with a diffuse light
component to their shader, they bounce off that surface and in the process accumulate color as
well (i.e. color bleeding). Objects that bounce photons become secondary light sources in
themselves. The following simple example shows you how to enable GI in a Maya scene and, in
particular, how to optimize GI settings to achieve a smooth lighting result.

Geometry set-up
To demonstrate the GI effect, well model a
simple forked tube that has a spotlight inside
one of its side branches. The main trunk of the
tube will have no lights and will therefore only be
lit from the spotlight emanating from this side
tube. Without GI, the spotlight will cast a single
area of light inside the tube with GI enabled,
however, photons emitted from the spotlight will
hit the wall of the tube and then continue to
bounce thereby extending the area of
illumination inside the tube. This basic set-up
will help to exaggerate GIs contribution to
lighting in a scene (an effect that can otherwise
be very subtle).

Scaled cube.

Well begin by modeling the forked tube using

polygons and explore how mental rays
approximation editor can take care of smoothing
the surface at render without the need to convert
to subdivision surfaces.
cube smoothened with 1 division

1. Create a polygon cube at the origin and set

the scale x y z to 6 6 & 60 respectively. Set the
subdivisions depth to 12.
2. Smooth the cube once (i.e. 1 subdivision only
using the default settings).
3. Select a few faces (see image on the right)
such that you have 2 that are on the same side
and 1 face apart and offset by 1 row (the idea is
to select faces that are not too far apart or the

selected faces on one side (one more selected on other side)

advTutorial 7: mental ray

Gal McGill

Spring 09 (Maya 2009)

1

indirect lighting will not reach far enough to light

the extrusions/forks you will create from these
faces).
4. Extrude the faces (make a small extrusion
first that scales the face in and out a bit and then
a second extrusion that pulls out the face quite a
bit see figure on right).
extruded faces

5. Now create a new camera and rename

RENDERCAM look through it, turn on its
resolution gate (the default 640 x 480 is fine)
and position it inside the main tube such that it
sees the openings of the 3 extruded faces (its
already inside the main tube by default, but
rotate it a bit to off-center it).
At this point if you were to fire a render, by
default the Maya software renderer would
launch and you would automatically get Mayas
default light turned on for you to avoid returning
a black render of your scene.
Next well add a spotlight inside the top right
extrusion (from the RENDERCAMs point of
view in the image to the right).

RENDERCAM set-up inside the tube with 3 openings visible

Create the spotlight (using default settings)

then move it with the transform/move
manipulator (or simply look through selected
and then use your camera motions to position it)
so that it faces down the side tube towards the
main trunk (see image on right).

Approximation Editor
Before we set-up GI lighting lets take quick look
at mental rays approximation editor. This
allows you to override Mayas tessellation for
objects with instructions derived from mental
ray. The powerful thing about this method is that
it all happens at render time - so there is no
need to use a higher poly-count version of your
model or to convert it to a subD model.

scene set-up with spotlight inside the an extrusion

To open the approximation editor, go to Window Approximation Editor: finding it (above) & the options window (below)
-> Rendering Editors -> mental ray ->
Approximation Editor. The window that opens
gives you a few choices depending on the type
of geometry you want to add a surface
approximation to. For our purposes, select your
polygonal object and click on the Create button
that is in the second panel labeled Subdivisions
(Polygon and Subdiv. Surfaces). Nothing seems
to happen other than the Create button
becomes grayed out. If you look through your
Finding the Approximation Editor
advTutorial 7: mental ray
Gal McGill

Spring 09 (Maya 2009)

2

perspective camera, frame one of the extrusions

on your main tube and render it (with Maya
software), youll see the expected boxy
appearance. Now switch the renderer to mental
ray in the render settings, render again, and the
model now appears smooth (as if it had been
converted to a subD surface). If you look
closely, youll notice that there are still some
visible edges that might be nice to remove.
To access the settings for the surface
approximation that mental ray calculates at
render time, either go back to the approximation
editor, select the surface and click the Edit
button, or select the surface in the viewport and
open the Hypergraph:Connections window and
click on the mentalraySubdivApprox1 node. In
both cases, this opens the settings in the
Attribute Editor the key setting is the N
Subdivisions value: set it to 4 to smooth out the
approximation even more and rerender to make
sure you are satisfied with this value (see image
on the right).
Now if you go back and look through the
RENDERCAM and fire a render, youll notice
Approximation Editor settings are used increase smoothness
that the pattern of light cast by the spotlight on
the inside of the main tube wall is smooth (due
to mental rays surface approximation).
However, there is no secondary/indirect light in
the scene, so only regions that are directly hit by
the spotlight rays are lit and the rest of the
geometry is dark.
The approximation node can be selected in the Hypergraph

Increasing the N Subdivisions value yields even smoother results

Render of the inside of the main tube without GI

advTutorial 7: mental ray

Gal McGill

Spring 09 (Maya 2009)

3

Enabling Global Illumination

Now that the geometry is set-up and you have a
spotlight illuminating the inside tube, we need to
start emitting photons from this light so these will
bounce around inside and extend the
illumination. Notice that in the current render,
the other openings created by your extrusion are
not visible because the rest of the scene is
completely black the goal of our GI set-up will
be to use indirect illumination to reveal the
opening of the other extrusions you created in
the main tube.
There are 2 places to enable GI in a Maya
scene: the light source and the render settings.
With the light source settings you control the
number and intensity of the GI photons emitted.
In the render settings, you control the radius of
the photons and the accuracy of the GI effect.

Enabling GI in the mental ray section of the Light Attributes

Select the spotlight and lets begin by making a

few adjustments to the main Attributes before
we enable GI: change the decay to quadratic
(which is closest to the behavior of real physical
light), set the Penumbra Angle to 10 and
increase the intensity to 20. We can come back
and adjust these settings after we have worked
with GI.
In the mental ray panel of the attribute editor,
under the Caustic and Global Illumination
section, click the Emit photons checkbox. Now
open the render settings and, under the Indirect
Lighting tab, go to the Global Illumination
section and click on the Global Illumination
checkbox. GI is now on try rendering the
scene from your RENDERCAM. What you
should see is a faint indirect lighting effect thats
a bit blotchy.

Optimizing Global Illumination

Optimizing GI in your scene involves a lot of
experimentation and tweaking on a number of
parameters. The first thing to assess is the
pattern of photon bouncing in the scene this
can be done a couple of ways. There is a utility
called the Map Visualizer (in the Photon Map
section of the Indirect Lighting tab of the
Render Settings a bit below the GI section) when checked, Maya will draw dots for every
photon bounce in your viewport. Turn on this
option in the render settings and rerender the

Enabling GI in the Render Settings

Opening mental rays Map Visualizer

advTutorial 7: mental ray

Gal McGill

Spring 09 (Maya 2009)

4

scene from the RENDERCAM. You should now

see a spattering of dots in your viewport.
You can customize some of these display
settings by going to Window -> Rendering
Editors -> mental ray -> Map Visualizer. The
window then provides you with options to control
the display of the dots in the viewport as well as
the direction photons are bouncing around. To
enable this feature, increase the Direction
scale to 0.2. You can also increase the point
size to 3 to make it easier to see what is the
photon and what is the direction tail. Feel free
to play with the settings to see how they work
return the direction scale to 0 and the point size
to ~2 before proceeding with the rest of the
Settings in the Map Visualizer
tutorial.
Another way to assess the pattern of photon
bounces in your scene is to do it through the
rendered image. In the render settings, in the
Global Illumination panel, notice that the
Radius value is set to 0. This default value lets
Maya decide what to make the radius. Change
the value to something small like 0.1 and rerender the scene form the RENDERCAM. You
should now see the individual photons as little
spots of light along the inside of the tube.
Clearly this is not the value we will use for the
photons in the final image in fact the radius
value is one of the settings we will be using to
smooth and blend together the illumination
generated by the photons. But in this mode, it
provides a useful sense of where the photons
are landing and, most importantly, whether we
have enough. Even though we are starting to
see the effect we are after (i.e. the other
extrusion openings are becoming faintly visible),
we will now increase the number of photons.
Make sure to save the images you render into
memory at every round of rendering to get a
sense for the effect of the settings updates.

Changing the default Radius value from 0 to 0.1

Go to the attributes for the spotlight and, in the

mental ray | Caustics & Global Illumination
section, increase the Global Illum Photons
value to 100,000. Rerender the scene overall,
the effect is now brighter. There is also a bit
more splotchiness on the surfaces. To brighten
the GI effect a bit more, increase the Photon
Intensity from 8000 to 16000.
Now to address the splotchiness, we need to
tackle the Radius value and GI Accuracy both
in the Render Settings. Increase the Radius to 4
and set the Accuracy to 600. You might notice
Increasing the number of photons emitted and
that a tiny amount of smoothing occurred on the increasing the photon intensity brightens the GI effect.
advTutorial 7: mental ray
Gal McGill

Spring 09 (Maya 2009)

5

walls lets increase the Radius to 8 and the

Accuracy to 900 - rerender. This shows marked
improvement in terms of removing some of the
blotchiness. The typical workflow here is to
increase the Accuracy value until you can no
longer tell that it makes a difference. Practically
speaking, starting with a value of 600 and
increasing in increments of 300 should give you
a good rule of thumb. Lets increase the
accuracy to 1200 and see if it makes a
difference even better. It is getting subtle, but
definitely still a discernible effect. Keep going to
1500 and rerender. Still barely an effect lets
now leave it at 1500.

Optimized GI

Depending on the effect youre after you could

now play with Photon Intensity (back in the
spotlight attributes) to brighten or lighten the
indirect illumination effect. Another element we
havent played with yet is light color. You are
used to changing the light color, but remember
to consider your indirect light color as well this
would be set using the Photon Color attribute. A
common way to set this for photorealistic effects
is to color pick the photon color from the light
color swatch. A final attribute to touch upon
the Exponent value determines the fall-off for
the photon-generated indirect lighting effect. By
default, this is set to 2 which corresponds to
quadratic fall-off (1 would be a linear fall-off).
You can test setting this to 1 and you will notice
how much brighter the GI effect spread into the
tube (it is also overblown!). Make sure to set it
back to 2 before proceeding.
Finally, we can start to play with the shader on
the tube wall. For tweaking purposes we stayed
with a grey to accentuate the splotchy GI effects
that are seen during optimization with textured
surfaces, one can often get away with lower
Accuracy values. Since GI photons pick-up
diffuse light on objects, changing this setting on
our tube shader should have the effect of
brightening the scene. First, apply a new
lambert to the geometry and color it light red.
Take a render. Now increase the Diffuse setting
from the default of 0.8 to 1.5 and rerender the
scene. Not only does the whole scene appear
brighter, but the indirect illumination now
extends further into the body of the main tube.
GI can be a powerful lighting approach to add
soft, indirect lighting to your scenes. However
make sure that it is warranted as you have no
doubt noticed, GI means significant increases in
render time as well as more time to optimize the
effect properly.
advTutorial 7: mental ray
Gal McGill

Exponent set to 1 (linear) instead of 2 (quadratic)

Adding a new red Lambert with the default Diffuse (0.8)

Increasing the Diffuse value to 1.5 extends the GI effect

Spring 09 (Maya 2009)

Ambient Occlusion
Ambient occlusion is another technique that
improves realism in your renders it calculates
the degree of occlusion between objects and
darkens this region based on the distance
between the objects. Unlike the GI process that
adds light to the scene by emitting GI photons
from light sources (and these photons bounce
around picking up color information as they go),
ambient occlusion does not take into
consideration any lighting information only the
proximity of objects to one another. It works by
darkening these areas and thereby adds realism
by removing light from the scene. There are
several ways to get ambient occlusion into your
renders either as part of a color pass (i.e.
where the ambient occlusion shader is part of a Default render of the handheld scene
shading network that also takes lights and color
into consideration) or as a separate pass
generated from a render layer/pass preset. We
will begin with the latter.
Open the handheld.ma file that is provided along
with this tutorial. Youll notice a device (initially
modeled as a polygon and then converted to a
subdivision surface) that is sitting a flat surface
(the benchtop). It is also broken into pieces to
facilitate applying different shaders in this
exercise (see the Outliner). Take a render from
the RENDERCAM to see what the default scene
Creating a new render layer and applying the occlusion preset
looks like.
To take advantage of mental rays out-of-thebox occlusion preset, select all the objects in
the scene using the Outliner and go to the
render layers section of the channel box column
click on the right-most icon (the one with the
little blur ball sitting atop a grey layer). This will
automatically create a new render layer for you
and automatically add all the selected geometry
to it. Notice that a default masterLayer is also
created for you this contains all of the objects
in your scene with their normal or existing
lighting shading information. Double click on the
new layer (labeled layer 1) and rename it to
occlusion.
Maya 2009 has a whole new system for setting
up and managing render layers and render
passes. Once you have created a new render
layer with the desired objects in it and renamed
it, open the Render Settings and make sure that
this new layer is selected (at the very top of the
Render Settings window, should say Render

With the pass set-up completed, you should have the AO pass
in the Associated Passes section of the Render Settings.

advTutorial 7: mental ray

Gal McGill

Spring 09 (Maya 2009)

7

Layer: your new layer). Make sure the rendered

is set to mental ray and that you are in the
Passes tab. Click the icon to the top left of the
area called Scene Passes this opens up a
window with all the available render pass
presets. Select Ambient Occlusion and click
Create and Close at the bottom. Notice that
this pass preset now populates the Scenes
Passes window. The next step is to actual
connect that pass preset to your layer select
the pass and click the icon with the downwardpointing arrow between the Scene Passes and
Associated Passes sections (the pass moves
down to the lower panel).
Now select the Features tab in the Render
Settings window and, under the Secondary
Effects, check on Ambient Occlusion. Finally,
launch a render but what you will see in the
Render View window is unchanged! You need
to go to the File -> Load Render Pass -> AO,
and that pass will now appear in a separate
floating window. They really make you work
hard in Maya 2009 to get an ambient occlusion
pass!
Notice the soft shadows that are generated at
the base of the device where it is closest to the
benchtop. Again, these are not real shadows as
we think of them because they are not a result of
lighting information simply proximity between
geometry. The default quality of ambient
occlusion in Maya 2009 is set to 256 rays this
setting can be changed in the Ambient
Occlusion panel of the Indirect Lighting tab of
the Render Settings (change the Rays to 16 and
launch another render to see the effect on
image quality).
When you are at the compositing stage in a
program like Photoshop or After Effects (i.e. and
want to integrate this pass with other passes like Ambient Occlusion pass with default quality of 256 rays.
color), the occlusion pass is typically set to the
Multiply blend mode atop other layers in the
composite: any areas of the pass that are white
(i.e. that have a value of 1) will not affect the
color pixels values of the underlying passes,
whereas darker areas of the occlusion pass
(with values less than 1 all the way to 0) will
reduce the underlying RGB pixel values and
thereby blend in only the shadowed areas.

An example of a composite generated using the occlusion preset

render pass
Spring 09 (Maya 2009)

advTutorial 7: mental ray

Gal McGill
8

Another set-up that is less common for

generating a separate render pass is to add the
ambient occlusion effect as part of a larger
shading network that also includes other types
of information (such as color, lighting etc). For
this approach, the mib_amb_occlusion mental
ray texture node is plugged into the Ambient
Color channel of a typical shader like a lambert
or a blinn for example. A convenient set-up is to mib_amb_occlusion outValue is connected to ambient color
connect the same mib_amb_occlusion node to
the Ambient Color channel of multiple shaders
(this lets you adjust ambient occlusion settings
for all the shaders at the same time).
A benefit of using the mib_amb_occlusion
mental ray texture node (instead of the new
Maya 2009 preset render pass) is that you get
more options to edit the occlusion effect in the
Attribute Editor (and use it creative ways see
below). If you select/double click the node in the
Hypershade and look at the options in the
Attribute editor, youll now notice a Samples
Example of shading network where multiple shaders conveniently
setting this is the equivalent of the Rays
draw upon the same mib_amb_occlusion node
attribute you encountered earlier in the Render
Settings (increasing the samples ups the quality
and reduces splotchiness of the render). The
Max Distance and Spread attributes also give
you more control over how the occlusion effect
is calculated.
It would be very difficult to achieve this kind of
soft surrounding shadows occlusion provides
using methods that shoot photons/rays into a
scene from lights or cameras because of the
number of rays that would be required to enter
these increasingly small spaces between
objects. Instead, by generating rays from the
surfaces themselves, the ambient occlusion
texture is able to sample areas of close
proximity.

Example of a scene where multiple shaders make use of the

occlusion texture driving their ambient color channels.

Also note that mental rays ambient occlusion

texture can be used for creative effects. For
example if you switch the dark and light color
swatches in the attribute editor for this texture,
the result will be to generate light patterns in
areas of object proximity one can create
halo/ghost effects with this technique. Refer to
Eric Kellers Abusing Ambient Occlusion: The
SciFi Scanner section (p32) in his book Maya
Visual Effects The Innovators Guide for a
more complete description of how to set up this
kind of effect.
Inverting the light and dark color swatches in the ambient
occlusion texture node and feeding the outValue of the texture to
the incandescence channel of a typical shader (i.e. Lambert)
advTutorial 7: mental ray
Gal McGill

Spring 09 (Maya 2009)

9