3D animation and Motion Graphics
Compositing Maya 2009 Render Passes with After Effects
I’ve been experimenting with Maya 2009′s Mental Ray Render Passes feature. It’s intended to easily allow separation of different elements of the render into separate images. An easy example would be separating the diffuse light from the shadow. In practice it seems to be way more finicky ( or broken) than it ought to be. After many trials I’ve developed a workflow via After Effects that successfully composites all my passes so that they just about match the default composite render.
Step 1. Understanding The Problem
There seems to be a bug with the color clamping in most Maya image output file formats when using render passes. Even with most of the file types that support higher bit depths there seems to be a tendency to clamp down to an 8 bit range, and also do it incorrectly (for compositing needs). The shadow pass is the first area where I noticed this. Instead of a “normal” shadow pass that only involves black and white values, this Maya Mental Ray shadow pass now includes hues from the scene and is also luminance inverted.
This is the shadow pass image as it comes out of Maya.
The correct and normal method to composite this type of shadow pass would be to combine or merge it on top of the diffuse color pass using a “difference” apply mode. This almost works, but for some reason (improper clamping) it causes inverted colors to appear in the deeper shadows.
This is how the shadow composite should not look. Notice the incorrect green hues in the deeper shadow areas.
This is the Beauty Pass. This is what the shadows are supposed to look like.
This clamping issue causes inconsistency between the different color layers that need to fit together just so in your compositor. This bug also affects other render pass types. Direct Irradiance pass doesn’t composite properly with diffuse material color pass. depth maps do not create proper values over the entire scene area, instead only the front part of the scene. A whole mess of problems emerge. So now you are stuck using a 32 bit file format to get those render passes to work right. Well that’s fine.. but then on top of that it turns out that only one of the possible image file format works properly, and the rest clamp down to 8 bit even though they may be theoretically compatible with higher bit depths. EXR is our savior. For some reason there seems to be a bug with many other file types such as targa, tiff, even iff, do not work right, even if the frame buffer is set to 32 bit. Until Autodesk fixes this issue, we need to work around it.
Step 2. So Now What??
Here are the steps to output 32 bit EXR sequences out of Maya.
1. Maya-> Render Settings-> Common tab-> Image Format-> OpenEXR
2. Maya-> Render Settings-> Quality tab-> Data Type-> RGBA [float] 4×32 Bit
Now you should end up with a sequence of EXR files, either a bunch of different passes in separate files or a ton of channels all crammed into the same file. Here’s how to deal with them inside of After Effects.
1. After Effects->Import->select your EXR sequence passes
2. Lay your passes onto the timeline. If you’ve made one EXR with all your passes inside of it, then make a duplicate layer on the timeline for each pass type you made.
3. Select each layer and apply Effect-> 3D Channel-> EXtractoR
This plugin is now included with After Effect CS4 . If you have an earlier version then you can still download it for free from this site. Also, I find that version 1.3 is much more stable than version 1.2 (that shipped with CS4), so you may want to update in any case.
4. Click the EXtratoR Channel Info Box in the Effect Controls tab. Here you will need to pick the R, G, and B layers that are appropriate to the pass you are trying to display in each layer. If you have separate EXR pass files then there will probably only be three choices in each, but if you have a composite EXR with all your passes then you may have a very long list. You can see how I chose the RGB shadow pass channels for my shadow layer in the image above.
5. Select the appropriate apply modes for all your layers. Shadow should be on Difference, Specular should be on Add, Direct Irradiance should be on Multiply.
At this point you should still be seeing a messed up looking composite that does not match your beauty pass- but we’re almost there! We need to set the entire project to a 32 bit color space.
6. After Effects-> File -> Project Settings -> Color Settings -> Depth -> 32 bits per channel (float)
Now at last you should have a 3D composite that matches your beauty pass!
| Print article | This entry was posted by aloedesign on November 18, 2008 at 6:45 pm, and is filed under Tutorials. Follow any responses to this post through RSS 2.0. You can leave a response or trackback from your own site. |
about 4 years ago
Heya! So what if you comp your result with the beauty with a difference operator. And after that crank up the exposure. Any difference? Or just total black as one would expect…?
about 4 years ago
It isn’t total black on my end either. in fact there are visible outlines around some of the geometry even before the “crank up the exposure” part. eyeballing it here in my comp, my guess is the direct irradiance pass is mostly to blame for my geometry outlines.. but maybe all of them are a little off, who knows? After the exposure change, things look red with white outlines, and it appears that in comparison master beauty has less noise in the difference than beauty. Matching the beauty passes to that level of perfection isn’t critical for me- but it’s true that it should match, and that they should fix it!
about 4 years ago
thank you so much for sharing what you have discovered with this rediculus bug.
When I render with the EXR settings, all my images output black.
Is there another setting I over looked?
Thanks,
F
about 4 years ago
Hi Frank,
Does the exr work as expected when you do not assign any render passes?
Did it ever work for you previously?
How are you trying to view the exr files?
about 4 years ago
I also tried switching each individual render pass attribute to 32-bit. It I still get black. .
will ned to research this more!
about 4 years ago
Hi
I cannot manage to get a shadow pass out of my scene that has objects instanced to nParticle systems.
Many of the Illumination passes get through and are saved out in a single EXR file, and in a test scene I made did get a raw shadow in an EXR when using depth map shadows, however the channels were all mixed up. In my production scene though, no matter how I try to setup the lights or optimize the scene, I cannot for the life of me, get out a shadow pass.
This behavior is happening on both Mac OS X and windows XP Pro x64 versions of Maya 2009.
Any info will be appreciated because in this particular shot we need to have an additional shadow pass and after 3 days of poking and prodding at maya’s settings we have gotten nowhere.
Thanks
D
about 4 years ago
Does the particle shadow show properly on a composite render?
I would suggest not trying to use a render pass for the shadows (ever- in maya 2009 since it’s so broken) you set up a separate render layer and there you should set up a shadow render according to the method used in versions previous to maya 2009. Autodesk support can walk you through the old method if you aren’t familiar. I have not experimented with instanced nParticles, so I can’t offer a workaround if there is a bug there.
Hope this helps! Good luck.
about 4 years ago
Well, you don’t see the particles, basically our poly geo is taking the place of particles using maya’s instancer. But shadows do show up on the geometry in beauty passes. All the other MR shaders work, and we are using mia_material_x_passes for our geo, btw.
By the old way do you mean just ticking off that box in the render layer attributes? That doesn’t always work either
The only workaround I could think of was maybe creating a white shader and adding that to the objects, shutting off FG, render that and and multiply it in post.
D
about 4 years ago
the old method of “render layer shadow” is that there used to be a render layer preset for shadow renders. now it is disabled and replaced with broken render passes, but it still exists in the application if you use mel:
creating a new render layer and then:
renderLayerBuiltinPreset shadow Render_Layer_Name;As for your workaround- there are various things you can do if you want to set it up manually. white would work okay but you might also experiment with a “use background” shader to receive the shadows in a way that involved an alpha.
Here’s a third way: http://www.3drender.com/light/shadpass.htm
about 4 years ago
Jake,
Just upgrading to Maya 2009 and therefor SPa1 too. So I don’t have much testing time at this point but seeing the issues here but thought I’d throw out that under the MR Globals>Quality Settings>Framebuffer>Color Clip…
Maya is set to RAW unlike RGB modes will clamp 0-1 color space and affect premultiplication of rendered images. I’m guessing this could end in a result where a clamped render might get close to a Beauty render or be way off based on the particular scene. Wondered if you’d investigated this yet with the new renderer? Don’t have time myself but if I get to it will post my results then.
Thanks for your help on this topic!
Jason
about 4 years ago
I see, it uses maya software renderer. I can’t believe I forgot the use background shader. I’d stopped using it in past versions because it had some quirks with MR. Going to give it a whirl though. I used to use that to grab shadow and spec.
thanks for the help
D
about 3 years ago
Hi,
My AE CS3 keeps crashing when I try to import the exr sequence. Does anyone know why this happens? I tried downloading the plug ins from the proexr website and placing them in the respective folders. I have a MAC. The website doesn’t do a good job explaining where to place the replacement plug ins.
Has anyone needed to troubleshoot this?
about 3 years ago
This was happening to me with the extractor plugin that shipped with after effects. I put the new one in
applications > after effects > plug-ins > effectsreplacing the old version, and then it worked after that. Hope this helps!
about 3 years ago
for the render pass depthRemapped and depth in mental ray EXR 32 bits work great because it seems to be working in the original maya units i was working in respect to distance, if you see a black image for depth pass then you need to clamp to to find the correct values, to do this add the extractor then a levels, change the extractor values to depth and then look for the black point and white point range close to those you use in maya clip planes then you will be able to see in the levels historigram effect the values appear, then just clamp the values to the desire range using levels. hope this helps, this tut just made my life so easier because i had so much problems with this subject in the past THANKS
about 3 years ago
From the Maya 2010 docs:
Colorclip
Controls how colors are clipped into a valid range [0, 1] before being written to a non-floating point frame buffer or file.
In all modes, the RGB components are clipped as specified by the desaturate option. The RGB and alpha modes ensure that the resulting color is a valid premultiplied color.
Clipping occurs only in 8-bit integer and 16-bit integer frame buffers. if you are using float and half-float, no clipping occurs.
RGB
RGB is first clipped to [0, 1] and alpha subsequently to [max(R, G, B), 1]. Use RGB if the alpha channel is considered less important than preserving the RGB color and intensity.
Alpha
Alpha is first clipped to [0, 1] and RGB subsequently to [0, A]. Alpha mode is intended for alpha compositing, where the alpha channel is more important than the absolute color value to preserve correct transparencies.
Raw (default)
RGB and A are both clipped to [0, 1] independently of each other. Use Raw mode only if no layering based on alpha is going to take place. This mode turns Premultiply on, so use it with care because shaders might receive colors that cannot be composited in standard ways.