Join Ru Kuwahata for an in-depth discussion in this video In depth: Compositing 2D and 3D animation, part of The Creative Spark: Between Two Worlds, The Hybrid Animation of Tiny Inventions.
Scott Erickson: So, this is a transitional scene in the movie. It's the first time you see this character. There's another character who's waiting for her to arrive. And then we wanted, we were close in on the previous character, so we wanted to open with a wider shot. So, this is just really rough storyboard. I'm just blocking out different ideas for camera angle. So once we were generally happy with how the shot fits into the context of the overall sequence we begin to plan out the animation.
So she's running, she takes a deep breath and the camera movement, which is just a very subtle drift. Once we're, we were happy with the camera movement and the character animation more functioning in the context of the scene we would actually go and shoot this with Stop Motion. And because we planned it in the computer beforehand, we knew the exact number of frames necessary for the, the final camera movement. So this is the raw file of the scene and it maybe difficult to see but we actually introduced a focal shift as well.
So we started on with the bicycle sign in focus and we end up shifting focus to the the woman who is running into the scene. And we thought that would help with the transitional function on this shot. And we knew that this stop motion or the frame by frame camera movement would have to fit with the CG world. So we made a choice early on to shoot the stop motion as smooth as possible. So we shot it on ones, so that means that there's 24 unique photographs per second of animation.
So rather than trying to make the CG fit with choppy, or stop motion, we wanted to make the stop motion fit with smoother CG. But there are some inherent sort of aberrations that happen in the shooting process. One of them is the camera track is not perfectly smooth, so there's lots of little glitches along the way as the camera's shooting. And the other is when a camera shutter opens and closes, the aperture fluctuates slightly.
And when you look at still images next to each other, you can't see that fluctuation. But when you play them in sequence, it comes across as a flicker. So, to fix that we have 2 filters that we use in AfterEffects. The first one is the warp stabilizer is bundled with any version of AfterEffects post CS 5 I believe. And it's a great, great filter to, to remove some of the bumps and glitches.
The second plug-in, GBdeflicker is a paid plug-in and it's used quite frequently by time lapse photographers. And it does a terrific job removing some of the flicker that you see in the aperture fluctuation. The next step of the process is we pull a key on all the green. So we basically replacing everything behind the initial set play with a larger map painting. So once the key was pulled and we knew that everything is working, we would do a camera solve for the scene.
And that's basically just rebuilding this physical camera so that we can bring it into the virtual computer graphics world. And we use a plug-in called Camera Tracker from Foundry and we do all the tracking directly in AfterEffects. Cinema 40 and AfterEffects work really nicely together. So once we have this file we do a little test with solids and After Effects to make sure the solids are sticking to the ground can just go up to File>Export and kick a cinema 4D file right out from our AfterEffects.
So the actual character animation was created in Cinema 4D and we have all of these rig spines which are the different controls, so we can basically treat her like a virtual puppet. The camera which we imported from AfterEffects is synchronized perfectly with Start Motion World and the lights are also based off our real-world lighting from this scene.
And we'll even go so far as to name the lights, based on real world reference points. So we have like a foam core bounce, a floor bounce a key light, a back room, a second back room. From Cinema 4D, our rendered files come into AfterEffects as these individual character elements. From there, we go in and fine tune all of the details to make sure the stop motion world and the computer world are synchronized.
So the shadows are sent out of Cinema 4D as a separate render pass. But they don't synchronize with the stop motion site yet. And that's something that we do in AfterEffects. You can see, this is just a flat shadow. It's not wrapping around the curb. So to to make that happen, we'll actually build a very simple gray scale representation of the height of the scene. So the white of the curve is higher than the black of the street.
In using that information we can create this bend. When we first started out on this project, because it's about the passing of time we knew that we wanted light to be present in every step of the process. So I knew that in the compositing phase because light was really the focal point of the overall cinematography, shadows will be really important. We want to the shadows to feel as believable as possible. Just going to the extra effort of wrapping the shadows around the building or the curb it helps the audience just feel that the light and the characters are one thing.
Once we've finished the the rough composite and this is full frame, because we can see it's more of a square composition than the standard 16 9 we do a color grade and we don't do too much in the color grading process other, other than kind of desaturating it a bit. We want it to feel sepia and nostalgic without being too overtly so. I think this for me is, is a frustrating part of the process because it's so clear when things aren't right.
And, if they're not right I, I feel like I haven't done my job and I feel like it's just not going to work. But when things come together in a, in a harmonious fashion it's, it's really satisfying. And the photographing these sets is just magical because for almost a year, we were used to watching our cut looking like this which is so far away in tone and feel from the finished product or from the finished film that we almost forgot that there would be like real textures and real light and like emotion that's tangible, and when you watch it in the final scene, it is just, it is for us just the most magical part of the process.