Discussing the green screen workflow on Sesame Street

- All right, so for those at home we have Super Grover helping a duck across a river. - Yes. - Okay. - Super Grover 2.0. - Super Grover 2.0 helping a duck across a river. So we have a little bit of an intro, and then into it. - He observes... - Oh look! Look, arrrghh! - He questions... - Hubba-wha? - He investigates... - Hmm, what does this button do? Arrrggghhh! - Super Grover 2.0. He shows up. Somewhere by a woodland stream, one little duckling is about to waddle into a big problem. - (laughs) Yippee! Oh boy, oh boy, oh boy, oh boy! I can't wait to get to the party. It's going to be so much fun! Oh, a stream? And the party is on the other side. Oh no, how will I never get across? Help! Help! (crash) (groaning) Super Grover 2.0, you showed up! - It is what I do. So, what seems to be the problem, little one? - I'm trying to get to that party over there, but I don't know how to get across the stream. - First, I shall unleash the power of observation. - Oh, I don't want to miss the party, oh. - Mmm, I see with my super eyes that you walk like a duck. - Quack! - And I hear with my super ears that you quack like a duck. - Quack! - My super powers of observation tell me you are a duck! - Of course I'm a duck, quack. - Well, then do not be daffy, Duck. You can swim across the stream. Everyone knows that ducks can swim. Yes, but, if I swim across, I'll ruin my brand new-- - So take us through the process of creating something like this. What happens when you're working with the green screen, and, well, take us from the very beginning. So you have obviously green screen material, and then? - Right, yes. Well, this piece in particular was directed by Matt Vogel, who is a puppeteer, he is also a performer. And I mention that because I think there's a great advantage, we have incredible directors, but someone like Matt Vogel is just absolutely outstanding. Because he understands the challenges of what these performers are going to deal with, because behind them is nothing, it's green. So when we're on, when we shoot this on stage, the 3D team that makes these backgrounds will give us temporary backgrounds. Kind of a low-res example of what we're intending it to be. And the performers are seeing that temporary background in their monitors while they're performing. So they can see, they can get an idea of what they're performing against. I work in the control room with the TD to help provide him with the temporary backgrounds, but then also to create some temporary moves. Like as you saw at the beginning, we move around the duck to where she ends up at the stream. So all of that is incredibly helpful for the performers to be able to ground themselves, again for eye line, and also to understand where in the environment they are. Some of the 3-dimensional work can be so complicated that it's helpful to have an editor on set in order to make sure it's working. Because they might be interacting with a completely 3-dimensional character, or stepping onto a 3-dimensional plane that doesn't exist but will then later be replaced. And so I'm there to help provide that automatic, you know, answer to whether or not it's working, if they need to reshoot something. But then also a piece like Super Grover 2.0 is a specific time, and so I'm there to help trim it back down. - Okay. - But once this has been cut with the temporary backgrounds, I then take it to Sesame Workshop and work on the real cut, the real version of it. Again with temporary backgrounds, but then deliver it on green to our 3-dimensional team, who will then go back and replace the backgrounds and put them into these worlds. - And then, so that's really great that you're also able to often be on set and advise in that way. You talked about reshoots. Are you talking about in that moment where you have, you're consulting and you say, you know, "That really didn't work, I need you to reshoot it now," or are you talking about down the line. - Oh, no, yeah. At the moment. - Yeah. - So the second that they say "cut", I'm digitizing on the fly-- - Right, okay. - In the control room, and I'm right away trying to put it together, and I'll tell them, you know, some sort of note that it looks like she's looking the wrong way, or-- - I see, okay. - Any way I can help possible, that way they can continue to move as fast as possible. Because they're achieving a piece like this, which is this one is seven minutes, with, on green, but doing it in a day on top of other multiple things that they're shooting. So it's an incredible amount of work. So any way that I can help educate that, that they're achieving what it is that they're intending to, then I'm on it. - And what do you like the most about this and what are the biggest challenges in this sort of workflow? - I like, I love, when we shoot on green, I personally absolutely love and adore our set. I'm happy to be on a production, a children's production that is still, you know, grounded in a real world. But I love when we shoot on these 3-dimensional backgrounds because it allows our characters to be anywhere. We can put Cookie Monster in the Star Wars universe, we can put Super Grover on a place like this. But for me, at its (mumbles) it's gratifying because I get to help decide, maybe the stream was in a different place when we shot it with our temp graphics, and then with the director I'll work on, okay, actually we should move it over here or, you know, help decide not only the performance and the pacing and getting the piece to time and working, but also the background. Educating the 3-dimensional team on placement of things and where we're intending the shot to be and how far away it should be, and since now we want this to be wider than before, I'll then put, maybe add additional, temporary footage to help educate them on how the world should be that they're built. - And when you receive the final 3D background, or is it final, like can you consult with the motion graphics artists that create it and alter it in any way, do you have that authority? - Yes, yeah. The team that builds these is called Magnetic Dreams. They're out of Tennessee. - Okay. And just incredible, professional team. I mean, they achieve masterfully everything that they do, and they're actually able, they're able to make it look photorealistic, but also Muppet-y, in a way. So that way our characters feel like they belong in that world. But they're incredibly humble, generous people. And there's a constant dialog between us-- - Wonderful! - Whether or not what I delivered to them is working, or if there's a problem with the XML or the EDL that I provided. And again, like, being able to send them temporary refs, I mean, throughout the entire process, all the teams are talking to each other. - That's great. So what's the timeline, from green screen shoot to when you get the 3D background back and the dialog that occurs within that? - In terms of creating the green cut, that's pretty quick. - Mm-hm. I'll usually cut that in, if all goes well I'll cut it in about a day, and then work with the director to achieve their cut, which can take another maybe half a day. And then eventually with the producers and our research team. And once all that has been agreed upon, it's sent over to the 3-dimensional team, and we move on. So they're working in tandem with us, creating these backgrounds and these pieces. So for them it ranges on how long it takes, depending on how much other work they have from us, and also how complicated the backgrounds are for them. But the second that I've delivered to that team, I'm moving on to whatever's next. - And then you get it back, and make tweaks, and they make tweaks if necessary. - Right. - Then you arrive with the finished product, and then it sits until you're onto your second part of the job later on in the season. - Correct. - Which is reassembling everything. - Right, right, right. - Wow, okay. - Yeah, so part of the year is creating and collecting all of these pieces, and then later putting that puzzle together for the final broadcast. - Very nice. - And I can't, I have to, I have to say, I love my job and I believe it's very crucial to the process, but, I mean, we have just the most incredible directors and producers and writers that make my part so much fun. I mean, I really do feel, it's still surreal for me to love what I do. And because there's so many different pieces to make up that final show, every day is a completely different requirement for me. So one day could be straight editing, another day is doing particle effects and after effects, another day is dealing with keying out a celebrity. And so always, every day is a new adventure. But they make it enjoyable because they consistently give 110%. And so really for me, it's this feeling of pressure to show up to the level that they have already shown up to by the time it gets to me. - Well, that's quite a compliment for the rest of your team, and that sounds lovely. (laughter)