How Netflix Klaus 2D Animation In 3D

When the animation process for Netflix’s “Klaus” began, it looked something like this. But then the animators went one step further to create a film that looked like this. Suddenly the characters looked three-dimensional. But unlike most animated movies these days, the characters in “Klaus” aren’t CGI and can’t even be considered 3D. It’s all just a trick of the light. About 300 people, including 40 animators, worked on the movie “Klaus,” which took over two years to make just like how long time to select the game of thrones bastard names for the popular television series game of thrones. And it was completed under the wire, just one month before it premiered on Netflix.

So, why did it take so many people and so much time? To understand, we have to go back to 2010, long before “Klaus” was nominated for an Academy Award, to when director Sergio Pablos came up with the idea. Because his story was about the origin of Santa Claus, it appealed to nostalgia. And he thought a nostalgic,2D animation style as we saw in the ’90s Disney films would be a better fit for the story. But he also wanted to advance the look, so his team at SPA Studios in Madrid added a few new crucial steps to the animation process. Sergio Pablos: I never looked at 3D as an evolution of 2D. I looked at it as a split, like there’s a new way of making animation now.

Narrator: First, they storyboarded the script and made a cut using temporary voices for the characters. They swapped these out later once the real cast was recorded. The next step was the layout, where the team designed backgrounds and figured out the placement of the cameras. Animating the characters and coloring the backgrounds happened simultaneously. The end goal was to have both blends together seamlessly and look like their part of the same world.

The characters were all hand-drawn using digital tablets and a program called Harmony by Toon Boom. The animators used live-action reference videos of themselves as a guide. The initial sketches were very rough, as you can see here. But there was a cleanup stage in which artists refined the drawings with crisp, bold lines. Then they painted the characters with basic flat colors. Here, everything still looks very 2D, but they will soon bring the characters to life with a very important addition usually reserved for 3D animation: lighting.

His team tested out a new method of lighting 2D characters and released a two-minute, 30-second-long proof-of-concept teaser back in 2015. The proof of concept looked good and secured them a deal with Netflix, but the process was too labor-intensive. So they partnered with a French company called Les Films du Poisson Rouge to help advance the technology, which they called Klas, short for Klaus Light and Shadow. Poisson Rouge was able to make the tool much more efficient and easier for the artists to work with. The Klas tool allows the artists to paint with light using a number of different types of lighting in various combinations, like “key light” and “ambient light.”

With 3D CGI, light is added automatically to objects, but it’s trickier with 2D. Pablos: Well, with the lines, with drawings, the computer needs a certain level of AI to even understand that this line corresponds to this line and this hand is also this hand. Narrator: The software tracks the movement of the characters so the light and shadows will move with it. The program takes a very educated guess, but it’s not 100% accurate, so the artists can go in and fine-tune it by hand. Painting with light allowed the artists to get creative with details down to the tiniest reflection in their eyes, as you can see here. The team used lighting not only to make the characters feel more real but also to help tell the story.

For example, when Jesper is handing out papers to the kids like a drug dealer, he’s always standing in the dark to illustrate his shady behavior. And when he’s exposed at the end of the film by his father, he’s the only one standing in the light, while the others are in the dark. Inspiration for detailed lighting techniques came from movies and TV shows, like using just a sliver of light to illuminate a character similar to this scene in “Apocalypse Now.” And this scene, when Jesper confronts the bully, was inspired by “Breaking Bad.” It’s important that the backgrounds also look three-dimensional and follow the same lighting pattern as the characters, so they used “color keys” as a guide. Pablos: They’re quick doodles, you know, they don’t have a lot of detail, but they tell you exactly what the light direction is and how it’s gonna affect both characters and backgrounds.

Narrator: For example, this color keys show how Jesper and Alva will be backlit by the sun coming through the window. And this one shows a progression of how the light will change on Alva as she steps towards Jesper. To make the backgrounds pop and appear 3D like the characters, the animators used several different techniques, such as multiplanes, where you have layers on top of layers to give the illusion of depth. The team created a total of 3,160 scenic layouts for the movie. After they’d merged the characters with the backgrounds, they used a second major step that really gave the 3D characters that intricate detail to bring them to life: texture.

With another tracking tool, they used to contour, lighting, and motion to add various effects to specific parts of a character. Pablos: So now you could say, well, I don’t want a lot of roughness on the skin, but I want the coat to feel rougher. Narrator: For example, they can make them look like an oil painting or a watercolor. These textures are subtle, but if you look closely, you can notice the difference. In the end, the characters looked much more 3D and like a part of their environment, as opposed to looking like stickers on top of an elaborate painting.

Pablos: And that’s what throws people off when they say, “This is 3D,” because it’s volume and its moving and it has texture. But it’s really a combination of the light and the texture that makes that illusion. Narrator: The final stage is final composition, in which any last-minute details are added to the image, like particles. While the majority of the film followed the 2D process, the animators did use 3D models for some characters and objects and combined the two seamlessly. And even though these were created using CGI, they were lit the same way as the 2D characters were: by hand. Pablos: There’s things that benefit from being drawn because they feel more organic, and there’s, you know, things that are not supposed to look organic.

There are things that are supposed to look solid, like wagons and doors and props, and it’s really hard to make it feel consistent and solid through drawing. Narrator: If you look closely, Jesper’s wagon is 3D, and so are some of the reindeer. Pablos: Whenever the reindeer had to do something, it was particularly challenging for 3D to look right with the 2D. We just animate the reindeers in 2D, and sometimes we would animate one reindeer in 2D and the rest in the shot in 3D. Narrator: Scenes like this chase scene at the end of the film so seamlessly combine 2D and 3D elements that it can be difficult to tell which is which. Pablos: There was a shot where Jesper lifts a plate cover at one point, and I commented on how good the integration of that 3D plate overlooked with the 2D actor, and they said, “Oh, no, no, that’s actually 2D, we just painted it to look like metal.”

Technology