Shifting animation's creative focus

Answering these questions from quarantine, Head of 3D at The Mill, New York, Ruben Vandebroek, speaks about how to stay focused, sharp, and keep your animators happy while working from home.

We know it’s the question on everyone’s minds, so we’re going to get it out of the way first - How do you feel COVID-19 going to change the animation industry?    

Many things are already changing, but change isn’t always bad and I definitely think we can implement some positive learnings from this. First, more clients are asking about different solutions, ranging from re-using live-action footage to augmenting stock footage, and matte paintings to fully animated CGI content.

A second obvious change is that we are all working from home. We are connected remotely to our machines in the office, giving every artist exactly the same functionality and access to all our software and proprietary tools. We communicate mostly through webcam, group chat and discussion boards. I hope some of this workflow will stick around after COVID-19 and we can implement some of these elements into our regular workflow.

 If one little thing is off, the viewer will pick up on it.

Do you think clients are more willing to allow experimental animation?

We are definitely seeing more interest from our clients in the animation area and CGI area, however, I think it will take some time for clients to adjust to new creative briefs, technical possibilities, and ways of working. I think it’s important to use technology in the right way for the right project. The idea and concept should lead, and the execution should be tailored to that idea. Utilizing technologies and solutions because others are not available is never the best way to achieve the desired result. 

How do you build an agile team of creative artists

Working in the advertising space, we have always been used to fast turn around times and changes on a daily basis. Due to the nature of our business, we’ve relied on artists with a generalist background for years. Even though we have many specialist artists on the team that are experts within their discipline, many have more than one skillset which is crucial to the efficiency of our team. Having a team of strong 3D generalists means artists can jump between projects and help out where necessary.

How are CG artists responding to the increased pressure and workload?

Artists are dealing with it pretty well given the circumstances. I think we all understand that this situation is out of our control and everyone is working together to get through this. On a positive note, working from home has a lot of benefits for many artists in that they feel much more efficient with less office distractions and no commute to worry about, giving them more time with their families.  

We have to match all the properties of the live-action shoot, the lighting and lensing, making sure that there is proper interaction between the CG elements and the live-action

Are you reusing older live-action pieces with additions from animation or computer graphics to get around some of the restrictions on shooting?

Yes - one of the key solutions we have presented to our clients is the use of older live-action plates, stock footage, and custom digital matte paintings augmented with 3D assets. The combination of these elements can often be enough to create a unique piece that delivers the desired message. It is also a very budget-friendly solution, as it does not require us to create all the content from scratch. We often rely on this solution when there is a limited turn around time.

How do you successfully integrate VFX into live-action without it looking fake, or worse, falling into the Uncanny Valley?

To be able to successfully combine any CG element into a live-action plate, every single stage of production and post-production has to be properly planned. From the shoot to the final compositing.

Machine learning has the ability to capture a full performance in such a way that previous techniques haven’t been able to.

It all starts with a good concept of the actual character, creature or object you plan to integrate. When designing something using CG (because it doesn’t exist in real life) we always ground the design in real-world elements and references. For example, when creating a dragon we look at alligators, bats and other animals to take reference and ideas from, the more elements and features you use that stem from the real world the more grounded in reality it will feel.

The same goes for texturing, animating, and lighting. Besides a good concept, we have to match all the properties of the live-action shoot, the lighting and lensing, making sure that there is proper interaction between the CG elements and the live-action. This could be as simple as ground interaction, or more complex like interacting with a CG creature. Every little detail helps sell the final picture.

These are skilled artists that can direct an animation team in achieving the desired results, breaking down animations into manageable chunks and are able to identify what works and what gets the message across.

Viewers can easily tell when something looks fake but identifying what exactly it is and how to fix it is something that is a lot harder. This is where the Uncanny Valley comes in, a term usually referring to CG photoreal humans.

We are in contact with people every day, all the time, so we know very well what they should look like, this is why it’s one of the hardest things to achieve in CG. The amount of subtleties in the way skin catches the light, muscles moving underneath the skin, movement of the lips and the look into someone’s eyes, they all tell you something different, an emotion, feeling, to be able to portray a photo-real human, all these little details have to be correct, and they also have to be correct in the context of the shot. If one little thing is off, the viewer will pick up on it.

How do artists lead the animation process?

Everything starts with a pre-viz based on storyboards or an animatic. This is where you roughly layout all the shots, it has basic representations of the environments and characters. If there's live-action involved, this work usually happens before the shoot. Once a pre-viz is approved, we start blocking out the shots with our characters -  we block cameras and build our edits.

 I think it’s important to use technology in the right way for the right project.

The next stage is where we start animating in more detail, between the blocking, making smooth animations. We also start adding secondary animation, the subtle details that create realism and emotion. During this process there is a lot of back and forth with our FX department, the artists that will run the necessary simulations required for muscle and skin deformations, cloth simulations, etc. After simulating, we often have to adjust objects using manual animation to make sure it all works together in a coherent way.

This entire process is lead by an animation lead or animation director. These are skilled artists that can direct an animation team in achieving the desired results, breaking down animations into manageable chunks and are able to identify what works and what gets the message across.

How do you design an animated character that will resonate on screen? How do you tailor your character to an audience?

It all comes down to the story you want to tell and what kind of personality you want your character to have. Is it supposed to be funny or does it have to be scary? Is it photo-real or cartoony? Which audience are you trying to reach - children or adults?

When designing something using CG (because it doesn’t exist in real life) we always ground the design in real-world elements and references.

These are all questions that need to be answered before production starts and this will inform the path to take. When designing your character you have to keep in mind all the next steps in the process as well, how a character will move will also determine how it has to be modeled. Do we stick to the forces of nature or can our character move in impossible ways? If our character is cartoony and can be animated in stretchy ways, what happens to properties of the surface that stretches? All these things have to be all thought out so that it all makes sense when it comes together in the end.