Making light work: the DPs new tool box

Complete control over your environment is a dream shared by the hoary old magus magicians of old, as well as their modern-day equivalents, the directors, producers and cinematographers of the film world, with a stack of riches to play with and immortal stories to tell. 

Virtual production is as much part of the new landscape as a traditional location, or a four-waller studio, and is here to stay.

The art of film is not much more than a century old – in three years’ time, we can mark the centenary of what is widely considered the world’s first feature-length talkie, The Jazz Singer – and for much of that time, controlling your environment according to your will has been a process fine-tuned for the vagaries of location and the limitations of sound stage, studio and camera. 

ARRI's VP event for the APA. Image courtesy of Ian Wallman.

But recent, much-heralded advances in virtual production, and the manifestation of that new liminal space in filmmaking, the Unreal Engine Volume film set (which may look like a random word salad to the uninitiated) promise complete control of the filmed environment generated in real time, on LED panels, and delivering the usurpation of not only the green screen, but of location shooting itself. 

The location is coming to you.

You’re still dealing with storytelling, with talent interacting with their environment. The craft of cinematography is still very much at the forefront.

 Not that virtual production stages herald the end of real-world location shooting. Yet.

 As Rob Payton, Production Specialist and consultant for ARRI Stage London, says, ‘Real world’ is something of a misnomer. “Productions have always taken a location, or a studio, and embellished it through art direction, lighting, and set dressing to match the script and the director’s intentions. Virtual production is as much part of the new landscape as a traditional location, or a four-waller studio, and is here to stay.”

Above and main image: Chevrolet spot directed by Brett Danton.

Arri has a good back story. Founded in 1917, it sold its first film camera in 1924, its first digital camera in 2005, and its first LED Sky Panel in 2015. The Arri Stage London opened its doors in 2021. Rob Payton was there waiting. “Many talented and award-winning DPs have shot there,” he says, “and all use the volume differently. Our advice is to treat it like any other location. You’re still dealing with storytelling, with talent interacting with their environment. 

[VP] has added a whole new dimension to filmmaking and given filmmakers and creators an additional tool to craft their stories. 

“The craft of cinematography is still very much at the forefront. And the traditional skills of lensing, lighting, and framing remain at the core of any successful VP shoot.”

Will Case, ECD, at global AV supplier Creative Technology, sees VP stages as a key to unlocking the previously impossible. “It’s added a whole new dimension to filmmaking and given filmmakers and creators an additional tool to craft their stories, allowing them to film in environments that were once difficult, or even impossible, to shoot in.” 

I use my knowledge of real-world lighting and put it inside the 3D world.

DPs, he says, can approach virtual production as they would a traditional shoot, bringing their craft and creativity to the virtual stage just as they would on location. And neither Case nor Payton believe VPs spell the end of the location shoot. “It will always have its place,” says Case. “Virtual production is a powerful tool that can complement traditional shoots. It’s about giving filmmakers more options. It’s helping to open up new creative doors.”

For Payton, those doors are opening up the real world to the infinite possibilities of the metaverse. “Realtime rendering, AI, and advances in the metaverse are going to fundamentally change the way that we create films,” he says.

Above: Brett Danton directed this video for Bastille using a volume stage built inside an aircraft hanger.

Director and cinematographer Brett Danton is an ambassador for Canon in between shooting and directing his own work for NM Productions, which includes a range of high-end car spots. He’s also working with AI powerhouse Nvidia.

Having wielded Canon’s new C400 around the Arri stage a few times, he is well placed to assess and critique the impact of VP and mechanical learning on the cinematographer’s art and craft, and on the future of film itself. “I feel there’s a new job title coming in – Digital DP,” he says. “I wouldn’t be as good at the work I’m doing now if I hadn’t come from a traditional filmmaking path. I use my knowledge of real-world lighting and put it inside the 3D world.”

The guys who are building this stuff are games engineers, and they make it look like a game.

To highlight VP’s real-world usefulness, he points to a recent car shoot that would in previous times have been shot in the Namibian Desert. “But when I went to Namibia, where Mad Max had been shot, the government had changed the rules because of the damage that had been done – all the animals had been scared off, and it took three years for them to return to the location. Now they only allow a maximum of 10 people in there, so you can’t shoot a car ad there anymore. The only way is to create a digital twin of that environment. And bring that back to the stage.”

The Chemical Brothers promo for Live Again, shot at ARRI Stage London, utilised VP to tell a story that could not be achieved in any other way. Still courtesy of Will Case at Creative Technology.

That conjoined digital twin needs some deft surgery to fully come alive, however. And that, says Danton, is down to that barest necessity of any film – the light. “The first thing I’ve done with 90 per cent of the scenes I’ve got is to throw the lighting away. Particularly the Unreal scenes, because the guys who are building this stuff are games engineers, and they make it look like a game.

I feel there’s a new job coming along, and it’s very exciting – I can now control the whole pipeline, whereas before, DPs were only getting to shoot.

“But I want to make it look real world. Now, when I’m shooting scenes in the real world, I know how I can light that in the digital world. That’s why I feel there’s a new job coming along, and it’s very exciting – I can now control the whole pipeline, whereas before, DPs were only getting to shoot. Now I can be in charge of all the special effects, or all the composite work. That will get more and more common. And we can see where it will head – back into the camera.”

You’re seeing cameras coming out with 15 stops of raw information, so you’ve got more options to make decisions from within the camera.”

So does that mean the future of cinematography will encompass post? “I think so. What I’m doing now is adding a 3D element on top of the grading, so it’s becoming a multifaceted job again. You’re seeing cameras coming out with 15 stops of raw information, so you’ve got more options to make decisions from within the camera.”

Nvidia recently released this video demonstrating its collaboration with WPP to create 3D generative worlds. 

At first, says Danton, he was unimpressed by the background image and quality of light in the VP environment. “But it’s getting better all the time. We’ve just worked with some 10K path-traced backgrounds, and they look almost real-world. But it’s tricky lighting-wise, so you do have to think about how you’re going to light stuff.

The computer is telling you that it should be accurate, and your eye is telling you it’s not. You can be as computer savvy as you want, but you’re still relying on your eyes.

“You must work the lighting quite hard, because the eye’s perception is quite incredible – you can tell by shadows and things if it isn’t quite right. And while it’s easy to get it looking good, it’s that last 10 per cent of making it look really realistic that you have to really work at. I’ve had occasions where the computer is telling you that it should be accurate, and your eye is telling you it’s not. You can be as computer savvy as you want, but you’re still relying on your eyes.”

It’s taking time for AI to understand 3D – it’s hard for AI to work out depth and it gets stuff wrong, but as a tool it’ll get better and better.

The eyes still have it, too, when it comes to AI’s encroachment on cinematography and world-building. "It’s taking time for AI to understand 3D – it’s hard for AI to work out depth and it gets stuff wrong, but as a tool it’ll get better and better.” Danton’s main use for it is in storyboarding. “It’s a tool, and people are scared of it, but they shouldn’t be. It’s going to open things up to people who haven’t had the opportunity.”

Above: FilmLight created the lush colour in this Martini spot directed by Floria Sigismondi.

So in cinematography’s future, Unreal-driven volume stages will host a scanned-in world with plucky AI assistants popping up at every tech update, allowing the DPs of the world to do more in camera and expand out that camera to encompass post, thereby removing the borders from what were once very separate processes.

But let’s add a note of caution from the other side of the equation. FilmLight, now in its third decade, is a cinematographer’s go-to for colour grading and image processing. Co-founder Wolfgang Lempp has seen a lot in his career and is less enamoured than Danton when it comes to the synthetic, if convincing, reality provided by the likes of Arri and pursued by AI machine-learning systems.

 In terms of colour there are some very difficult issues [with VP], to do with how the camera sees things and how we see things, in terms of the different spectra of the LED.

“It’s happening, but it is very much an exception rather than a general trend,” he says. “VP stages are used primarily for commercials, which tend to try and push towards a heightened reality that is more difficult to shoot for real. But it’s still not really the big thing that people expected after Covid.