Production’s virtual future
As Star Wars series The Mandalorian paves the way for new technology to shape how TV series are made, DQ looks at the impact virtual production is set to have in the next few years.
For the past year, television producers have been forced to think on their feet as they try to find a way to continue filming amid global lockdowns, travel bans and additional health and safety requirements. But while the long-term effects of the coronavirus pandemic on the industry are still to be determined, new technology has allowed series to continue to be developed and made, ensuring viewers’ insatiable appetite for new programmes continues to be fed.
Online writers rooms and script meetings are now commonplace, while those in post-production were among the first to conquer remote working, completing editing and sound mixing via video conferencing and recording additional dialogue from the comfort of actors’ homes.
The hope is that, in time, these practices will once again become in-person activities – writers all sitting around a large table together breaking down plot lines; producers and editors huddled together in an editing suite.
However, one recent development looks here to stay. While virtual production with green-screen and motion-capture technology has been used in various forms for a number of years, new LED-screen techniques honed on Star Wars series The Mandalorian are set to change the way films and series are shot over the next few years.
In the 2021 Nostradamus Report, published during the Göteborg Film Festival last week, virtual production was among the trends predicted to become widespread over the next five years, by which time it will be “completely normalised” across the industry and offer opportunities for greater artistic collaboration across different departments.
Films like Avatar and The Polar Express were among the first to harness the potential of virtual production, but a leap in computer power allowing 3D images to be rendered in real time, a process that might previously have taken hours, is the real game-changer. The technology that first allowed players to move freely through computer games is now being applied to film and television.
One example of virtual production is where all the visuals are computer generated. Then there are productions like The Mandalorian that use a soundstage surround by LED walls, onto which the virtual environment is projected, allowing actors to interact with the digital world as it is captured in-camera. Then there are mixed setups, such as that used for Avatar, where a physical shoot is set against a green screen and the environment is added afterwards.
“Achieving the level of realism of something like The Mandalorian on set, in camera, is very complicated,” said Alex Stolz, the founder of Future of Film, who is quoted in the report. “The mise en scène is incredibly important to achieve the illusion that you are not looking at an LED screen, but into a 3D landscape. The set dressing in front of the screen, and the way the performers are positioned, is critical. You need that depth of field to achieve that parallax whereby the viewer is tricked.
“Five years from now, many high-end series and films will be using at least some element of virtual in-camera visual effects. It’s a question of cost and growing the ecosystem – building the installations, the infrastructure and the expertise. But there’s lots of activity in that space, which hopefully will open it up.”
As a result of the pandemic, virtual production is already being explored on a wider scale, with the possibility that digital environments might one day complement or replace shooting on physical sets or locations.
“These techniques are being used in animation today, and using that in animation means you spend a 10th of what you’re used to spending,” said Kjartan Thor Thordarson, CEO of Icelandic producer Sagafilm, during a session on virtual production during Göteborg’s TV Drama Vision event. “The technology was first adapted in our industry where you needed to pre-visualise what you were doing. Now it has developed into a full production capability technology, and the engines that were originally created for gaming are now being used for filming, like Unreal and others being adapted. We can thank Disney and The Mandalorian for a lot of new tools being added into these engines to allow us filmmakers to use them.”
Thordarson identified the role the pandemic has played in accelerating development in this area, with crews have forced to work remotely and being unable to visit international locations.
“This also solves a lot of problems that you can solve in a studio now, rather than going on location,” he said. “A lot of filmmakers come to Iceland to shoot glaciers because they’re accessible. Now we can also shoot the glacier scenes in a studio rather than on location, which comes with much more expense.”
One series being developed with a view to virtual production is Whiskey on the Rocks. Produced by Stockholm-based Dramacorp, the Cold War miniseries is based on a notorious incident that almost led to war between Sweden and the USSR, when a Soviet submarine ran aground in Swedish waters during a Swedish naval exercise in 1981. The stand-off between the two states cooled only after 11 days of negotiations.
A Scandinavian and international cast is being lined up for the series, which will be set in Moscow, Stockholm and NATO’s headquarters in Brussels and is being pitched as one of the first European dramas to use green-screen and in-camera VR technology.
“It’s these three locations we want to recreate with this virtual way of producing,” said executive producer Henrik Jansson-Schweizer. “That’s why we’re looking at this, and for filming inside the submarine as well. We think this project is a really good fit for producing it this way.”
“We’re looking at different ways,” said Dramacorp founder Patrick Nebout. “There is no integrated solution in Europe. What we have discovered by doing exploration and research is that France is one of the countries that has come the furthest with this kind of technology. There are two companies in Paris using Mandalorian virtual production, mostly for commercials. It all comes down to where it is most efficient to shoot. From a Nordic and Swedish perspective, unfortunately Sweden still doesn’t have any tax incentive and that’s why a lot of Swedish productions are being shot elsewhere, as in Iceland and Eastern Europe.”
One challenge facing virtual production in television is the lack of studios equipped with LED screens, while the shortage of expertise is also an issue. “We are sourcing that in gaming companies but that’s a very competitive industry,” Thordarson said. “It’s very hard to get these people to move to work for the film industry. Gaming is very highly paid, so we need to invest in growing the talent base of people working in this medium.
“DOPs, writers and directors are adapting to this technology quite fast because we’re not changing the way we make films. What we need is the talent that creates the digital assets. That’s where I have found the biggest difficulty, in finding the talent who have done this before.”
Working in virtual production also needs a different mindset from the traditional television development process, with work usually completed in post-production required to be done before production begins.
“Before it was always like, ‘Fix it in post,’ but now you have to fix it in pre-production,” said Jansson-Schweizer. “You have to switch everything and do it a different way. You have to find new ways to develop projects with this technique that don’t look like the way you did it before.”
But this emerging technology is not the holy grail, Nebout warned. “It doesn’t apply to all projects. With Whiskey on the Rocks, we have a project that is very suited to that because our idea is to shoot the whole production in one studio, so at the click of a button you get a new set: you’re in the White House, in the Kremlin, the Swedish State Department and Brussels, and you can also be in exteriors outside the submarine.
“That suits this kind of production, but it doesn’t suit huge battlegrounds and stuff like that. That’s something the US or huge productions can afford. We have to find our way in Europe. The different solutions being developed now in Iceland, France and elsewhere are really exciting and interesting.”
Thordarson believes virtual production can take television “to a different level,” opening up the small screen to limitless possibilities for storytelling.
“It’s hard to compare anything to The Mandalorian. It’s such a big, crazy production but it shows what kind of things you can do with this technology,” he said. “We in Europe are definitely adapting in a different way, a smaller way, but it allows us to create new ideas we couldn’t do before because building huge sets won’t be a problem anymore. We can do it in the virtual world.”
tagged in: Alex Stolz, Disney, Henrik Jansson-Schweizer, Kjartan Thor Thordarson, The Mandalorian, Whiskey on the Rocks