Virtual production is currently a buzzword that most, if not all, are aware of. Movies like Dune, Midnight Sky & The Mandalorian, have all featured the technique with great success. Anyone who has recently been to Cannes Lions, would have seen: Epic, Unity, and Pixmondo, all touting their wares.
Now, advertising agencies like Hogarth and WPP are heavily committed to VP, largely because of the green and sustainability credentials. With brands like Unilever and McDonalds already shooting VP. I think it’s worth taking a look at exactly what this new technology means for the Food commercial and Tabletop industry.
I have been lucky enough to be involved in setting up 3 permanent LED stages in Europe and Africa. Have shot over 60 days in ‘The Volume’- the name given to LED studios that are popping up around the world. With that, I have a reasonable insight into what they can achieve and equally essential, what they cannot achieve.
Firstly, virtual production is somewhat of a misnomer. To all intents and purposes, as commercial makers, we are involved in mixed reality production, rather than virtual production. We are generally taking a conventional foreground scene with talents and props and integrating it with a 3D or 2 Plate background. Combining the skills of traditional film makers, with those of video game designers; display and event technology, to provide real time VFX.
The key to the success of Mixed reality production is integration. Integrating: your favourite cameras with your favourite lenses and lights; lens profiling, FIZ data and pixel mapping of lighting features; into Unreal Engine. I have been working alongside ARRI who are at the cutting edge of this integration of VP and cinematography. By adopting this integrated approach we end up using the tools we are comfortable with; albeit in an unfamiliar environment. Working in f-stops, not percentages; with foot-candles and not NITs. Being able to replicate your favourite lighting fixtures and filtration within a 3D environment. Accurate tracking data and some fairly hefty processing power- help, too!
image courtesy of the Author
So let’s just take a look at a few of the advantages of VP in an advertising environment. One thing clients love is that they own the location. Once a virtual location has been built in 3D: whether it’s a supermarket, a restaurant, an industrial kitchen, or a family home; it is owned by the client or on occasions, the agency. They have that asset available to them in perpetuity – no more location fees, no more long traveling, no more issues with IP. It is theirs to shoot in- wherever a LED studio exists. This is particularly useful when a campaign returns to the same location many times throughout the year. When you’ve got a real location that is only available one day a week, or only available to shoot out of store hours: this can be lidar scanned, photogrammetry captured and built in CG. You can shoot in it day after day, from then on. Famous Restaurants, Markets, Airports; all those nightmare locations become tamed.
Secondly, how about a magic hour picnic or BBQ that lasts 12 hours? You position your sun: its height, its intensity and its colour; and it sits there in perfect position for take after take. How many times have we been chasing the light or having to compromise our shots as the sun disappears over the horizon – this really is game changing!
Directors and cinematographers love this technology because of the increased creativity that it brings, and producers love it because of the control it gives them over the shooting day. After all, time is money.
When I was first introduced to this subject, I made the mistake of thinking that these LED walls were just a glorified green screen; sure, they can do everything a green screen can do – and better, but it offers far more than that.
Photo: Will Case | ARRI
It offers immersive volumetric lighting – in simple terms , where your LED source provides the lighting for your artists and foreground. Now we have tracked 3D backgrounds that respond in real time as you dolly the camera or crane, and a screen that responds to a change in focus or T stop.
Talents, love this environment over green screen, because they are interacting with ‘real’ location in realtime; not having to perform to a white cross on a sea of green. Their location comes to them.
Imagine that shot of a lumberjack enjoying his favourite soda at the top of a tree in the Tatra Mountains; all shot just a metre or two off the ground, in the safety of a weather controllable studio. No permits, no expensive rigging, no stunt team, no weather cover – yet a fully interactive 3D studio – the opportunities are endless.
So, there is always a catch. This is not cheap technology! – A virtual reality studio, demands a specialist crew. The hardware is expensive, and requires constant updating, and to install film friendly high quality LED walls with small pixel pitch; you are looking at seven figure sums.
Many smaller studios have realised that it is not a simple case of buying LED walls and pointing a camera at it. It requires a team of dedicated professionals to run it – VP supervisors, Virtual gaffers, IT network technicians, Unreal artists, Board operators and LED technicians. No sooner do these small studios open; only to close again.
That said, I do know of 2 food photographers who have bought 70” high spec OLED TV’s; and regularly use them to provide animated backgrounds: for cinemagraphs and video clips for Tiktok; and Youtube – that looks great and it’s at a very low cost.
Photo: Will Case | ARRI
Putting together a mixed reality production, requires productions, to rethink their budgets and reallocate resources. If you aim to achieve final pixel in the volume, then traditional pre -production, production and post production lines blur. If you don’t have to build a physical set for example, then you have budget released from Art Dept. to create an environment. I intend to chat about this in a guide, on Directory to Virtual Reality, for agency producers and clients.
However, back to the needs of Directors and Cinematographers. So the elephant in the room for those of us who love to shoot high speed- the high frame rates required for many shoots, are simply not achievable using existing LED screen and processing technology. As a general rule we are limited to 60fps. Currently at 25fps we’re looking at a 40 m/s refresh rate, going to only 100fps requires a 10 m/s refresh rate. Which puts huge pressure on processing, on bandwidth and requires a huge amount of high end hardware. I have been experimenting with the use of high end tournament gaming hardware, which in theory would allow 144fps. It is fair to say that the evolution of this tech is very fast, and the needs of the high speed community like tabletop are being considered by camera manufacturers and LED manufacturers alike.
Personally, I will be happy when I can run an Alexa LF or the New Alexa35 at 120fps on LED volume. This will cover most of my needs as a food filmmaker.
Another potential issue is the relatively long lead time it takes to create these assets. With commercials getting green lit later and later, it will need some re education of agency; that we need longer lead times to prepare the assets for the screen. For a simple plate background this may be just a couple of days, but for fully integrated 3D realtime environments the lead time could run into months. As more and more off the shelf assets become available and Unreal Engine Artists transition from gaming to film production; then these lead times will drop significantly.
Photo: Will Case | ARRI
I recently had a client who was scripting a spices campaign; it required filming in market stalls across 5 Far East and South Asian countries; fast cutting between them all. We were able to propose that a small plate unit be sent off to shoot the backgrounds in Asia; and the whole production be shot on the LED volume. This reduced costs drastically and allowed the director to really play with continuity and performance in one space; with interchangeable cast. It was a commercial that probably would never have been made on a conventional budget and timeframe.
Ultimately, this technology is another tool in our arsenal. It is not the golden bullet, but it is a technology that is being explored closely by food clients and agencies; and we need to be ready.
This is too big a subject to cover in one article, I am hoping to follow with a producers and agency guide to working on a Virtual Reality Production. Then a more technical article for DoP and directors: covering challenges of Moire CRI, aliasing, and sync skew – a few practical tips learned the hard way. Hopefully this has whetted your appetite.
We all continue on our journey to make the Unreal… Real; and provide mouth watering and eye catching imagery for our clients: on location, in studio, or on a VR volume.
