Any lingering resistance to virtual production involving next-generation elements like LED walls, in-camera visual effects (ICVFX) and mixed reality (MR) is rapidly disappearing across the media and entertainment industry as an increasingly viable hybrid approach to using the technology takes hold.
From broadcast programming to motion pictures and advertising, producers and directors are no longer faced with making an either-or choice between VP or traditional production modes. Instead, thanks to a spate of groundbreaking innovations that have made hybrid use-case flexibility a reality, virtual production (VP) in all of its permutations is now seen as an essential tool that can contribute time- and cost-saving efficiency to just about any project.
“Virtual production is an ever-evolving solution,”
Jaime Raffone, senior manager of cinematic production at Sony Electronics, says. “The move to immersive and hybrid technologies has been extremely helpful for us.”
Even before the hybrid VP paradigm took hold, researchers predicted big things for VP. In one outlook typical of many findings, Research and Markets projects the global VP market will grow at a 19.98% compound annual growth rate (CAGR) from $1.99 billion in 2022 to $7.13 billion by the end of 2029. According to Grand View Research, the VP spend in the U.S. over that span will increase at a 15.9% CAGR, reaching $1.09 billion in 2029.
Pros Are Embracing VP
By all accounts, resistance within professional production ranks to VP is rapidly giving way to enthusiastic embrace as benefits accrue from the more versatile hybrid approaches to exploiting the technology. Indeed, says Bob Caniglia, director of sales operations for the Americas at Blackmagic Design, the shift to VP is more creative-driven than a top-down, front-office phenomenon like the industry’s shift to video streaming.
“As effects guys and directors are pulled in, they start to realize they can really improve the entire workflow,” Caniglia says. While some directors are resistant to making the changes in production planning and execution required by reliance on LED volumes and ICFVX, many more want to benefit from things like providing “actors on set with live environments” and doing “more in shorter periods of time.”
The rise in VP adoption is having a big impact on Blackmagic’s output of cameras, switchers, editing tools and other products, as it meets demand for things like the locational flexibility enabled by cloud-based workflows and the LED wall-display quality stemming from the use of 12K and, soon with a forthcoming release, 16K cameras. The trendline in VP is “definitely moving upwards,” Caniglia says.
Sony’s Raffone, too, says the shift in perspectives on VP has been a big force behind new product development at her company. Notably, Sony responded to customer demand for LED solutions that aren’t fixed components of big studios by introducing the Verona line of modular LED displays in 2023. These employ the ground-breaking Crystal LED technology Sony created to achieve life-like realism through independent illumination of each pixel.
Over the past year, Verona displays have been deployed in multiple broadcast, cinema and VP educational projects, Raffone notes. New VP engagements in the broadcast industry include hybrid applications of Verona displays in live sports programming at WWE facilities in Stamford, Connecticut, and at the studios of another major sports broadcaster she declined to name.
Sony Bolsters Hybrid Use
Sony has also strengthened its position in the production space with a Virtual Production toolset that supports ICVFX (in-camera visual effects) within the Epic Games Unreal Engine multidisplay rendering platform to meld the real and virtual domains and cut production time through virtualized renderings of camera settings ahead of actual production. Integration partnerships with Brompton Technology and Megapixel VR ensure out-of-the box compatibility between Verona LED displays and leading production LED controllers, Raffone also notes.
The upshot is a hybrid-optimized VP portfolio that’s gaining traction worldwide across the company’s traditional market base as well as in new areas of business, education and government. “We continue to evolve with the needs of the market,” she says.
Ross Video is another major player benefitting from the hybrid VP wave. “Control and integration across all elements is probably our forte and strong suit,” Mike Paquin, senior product manager for virtual solutions at Ross Video, says.
Ross, which in December announced its intention to sell the G3 LED wall-supply business it acquired in 2021, is seeing widespread use of LED walls in VP, but primarily in hybrid situations where broadcast productions can take place on stages without a VP component or be augmented with LED and/or MR elements to whatever degree is warranted in a given situation.
A big contributor to this versatility is Ross’s support for rendering graphic elements in whatever dimensions fit a given scenario, Paquin says. The company’s XPression Tessera platform employs a distributed workflow system that allows users to configure graphic elements for pixel-accurate rendering across multiple display environments with a dynamism that enables real-time changes in content in tandem with live events.
Such flexibility has created a multiuse studio environment that’s saving broadcasters a ton of money. “Building three or four studios for a TV station doesn’t work anymore,” he says, noting that many Ross customers, including smaller stations that “see what their network parents are doing,” are in the process of refreshing their studio environments. “Virtual is a part of almost every one of them,” he says.
Toward that end, Ross at NAB Show in April will demonstrate support for a flexible workspace where a hard set and broadcast desk with display wall can become a weather reporting studio with the kind of green-wall functionality weathercasters are accustomed to, and then be turned instantly into a news or sports reporting venue with LED wall support for rendering the setting and attendant graphics. “You can rotate people through that space for different segments of the same live show,” Paquin says.
ICVFX Without the Walls
ENCO Systems, a pioneer in automated broadcast production workflows, has taken its own approach to VP by leveraging decades-old chroma key video monitoring technology with the compositing tool supplied by Miami-based Qimera. ENCO, the exclusive North American sales outlet for Qimera, markets the high-powered computing appliance Qimera Chroma as a way to weave VP capabilities typically supported by LED volumes into existing live broadcast production workflows without actually using LED walls to provide a visual context for onscreen personalities to work in.
Relying on Unreal Engine to pull together all the elements of virtual sets and augmented reality (AR) graphics created by the Qimera compositing tool, ENCO allows newscasters to view on their monitors what’s happening with their placement in the virtualized output their audiences are seeing on their home screens. “Just like VP with LED volumes, using chroma key with Qimera, you can change the sets and images with the click of a mouse,” says Bill Bennet, media solutions and account manager for ENCO.
This requires processing delivered by Nvidia’s GeForce RTX GPU and a 13th-generation Intel core along with other high-end hardware components in the Qimera appliance. Bennet says that’s a lot cheaper than investing in LED walls and all the processing they require. “The costs of LED volumes often start at $100,000 and go up from there,” he notes. “We built Qimera Chroma to avoid that. I love LED volumes, but we don’t always need them.”
Hybrid VP on Wheels
One of the more dramatic manifestations of the new hybrid flexibility in VP was recently on display in the middle of nowhere at Disney’s Golden Oak Ranch filming lot, a sprawling property north of Los Angeles. That’s where Brian Nowac, founder and CEO of Matchbox LLC, was on hand for production of a series of Bush’s Baked Beans commercials. While the setting was appropriate for shooting the outdoor scenes, the rest of the filming was taking place in a tractor trailer outfitted by Matchbox as a complete self-contained LED studio.
Speaking with TV Tech, Nowac describes the rapidly expanding role the uniquely equipped Matchbox trailers have played up and down the West Coast and will soon be playing elsewhere in hybrid M&E productions. “We’re building a fleet of super-studio products we’ll be deploying all over the country,” he says.
The Magicbox trailers, measuring 52 feet long, 32 feet wide and 13½ feet high, can be used with an 8½-foot high LED volume consuming 600 square feet of horizontal space or with individual LED walls. A motorized turntable floor makes it possible to film multidirectional car scenes that eliminate the need to shoot from a moving vehicle.
A recent Dodge Hornet commercial with a child version of an adult passenger at the wheel was shot entirely inside the Magicbox, Nowac notes, which would have been impossible with a child driving on city streets. The Bush’s Beans commercials will feature in-car scenes involving a driver talking with her dog, which resembles another in-car shoot involving a woman and her dog in a Subaru commercial that goes beyond what’s doable in the real world.
Along with the convenience of bringing LED studios to virtually any location, Nowac notes that Matchbox is making it possible for producers to save a lot of money by having another production space at hand when set-changing and other main studio downtimes would leave high-paid actors and staff standing idly by.
“Actors can walk right outside a sound stage into the Matchbox studio and do alternate takes or more of their key lines using the exact same lighting and other aspects as they’re using in the big studio,” Nowac says.
