HAMILTON, N.J.—No longer cartoonish or fake looking, virtual sets now display remarkably realistic HD and 4K environments that surround on-air presenters or give the illusion of an immense scene extending beyond the studio walls.
Recently, a new live virtual graphics technology has taken center stage—augmented reality (AR)—where 3D objects appear to leap out of a video wall, rise up from the floor, or animate into the scene in visually appealing ways.
Virtual sets and augmented reality are becoming increasingly popular—both among networks and local stations—because they offer creative new options for spicing up the look of live shows quickly and cost-effectively.
AUGMENTED VIRTUAL SETS
“By having virtual elements magically appear in the middle of the set, broadcasters can present very trendy, eye-catching studio looks and tell better stories to viewers,” said Ofir Benovici, senior director of broadcast products for Avid. “With sufficient processing power, augmented reality can now generate realistic virtual objects, such as virtual video walls that rival their physical counterparts. And, while physical video walls and touchscreens are expensive and complex to maintain, virtual video walls are easier to implement, with lower costs over time.”
Avid’s ProSet virtual set system can be implemented as a full turnkey solution or flexibly configured with third-party components.
One factor driving demand, according to Benovici, is that processing engines are more capable of highly complex, yet credible, live virtual sets and effects. Another factor Benovici cites is, “precise camera tracking, which is also essential to having virtual elements correspond properly with physical elements in a holistic way for a unified visual effect.”
Avid’s ProSet virtual set system can be implemented as a full turnkey solution or flexibly configured with third-party components, such as camera tracking technology. Benovici added that virtual sets could maximize physical studios by allowing multiple shows to be produced in the same space quickly and cost-effectively, each with their own distinctive, aesthetic broadcast look.
VIRTUAL SET WORKFLOWS
Broadcasters are blending physical and virtual elements for a more dynamic look. For example, instead of having large monitors and video walls on a physical set, augmented reality displays, like virtual monitors or data-driven graphics, can rise up from the floor and be waved away when no longer needed.
“When on-air presenters are talking about something displayed on a greenscreen backdrop, they tend to have their backs to the audience, which is not visually engaging,” said Dave Larson, general manager for virtual solutions at Ross Video. “But with augmented reality, a meteorologist can tell viewers about an impending tornado while a virtual tornado is spinning right in front of him for a real wow factor.”
Ross offers the entry-level Ross Trackless Studio or complete turnkey system. The Ross virtual studio workflow typically includes a Ross Xpression real-time motion graphics system, Furio and CamBot robotic systems, Solid Track for camera tracking, UX for virtual systems operation, OverDrive automated production control, Carbonite production switchers, Ultrachrome HR keyers, and the new Acid camera, which was designed specifically for greenscreens.
According to Larson, augmented reality implementations are accelerating since the technology can be adopted quickly and cost efficiently, without making changes to an existing studio. “While virtual sets require upfront hardware costs,” Larson said, “in most cases they could be designed and built for one-tenth the cost of physical sets, which can often off-set the incremental hardware costs.” He adds that by swapping out virtual sets, back-to-back shows could be produced from the same studio, which reduces production costs.
IP-DRIVEN WORKFLOW
With the introduction of its TriCaster Advanced Edition software last year, NewTek added the ability to integrate 3D augmented reality graphics, objects and text within the virtual studio. With sophisticated control of layered keying, these virtual objects can appear to be floating in the midst of practical fixtures in the physical studio.
“What we’re witnessing is an evolution, possibly a revolution, in the way that broadcasters blend virtual, graphical and physical elements together to produce fresh, novel, imaginative studio looks,” said Will Waters, director of product marketing for NewTek.
“Using dynamic, graphically intense virtual sets will help broadcasters compete for viewers, especially those in the coveted younger demo who tend to be visually and graphically oriented,” Waters said. While the NewTek TriCaster has its own integrated virtual studio, and library of hundreds of 3D virtual set options, it also interfaces with major third-party virtual set systems by Brainstorm, Vizrt and others.
With the new Network Device Interface (NDI)—an IP-based workflow that NewTek developed and provides to other manufacturers for free—NewTek is expanding live video productions with TriCaster and these benefits impact both virtual studios and augmented reality.
With NDI, TriCasters can send hi-res video over a standard Gigabit Ethernet IP network to virtual studio systems like Brainstorm and Vizrt for processing and the resulting virtual/augment reality effect becomes a TriCaster source, with no additional latency. “We’re continually expanding our live branding and studio toolkit to empower our customers to be as creative as their budgets and ambitions allow,” said Waters.
RAZZLE DAZZLE
David Jorba, president of Vizrt Americas, cites many factors that are driving the growing augmented reality trend, including advances in technology, such as video boards and graphics, and GPUs capable of HD and 4K, Specific to the Vizrt Viz Virtual Studio, he cites Vizrt’s new Viz Tracking Hub for easier set-up, and Viz Engine for faster processing and rendering of HD and 4K virtual sets.
At the NAB Show, Vizrt demonstrated augmented reality in conjunction with its Viz Virtual Studio on a 7.2-meter video wall.
“Broadcasters can now create engaging virtual studios, graphical looks and effects easier and at a price-point that wasn’t possible just a few years ago,” he said. “With augmented reality, on-air presenters can appear to toss 3D objects off the video wall behind them—such as popular sports figures or NASCAR cars—and create a seamless transition to bring the objects in front of the anchor desk for a more exciting live effect. These 3D objects can also be driven by real-time data, such as sports stats or elections results so the anchor can analyze the news while directing the augmented reality object.”
At the NAB Show, Vizrt demonstrated augmented reality in conjunction with its Viz Virtual Studio on a 7.2-meter video wall. Camera tracking systems, including Motion Analysis IR and Ncam handheld tracking, tracked and measured the physical cameras’ movement in relation to the staging area so that the virtual studio’s camera perspective would always change accordingly for a credible effect.
The demo showed how greater photorealism can be achieved by using software plug-ins, such as RTT Shaders, Allegorithmic Substance, and Enlighten Global Illumination, which add 3D lighting, textures, shading, reflections, and other finishing touches that change in relation to camera movement in real-time. The virtual sets for Viz Virtual Studio can be created using any major 3D software package, such as Cinema 4D and Softimage.
CAMERA SUPPORT IS KEY
In the early days of virtual sets, the physical camera used to have to remain in a fixed position and camera moves were limited to pan, tilt and zooms due to the complexity of lens calibration and tracking. But, “people eventually wanted to move the camera around the physical studio on a pedestal while tracking its position,” said James Eddershaw, sales director for Shotoku UK. “We’ve been doing this for years with our VR pedestals, which use high-performance encoders and processors to measure the camera’s exact position.” Shotoku’s VR pedestals’ tracking systems can output data to any leading virtual studio system.
“People today want to use dramatic camera moves, like the big sweeping motion of a jib or crane, with their virtual studios,” Eddershaw added. Shotoku’s VR robotic camera systems, specialized cranes and jibs can track these dynamic camera moves.
“They even want to move the physical camera freely around the studio—using Steadicam, shoulder-mount or handheld shooting styles—all with the same highly accurate camera tracking for a more immersive, interactive experience,” Eddershaw said.
When placed on the physical camera, Shotoku’s Free-D2 absolute tracking device frees cameras to move anywhere up to 360 degrees within the physical studio. It looks upwards at the lighting grid, and locates distinguishing markers that it can use to continually pinpoint the camera’s position, and outputs the data to any leading virtual studio system.
PRICE-CONSCIOUS MARKETS
At Broadcast Pix, Product Manager Tony Mastantuono finds that their customers in the small to mid-tier markets are moving away from full virtual sets, especially if they have tight budgets.
“The higher-end markets have the budgets to make their virtual sets look really good, with high-end tracking of multiple cameras for greater realism,” Mastantuono said. “But many price-conscious customers are moving away from virtual sets because, at their budget levels, the sets tend to look fake and they run into technical roadblocks. They’re opting instead for physical sets with virtual imagery keyed into the green or blue background.”
For those that need affordable solutions, Broadcast Pix Flint and Mica Series integrated production switchers offer virtual studios and backgrounds for up to eight cameras. They also offer a trackless virtual camera that can move within the virtual set, while the real camera remains stationary.
Mastantuono explained that the use of virtual sets largely depends on the customer’s unique needs and budget. While education and webcasting are two budget-conscious markets that are inclined to use virtual sets, the house of worship and corporate sectors generally do not. And mid-size stations may use virtual sets, but moreso in Studio B, to make the limited space appear larger.
