LOS ANGELES—The idea of “a new look” for football offense and defense is nothing new, and while the same could be said of television production of NFL games and their pre-game, post and half-time shows, what Fox Sports has planned for this NFL season is unlike anything the broadcaster has ever done for football coverage.
Relying on a massive LED volume, a combined three dozen Unreal and Viz Engines, multiple new cameras, precise camera tracking and state-of-the-art technology that seamlessly stitches video together in real-time for the volume, the broadcaster will debut a combination of Extended Reality (XR) and Augmented Reality (AR) Sept. 11 during “Fox NFL Sunday.”
The studio will enable Fox Sports to transport viewers one moment onto the middle of Lambeau Field during a Packers update and the next take them outside Sofi Stadium—all while giving hosts a better way to tell the stories of games.
The new Los Angeles studio, while the broadcaster’s first AR-XR deployment for NFL coverage, represents the next step in its adoption of the technology, following the upgrade of its Charlotte, N.C., studio, to AR and VR technology for NASCAR coverage nearly four years ago.
“AR and XR technology enhances the storytelling ability of our hosts,” says Zac Fields, senior vice president of graphic technology and integration at Fox Sports.
For a segment called “On the Field,” for example, the technology will help hosts better break down plays. “In the past, you were somewhat limited. You have a handful of guys on the field trying to tell a story,” says Fields.
“If you can add items and manipulate things within AR, you’re not quite as limited. Now you can add in players, objects and routes that someone runs and the viewer can see that virtually in the XR space. That’s exciting,” he says.
By The Numbers
Work began on the new 5,000-square foot studio in Los Angeles at the end of last season. The centerpiece of the two-story structure is the LED volume, which measures 19 feet, 8 inches (H) by 46 feet (W) by 24 feet, 7 inches (D), and takes up about one-third of the area of the set. The LED volume, which was built and commissioned by NEP Studios, consists of three LED walls and an LED floor—a combined 5,130 square feet of LED display—that enables Fox Sports to change environments quickly as different show segments air.
“[One of the] biggest design goals was to give unique segments their own identity. There are certain branded segments throughout our show where the hosts do different things,” says Fields. “The goal was to provide them with their own environments, their own unique functionality to tell their stories in a different way.”
Fox Sports developed much of the creative for the LED volume and also worked in concert with “out-of-house design studios” on some of the creative integrated into the studio. “They would help facilitate building an environment,” says Fields.
“Sometimes they would build functionality; other times we would build that functionality—things like generating graphics based on live data or any other sort of dynamic information that we then integrate into our studio,” he says.
Rendering relies on a combination of technologies. Eleven Viz Engines power the standard LED screens around the set while 25 Unreal Engines drive the XR rendering required for the LED volume as well as any AR rendering that’s needed, says Fields, adding that all engines used for the volume are 4K.
“One of the key technologies we are using is called GhostFrame [from Megapixel]. It allows multiple cameras to see their own image on the LED wall,” says Field. For this LED volume, four cameras can have their own view of and projection onto the LED volume at the same time.
To someone standing in the LED volume, the displays appear to have the pickup of the different cameras overlaid on each other. However, the technology makes it possible for each camera to see only its own image, thereby giving the director an accurate preview of LED volume shots from the four cameras before deciding which one to take to air, he says.
“GhostFrame is sort of like the secret sauce to making this work in a live broadcast environment,” says Fields.
To support the use of AR throughout the set, seven cameras are tracked and support object tracking that allows for elements in the shot to be interactive. “We deployed stYpe tracking for the cameras as well as object tracking,” says Fields. “That allows us to not only place objects in AR, but also to interact with objects in 3D space.”
Wireless tablets will be used to give talent control over AR elements and telestration. In total, the setup supports 41 real-time graphics output channels.
One fear many broadcasters have expressed about adopting any sort of virtual technology for sets is what happens in the event of an engine failure. Fox Sports, too, had those concerns but has learned to live with the possibility based on its experience in Charlotte.
“Our Charlotte studio was our first virtual set, and we’ve been doing it quite a while now, almost seven days a week, most weeks out of the year,” says Fields.
Originally, the broadcaster focused on having complete redundancy for every rendering engine as well as an automatic failover. “What we realized was in the time that it would take to get that [doing the failover], you’re better off switching to another camera and using the other cameras until the problem is corrected,” he says.
In the event of a catastrophic failure, the “beautiful thing” about the Los Angeles deployment is there is also a practical set so the show can go on uninterrupted, he says.
Beyond the AR and XR, the new studio is equally impressive and massive. The structure includes 63,000 pounds of steel, aluminum, rigging hardware and mechanical components, and required over 53,000 man hours to build.
Only The Beginning
In Fields’ view, the new Fox Sports studio is only the latest advancement in a long line of virtual technology—one that’s poised to transform television going forward.
“I think this is a technology that you’re going to hear a lot about in broadcast’s future. There’s been an iteration of virtual sets. It started with virtual sets used for weather segments and grew into a whole other thing as the technology progressed,” he says.
As photorealism entered the picture so did a new generation of LED display technology that enabled the next advancement.
“We’re now able to not only project incredible imagery, but the LED also provides this lighting on your subjects that creates a believability that hasn’t been able to be achieved before. That is why when this is seen on the air, I think people will really be impressed with the visual presentation,” he says.
