NEW YORK—In its 2013 NFL season inaugural telecast of the Cowboys/Giants game on NBC Sunday Night Football, Sept. 8, NBC Sports debuted 360-degree instant replays, using freeD technology developed by Replay-Technologies. The multicamera system—two camera setups, each consisting of 12 cameras at opposite ends of the field—installed at AT&T Stadium in Dallas allowed color commentator Cris Collinsworth to show replays within the 20-yard lines from every possible camera angle. Collinsworth was able to rotate the camera’s view around the action, as was done in “The Matrix.”
The result: Viewers will have a true 3D understanding of what happened on the field, because the replays will be viewable from an unlimited number of perspectives. “We wanted something that would take NBC Sunday Night Football’s coverage to the next level,” said NBC Sunday Night Coordinating Producer Fred Gaudelli.
NOT AN NHK CLONE
freeD’s system works by taking feeds from 24 cameras stationed throughout AT&T Stadium to create a 360-degree image. This is not the first time that freeD technology has been used to provide sports replays. Earlier instances include the 2012 London Olympics Games, and on MLB New York Yankees’ games ("From Bird's-Eye to Panorama with YES View"). “In those cases, our equipment only had to cover athletes or players working in a small space, without much movement,” said Oren Haimovitch-Yogev, Replay-Technologies’s CEO and co-founder. “NFL football is a game-changer: We have to cover the entire space between the end zone and the 20 yard line—with lots of player motion— in both ends of AT&T Stadium.”
NBC Sport’s NFL freeD plans have garnered a lot of mainstream media attention, with the freeD system being explained as a bigger version of NHK’s “bullet cam” system. In the NHK system, a single operator controls a team of connected cameras at different positions, all tracking on the same object in real-time.
Replay-Technologies’ freeD is different.
“We fix-mount 12 Teledyne Dalsa 12 megapixel cameras, which shoot in 4K,” said Haimovitch-Yogev. “These cameras shoot at 60 frames a second. They are typically used to provide precise product inspect in factories. There is no camera panning or zooming. The cameras and their focal lengths remain constant.”
The freeD system combines the camera feeds with high-level processing software. The latter was derived from reconnaissance software developed for the Israeli military, plus multidimensional computer vision software used in artificial intelligence applications.
“The system works by taking the 12 camera feeds—like images seen by 12 eyes—and sending them to our computer which acts like a brain in processing them,” Haimovitch-Yogev said. “Using our software, the computer is able to use the various camera angles and build an exact volumetric images for the video; one that can be looked at from virtually any angle.”
The fact that freeD uses 4K video cameras allows the system’s operators to zoom into specific items digitally, without compromising HDTV playback quality. “We can not only rotate to a specific angle, but go into for a closer look, and the quality is not degraded,” said Haimovitch-Yogev. “That’s what is possible when you start with video that has four times more resolution than required for HD broadcast.”
SYSTEM LIMITS
freeD uses 12 Teledyne Dalsa 12 megapixel cameras, which shoot in 4K. The freeD replay system is not without its limits. Specifically, it can take up to a minute for Replay-Technologies’ on-site techs to render a freeD replay, before being able to hand it off to NBC Sports for use by Collinsworth on air.
“The unavoidable time required for processing means that we can’t use the freeD replay immediately after the play,” Gaudelli said. “So we expect to be able to bring it in at the next pause in play. Still, the overall effect will be stunning to our viewers, and something that really makes NBC Sunday Night Football stand out.”
Meanwhile, the fact that the freeD system requires 24 fixed-mounted cameras, and their accompanying cabling/integration, means that the system will only be usable during games at AT&T Stadium this season. “This is not something that you can get into place quickly, like 4K cameras or even a CableCam,” Gaudelli said. “Unless or until other stadiums install it, we’ll only be able to offer freeD replays in Dallas.”
Practically speaking, there is no reason that freeD replays technology can’t be installed in all football stadiums, plus those that cover hockey, baseball, or indeed any sport whatsoever. Eventually as Replay- Technologies refines freeD and develops new generations that are easier to deploy and configure, the system may turn up in non-sports venues, and provide quicker replays and live streaming video as advances in computer processing speeds up rendering time.
“Is freeD the next big thing in TV? We certainly hope so,” said Haimovitch-Yogev. “But I think the prediction goes beyond hope: Unlike 3DTV, freeD provides real value to existing HDTV viewers, while giving broadcasters a solid reason to start using 4K in production.”
