PYEONGCHANG, SOUTH KOREA—Beyond athletic achievements, the XXIII Winter Games will hopefully be remembered for having North and South Korea delegations marching into the Olympic Stadium under a unified flag, and the fielding of a mixed North and South Korean Women’s ice hockey team.
The XXIII Winter Olympic Games will take place in PyeongChang, South Korea, Feb. 9-25
For NBC’s veteran production team, the run-up to the Games has been a busy, but manageable period where everything that had to get done was getting done, on and sometimes even ahead of schedule.
And that’s important given the task at hand.
NBC Broadcast Network was poised to begin its 176 hours of coverage on the night before the Opening Ceremony. That, of course, is only part of the story in today’s media environment. NBCUniversal will present more than 2,400 hours of coverage by the end of the Games across NBC, NBCSN, CNBC, USA Network, NBCOlympics.com, and the NBC Sports app. That’s the most hours ever for a Winter Olympics.
Notably, NBC will present Olympic primetime coverage live across all time zones—also a Winter Olympics first.
David Mazza, CTO & senior vice president Sports and Olympics, has worked 15 Olympic Games starting with Los Angeles in 1984 and was quick to praise the South Koreans in their preparation for the Games. “The country itself just works, and it works well,” he said. “The construction, the power, the technology, the people’s understanding of technology… all of this has made our setup easier than it has been for many years.”
WELL PREPARED
One particular challenge to this Games has been the weather. It’s not storms or tides that wash out a studio as in Rio, or deep snow—it’s just a bitter cold that does not stop. “I’ve worked in some cold climates, but none as consistently cold as here,” Mazza said. “It can be –30 degrees with wind chill in the mornings!”
Another factor that has made a difference is the use by host broadcaster Olympic Broadcasting Services of a pre-fabricated system for the International Broadcast Centre, home to broadcasters covering the Games. This was well-received in Rio and here as well.
This meant that as soon as the team arrived they could immediately start all of the work and all of the cabling, which is not typical, noted Terry Adams, vice president, IBC Engineering, NBC Olympics, working his 11th Games. “[In previous Games], we typically had to spend a month or more dealing with cable trays and powering and construction,” he said. “But this time when we arrived in Korea on Oct. 20, they could just get right to work.”
Mazza said the crew was actually 2-2½ weeks ahead of schedule by the time they went home for Christmas.
Beyond the assists from the host and the host broadcaster, NBC has also been refining its between-Games processes, and this is bearing fruit.
In the “short” 16-17 month break between the closing ceremonies in Rio and the PyeongChang Opening Ceremony, the IBC infrastructure RIBs (racks-in-a-box) were returned to Stamford, Conn., home of the NBC Sports Group complex. Here, the team had space and time to lay out the entire system, connect it and test it.
“Our configuration time has now exceeded our wiring time,” said Mazza. “We used to get on site and spend a lot of time wiring before it could be configured… now between the fiber deployment and the increased complexity of the configuration, it’s the other way around.”
This means that everything has to get preconfigured in the pre-fab before shipping. When the team arrived all they had to do was run all the fiber—which is pretty quick—and about 90 percent of the configuration was done.
Setting and configuring the system in Stamford had benefits beyond a fast setup once on-site. While in Stamford, the IBC racks could be connected to Stamford with short fiber runs for configuration mock-ups.
“Not to oversimply it, but that short piece of fiber just extended to 5,000 miles when we got here,” said Terry Adams, vice president IBC Engineering, NBC Olympics, working his 11th Games.
Getting so much done in Stamford also allowed for extensive size and scope modelling, particularly for the asset management systems and edit systems so crucial for all the remote production that has become standard.
“We didn’t have to wait to plug in the actual circuits to mock up hundreds of users and workflows, so we have greater confidence than ever before in the stability of the system,” said Darryl Jefferson, vice president Post and Digital Workflow, NBC Sports & Olympics, a relative “newcomer” with six Olympic Games under his belt. “We’ve had challenges in the past with size and scope, so we put a lot of energy into stability and fine tuning some features such as being able to put in more logs from more places, and more integration with data sources,” he said.
A birds-eye view of NBC Olympics’ primetime studio
The benefits of the pre-integration (and the broader remote production design) also rippled in to the venue systems deployed in South Korea more than in past Olympics.
“All of the pre-integration that we did in Stamford allowed us to do proofs-of-concept for the remote production we’re doing between venues and the IBC and for venue-to-venue efforts,” said Chip Adams, vice president, Venue Engineering, NBC Olympics, who is on his 13th Games.
In PyeongChang the pre-work has been particularly helpful, as the “field shop” where the venue equipment is typically configured and loaded onto trucks that can access the venues is about two hours away from the Olympic city, where the IBC is located.
“This was the other rationale for putting the systems all together in Stamford—containerizing them for easy shipping and deployment, and laying out every IP address for the configuration before they go in to the box,” said Chip Adams.
Jefferson said that one of the other curve balls that has become increasingly important is cyber security. “We’ve had to patch and repatch and triple patch every piece of gear based on whatever is happening out there in the big, bad world,” he said. “It’s a lot of Microsoft updates and security updates.”
@HOME KEEPS GROWING
Although it has roots going all the way back to the Centennial Olympic Games in Atlanta, where rooms at 30 Rock in New York connected to the Atlanta IBC via T1 lines, the @home remote production effort really started to come into its own in Torino in 2004 and more fully in Beijing by 2008.
The PyeongChang NBC production will have seven control rooms located in Stamford, up from just two in Sochi. Errol Foremaster, vice president, engineering operations and planning, NBC Olympics, who is working his 13th Games, said these control rooms were largely feeding the tonnage of cable hours across several NBC properties, with the network shows coming out of South Korea.
Foremaster said that because of the time difference (14 hours between Korea and EST) almost everything would be shown live in the U.S. Other than the overnight period in Korea when nothing new is happening, there will generally be live content on an NBC platform, often the broadcast.
“Part of the preparation and production challenge is related to the time differences, and keeping straight where we are with live content,” said Foremaster. “With today’s technology and society, live is just expected.”
NBC will have more live streaming from more sources than ever before at a Winter Games, and this will be the first Olympics where Opening Ceremonies will be streamed live to the states on all platforms.
NBC is also offering more course animations, more slo-mos and more ISO cameras as ancillary stream sources available to the general public.
“Enhanced streaming,” is new and available from a few PyeongChang venues. This service is for connected TV customers, and might typically consist of a double box screen, where one screen is the host feed and a second is an ISO camera, with stats or other content displayed outside those sources.
CONNECTIVITY
NBC Olympics is relying on 108 HD feeds from Korea, with 32 returns. Most feeds will go home twice, once in a streaming copy and once in a broadcast/cable copy.
The infrastructure for this is four 10 GB highly redundant paths provided by AT&T. Two of these are dedicated mostly to video and two are dedicated to IP streaming and command and control. About 2 percent of the video is also on satellite as extra back-up.
Connectivity and remote production does not have to be intercontinental. One trend on the upswing in South Korea is increased venue-to-venue connectivity and venue-to-IBC connectivity, all to aid cost-effective production.
An example is Slalom and Giant Slalom. These venues have 36 feeds (and four returns) connected to the Downhill venue which is located about an hour away, so one truck can manage the production. “For the first time we have a 10GB connection between venues, in order to consolidate the number of trucks,” said Chip Adams.
A similar situation is happening at ski jumping, but that venue has 12 feeds connected directly to “Control X” in the IBC. That same control room will receive 36 feeds from the Closing Ceremonies, which will be cut in the IBC.
SOCIAL MEDIA
Production for social media outlets is growing, and integrated into the effort. Of particular note this year is a partnership with BuzzFeed, which produces the NBC presence on Snapchat. BuzzFeed has studio space in South Korea, and field crews as well. NBCUniversal has an interest in BuzzFeed and Snapchat’s parent company, Snap Inc. NBC Olympics is also working with Vox Media on a new Games-themed podcast called “The Podium.”
SPECIALTY CAMERAS, HDR & VR
Chip Adams said the biggest change coming in high-speed and specialty cameras is the sheer number of units. “We are going to have six 6x cameras, and the host broadcaster has something like 44 super slo-mo cameras of various types,” he said. “They’ve also deployed 133 mini cams, 14 cable cams and 11 rail cams across the venues.”
It’s less about new technologies in specialty cameras and more about the expectation that more and more will be deployed, said Chip Adams.
Virtual reality coverage is intriguing and growing, if still finding its niche. OBS has several venues with 180 degree camera rigs and they are offering a line cut and the individual cameras to rightsholders. Unlike Rio, the content will not be on a one-day delay.
NBC’s VR coverage is greatly expanded from Rio, with more than 50 hours of live virtual reality coverage powered by Intel True VR, available to authenticated users with Windows Mixed Reality headsets, Samsung Gear VR, and both Google Cardboard and Google Daydream, with compatible iOS or Android devices via the NBC Sports VR app. This is the first time VR is available from a Winter Olympics and the first time Olympic VR programming will be live in the U.S. on a wide range of devices and platforms.
Comcast is sponsoring a technology demonstration of 4K HDR and ATMOS sound from Korea that is a substantial increase from the small trial done from Rio. The next-day delay channel will offer 6-12 hours daily. This increase is due to an expanded effort from the host and particularly Japan’s NHK, which is producing a good amount of 8K with HDR content from Korea.
NHK is providing NBC with a downconverted feed of their 8K HDR coverage, and OBS’s 4K SDR coverage from hockey will be “stretched” up to HDR. There are two 4K feeds from PyeongChang to SPOC (Sports Production Operations Center in Stamford) and one return. In SPOC, a small production team in a 4K mobile unit, two Off-Tube commentary booths and multiple sets of announcers are creating the voiceover for this coverage. This will be fed as linear and VOD content to Comcast and any other MVPDs who have signed up.
Those 4K HDR signals in Stamford are relying on an Evertz IP routing system, which is also supporting much of the @home live broadcast, as well as digital production products like enhanced streaming.
“New for these Games, we have increased our IP routing capability to accommodate 4K HDR signals between South Korea and Stamford, as well as providing 4K HDR signals to our Off-Tube announce booths, the 4K truck, and our distribution point in Englewood Cliffs, N.J.,” said Tim Canary, vice president, engineering Stamford & @home, who is on his 10th Games.
THE HARDWARE
So much of the hardware and software that supports the crew’s effort changes only slightly from year to year. Sony continues to be NBC’s biggest Olympic supplier, with studio cameras, high-speed cameras, OLED monitors, production switchers and more throughout the IBC and venues. “Also the service and integration teams who have been doing this for years are really key,” said Mazza.
Canon’s lenses, and its service team, are at the tip of the production chain. Harmonic spinning disks are central for a lot of the ingest, the MAM and the Highlights Factory clip production, and EVS servers and software also remain firmly entrenched in the workflow. Avid craft editing and Grass Valley glue and infrastructure are contributing heavily as well.
Mobile units—some of which were delayed due to heavy weather—are from Game Creek and NEP. The latter has a history with NBC that goes all the way back to the last time South Korea hosted an Olympic Games, in Seoul in 1988.
PEOPLE POWER
Past experience is something that cannot be understated when it comes to evolving the production over decades and just getting the current job done, according to Mazza. The six staffers quoted in this article have worked an amazing 68 Olympic Games.
And those six point to Senior Core Engineer Chris Jorgensen as the smartest engineer and foremost problem solver on staff. He’s done eleven Games.
“We could not have done all these Olympics without his expertise,” said Mazza. “Whenever we can’t solve a problem on anything from plumbing to physics to power or air conditioning, he comes up with a solution.”
Mazza added that about 85 percent of the freelancers on staff for the big broadcast are Olympic veterans as well, and this travelling road show shows up every two years and gets the job done.
“We could never hit the level of content complexity and output without coming in to it with all this experience,” said Mazza. “It’s been a long, evolutionary journey and we couldn’t be at the level we’re at without the series of successes and some failures along the way.”
