There is little doubt that the migration to the next generation of video (UHD/HDR) and advanced multichannel audio is well underway. 4K television sets are cheap, bandwidth is cheap(er), soundbars were the hit of 2019 International CES and content producers are beginning to understand the production benefits of UHD production with advanced audio features.
It is a different climate today than a decade ago with the transition to HD. The consumer technology implementation and installation is well underway. I believe that content producers and owners are seeing UHD/HDR and immersive sound as a premium feature for premium content. Adoption will accelerate as more content—movies, sports, games and music—become available in an enhanced spatialized format.
I asked a group of interested content producers, sound designers and mixers about their view of the path forward for immersive sound for sports and entertainment content. They included Stacey Foster, coordinating producer at “Saturday Night Live;” Fred Aldous, senior sound designer and mixer for Fox Sports; Mike Rokosa, technology executive at National Hot Rod Association (NHRA); and Karl Malone, director of sound design for NBC Sports and Olympics. (My comments are in italics.)
I started the conversation with the question: How long will it be before we see widespread adoption of immersive sound?
Stacey Foster: Every time I go to CES, I get encouraged with all the new technology. I think consumers are embracing soundbars and I expect the majority of the markets to be able to experience UHD with immersive sound somewhere between three to five years. I do not think it should take longer than that. At “SNL” we have worked very hard to push the entertainment side of NBC. Premium content is worth something. We took a couple of episodes and remixed them in Atmos and I was absolutely stunned, it galvanized me.
Mike Rokosa: I heard a Dolby Atmos demonstration for soccer and that started a discussion... OK, you can put the PA up there, but what else? With ball sports you are probably not going to put ball sounds above, but it got me thinking and talking about it. I felt there was something there for NHRA but I didn’t know what it was yet. So we worked with Dolby Labs and we came up with some unique approaches to our sound.
The entire panel agreed that immersive sound design will be guided by the content. Fred Aldous was excited about the prospects of immersive sound for NASCAR, but reserved on field sports like football and baseball.
Fred Aldous: I think if you were able to add overhead sounds to the in-car mix, that would enhance the mix and be a great effect to consume the listener. With a stick and ball sport, adding the additional ambiance and separating sources away from the front can give the consumer a more in-stadium experience.
Karl Malone: The first thing we need amongst ourselves, in our own networks, is the buy-in from production. Having producers and executive producers listen to immersive mixes and be familiar with what they can bring to their productions is very important.
Fred Aldous: I applaud the past chairman of Fox Sports, David Hill, because he wanted to change the way people watched and listened to sports. He gave us the resources to excel. At Fox, the sound is a key component of the brand, part of its signature.
NBC Sports and entertainment has been active to find production and distribution solutions for premium content produced in UHD with immersive sound.
Karl Malone: NBC began testing production practices and workflow for immersive sound in 2016 and continued with the PyeongChang Winter Olympics, where the remote venue A1’s sent various individual 5.1 beds, submixes and objects from some of the big venues back to our Stamford headquarters where we mixed them live in Atmos. Having one A1 mix all the Atmos events helped in having a consistent and reliable mix.
We also examined and tested a single (standalone) event. Our goal with HDR/ Atmos with Notre Dame was a single stream production. One truck outputting HDR and SDR with 5.1 and 5.1.4 respectively. One A1 mixing a 10-channel mix. The HD feed with 5.1 went to 30 Rock for commercial integration and direct to air and the HDR and 10-channel mix went to Englewood Cliffs for commercial integration, encoding and to air on DirecTV.
We worked out of NEP’s ND6, which was retrofitted with four full ceiling speakers and DP590 monitoring equipment. The room was redesigned to Dolby spec and they came in and tested for acoustic accuracy. Front desk production listened in stereo.
Our 10-channel truck mix was sent PCM to Englewood Cliffs (EC), the NBC SportsNet Master Control where commercial advertising was upmixed to immersive, integrated to the broadcast and then encoded to Dolby Atmos there. NBC has advanced technology labs in EC with HDR and Atmos monitoring with both professional “reference” and home “sitting room” confidence monitoring possibilities. We were given very accurate verbal and visual feedback from the labs to ensure that what we were airing was what we were monitoring using the “scene” emulations that the Dolby DP590 would output.
Stacey Foster: Beginning with the 2019 fall season of “SNL,” we are planning to produce 4K in the studio (cameras and graphics) and the switcher will send out a 2K live show and later distribute UHD/HDR with Atmos audio to our digital platform. Every mix position on the show will be able to work in an Atmosenabled environment. We are also working with Lawo and Calrec to implement a 12-channel-wide mixing and monitor matrix plus some other features. The after life of the show is in many ways the most important life of the show. It is going to be taxing to record all the stems, the whole archival side of the show needs a rethink, so there is plenty of homework to be done.
An early adopter has clearly been Mike Rokosa and the NHRA, which has been producing immersive sound since the beginning of their 2018 season.
Mike Rokosa: We had a lot to figure out from OB operation and workflow to replays, transmission and archive. Game Creek, our OB partner, did the work to integrate the listening environment into the truck—specifically fitting the A and B unit with overhead monitoring speakers. We had to build sound scenes for the replay machine. As much as was done with the live mix, there was equal if not more work on replays, records and distribution.
With transmission, we are doing a 10 channel mix and folding it down to 5.1 to Fox—actually what we deliver to Fox is 16 channels: 5.1 and the 10 stems for the Atmos. Our intent is to get the transmission people used to checking in 16 channels of audio through distribution.
Karl Malone: Any retrofit for HDR should include an Atmos upgrade and any new 4K trucks should offer the same. I can’t imagine the leading broadcast networks with the biggest sports events offering anything less.
Mike Rokosa: The audio community is aware of immersive sound, some are toying with it, others like NHRA are doing it. I do not think it is a big reach to get there. We only added a couple of microphones to our regular compliment… the place where you pick up the sound hasn’t changed, but what do we do with that sound is how we make it more compelling.
Fred Aldous: I really do not think that producing it (immersive sound) is going to be the bottleneck.
UHD/HDR and immersive sound content is already available with cinematic and dramatic productions and I would expect more major events like the Olympics to continue and expand the technology exhibition. Clearly a Super Bowl or a World Series is around the corner, but I see content with a strong fan base like the NHRA as a major driver to get the distribution systems ready sooner than later.
After talking to my colleagues and listening to immersive sound material for several years, I see that UHD and immersive sound has the potential to be much more than just “bigger”—it’s the new frontier for content producers and presenters.
Since ATSC 3.0 is a voluntary standard, I expect that we will see the adoption and implementation rate vary between content producers and providers. Additionally, there are still outstanding technical commitments and decisions to make, but the technology and technical options are on the table and ready. Clearly compelling content will drive consumer interest.
For most of us, the migration to immersive sound is a process and I think a lot can be learned from my discussion with my esteemed colleagues. Here are the takeaways:
1) Get everyone bought in—from management to production.
2) Start the dialog about sound design, with everybody—producers, directors, audio designers and engineering—and the sound design will follow.
3) Figure out the workflow—real time, replays, storage and archives.
4) Practice—capture, create and refine.
5) Push to get the distribution in place.
Immersive sound technology is getting ready. Are you?
Dennis Baxter has spent over 30 years as a sound designer contributing to hundreds of live events including nine Olympics Games.
He is the author of “A Practical Guide to Television Sound Engineering,” published in both English and Chinese and is currently working on a book about immersive sound practices and production. He can be reached atdbaxter@dennisbaxtersound.com.
