As we come to the end of another decade in broadcasting I think this period of time was as dynamic and influential as any for the world of audio. Not so much with new or better technology, but with a convergence of the technologies to make possible advances in multichannel audio production, delivery and consumption a reality.
THE OLYMPIC EFFECT
The pace of adopting the next generation of sound is on schedule. In 1996 we heard the full implementation of stereo sound at the Summer Olympics and in 2008 we saw 2K and heard 5.1-surround sound from Beijing. At the Summer Olympics in 2020 there will be the full rollout of 4K and 9.1 immersive sound showing a clear trend where every 12 years the broadcast industry uses the Olympics to introduce or prove technologies. There is no doubt that immersive sound is quickly finding its way to the consumer through streaming and from progressive networks around the world such as NHK in Japan, Korea’s KBS and NBC in the United States, who understand the future of broadcasting.
This past year I looked into various immersive sound applications and technologies and will continue this exploration into dimensional sound in 2020 when world broadcasters roll out the Olympics in full 4K HDR and immersive sound.
NBC is planning to mix immersive sound from a control room in Tokyo and will feature content from high viewership events such as beach volleyball and opening and closing ceremonies. Karl Malone, Audio and audio systems engineering designer for NBC Sports and Olympics said that NBC has benefitted from its experience at the 2016 and 2018 Olympics Games plus the 2019 Notre Dame football season where immersive sound was produced. I think Karl has a plan and NBC will commit the resources for a successful immersive sound production. I’ll make sure to cover it.
The future for dimensional and interactive audio is bright, but recently I read something that disturbed me: A sound manager for a large 2020 sporting event was complaining about inadequate resources to produce immersive sound. Resources or not, the requirement for immersive sound is on the table just as the requirement for surround sound was on the table in 2008—even with woefully inadequate resources and an inexperienced crew.
CREATING IMMERSIVE SOUND
After this year-long exploration of immersive sound in the pages of TV Technology I am further convinced that it is not hard to create and produce immersive sound with minimum resources and would like to offer a couple of observations:
First, to create immersive sound a mixer needs to be able to hear the dimensional sound-field. Second, the mixer must be able to adequately position the sound elements in the sound-field; and finally the mixer must be able to make and maintain an artistic balance of the elements in the dimensional sound-field.
In the April issue of TV Technology I visited an OB van with less than perfect but adequate immersive monitoring. I can only say that NHRA puts out a credible 5.1.4 immersive sound production as well as 5.1 surround from one OB control room.
An alternative workflow would change the location for mixing and monitoring immersive sound. For more than a decade, FIFA World Cup and NBC have been sending mono and stereo stems from the venues back to master control to be mixed and I can clearly see the further trend to centralized control rooms for multiple remote productions.
There also seems to be some schools of thought that you need ambisonic microphones to create immersive sound. I spent 2019 recording side-by-side a 1st, 3rd and 4th Order ambisonics microphone along with 10 correlated and non-correlated spot microphones. I’ve come to the conclusion that ambisonic microphones sound great, but are not essential to creating immersive sound.
Immersive sound for sports and live entertainment is a subjective balance between venue ambiance and atmo-sphere, as well as relevant event-specific sound. With any sport (event) minimum immersive sound production can be as simple as injecting additional ambiance and atmosphere into the height speakers. To me, American and European football (soccer) are the definitive example of overhead atmospheric enhancements and this method does not require any localization. 3D panning is helpful for precise localization, but bottom line—convincing immersive sound for most sports does not require precise localization.
Some sound above the viewer/listener will usually create a sense of aural space for the 2D picture, but sound designers are wrestling with the concept of what sounds should be heard above the viewer when there is no obvious reason. Sure, we can put some sound up in the height channels, but how long is more crowd sound going to be a compelling reason for immersive sound.
SUCCESSFUL IMMERSIVE SOUND
Immersive sound production will develop and evolve with creative sound design and imagination. The picture is still two-dimensional and sound could become a production differentiator where “made for TV” sound such as NHRA will become a viewer draw.
There is no doubt that field sports such as football have no sports-relevant sound in the vertical axis, only ambiance and atmosphere, and artificial embellishments are probably not appropriate. But consider that perhaps enhanced ambiance is enough of an embellishment for football fans—immersive sound does not need to be over the top to be effective.
Hoping for interesting and successful immersive sound is not going to advance our audio agenda. NHK in Japan has spent almost two decades preparing for immersive sound. Dreaming, theorizing, research, testing, evaluation, planning, preparing new sound designs and documents have contributed to and facilitated a successful implementation of immersive sound within the Japanese broadcaster. NHK will produce the entire 2020 Olympic Games in 22.2 immersive sound.
Immersive sound will be proven one step at a time. Immersive sound is not new, but the concept of immersive sound is evolving. Remember, I said NBC would produce 9.1 immersive sound and NHK would produce 22.2; both formats claim to be immersive sound. What is the difference on a soundbar?
Advance audio designs are eminent but will be built on proven successes. What is the channel configuration for immersive sound? Is it 5.1.2, 5.1.4 or 22.2? I do not know. I am currently testing sound schemes that concentrate the listener’s attention forward and uses the height element to draw the focus beyond the normal left and right peripherally. This is an interesting concept for immersive sound and seems to work well with soundbars.
The migration to immersive sound must be planned, methodical and most importantly successful. Immersive sound must be marketed to the consumer and easy to install with minimum wires. Stereo and surround sound had growing pains, but I could not imagine going back to mono sound or black-and-white TV.
Immersive sound has a promising future and I look forward to covering it in the pages of TV Technology.
Dennis Baxter has spent more than 35 years in live broadcasting contributing to hundreds of live events including sound design for nine Olympic Games. He has earned multiple Emmy awards and is the author of “A Practical Guide to Television Sound Engineering,” published in both English and Chinese. He is currently working on a book about immersive sound practices and production. He can be reached atdbaxter@dennisbaxtersound.comor atwww.dennisbaxtersound.com.
