Immersive sound was almost a causality of the pandemic because in-person broadcast and sports productions were shuttered and virtually all engineering and financial resources were diverted to getting and keeping live content flowing—by remote control. Remote production practices and technology advanced quickly and subsequently benefitted less popular secondary sports and content for smaller targeted audiences with an easier engineering setup and overall cheaper production costs.
Consequently, immersive sound was pushed to the back burner. However, I believe 2024 will see a revitalization of the dimensional sound format. Why? Because immersive sound production and reproduction technology is available and affordable, however the next task for practitioners is to create the “wow” factor and stoke the consumer attraction.
Consider this: Television is basically two dimensional with the picture parked directly in front of the viewer and significantly, television depends on the sound to give dimensionality to the experience. Since the picture is in front and the screen stretches vertically, why not fill the front sound space with sound from top to bottom? It may not seem intuitive to put more sound above the viewer and it may not be appropriate for all applications, but a sport like basketball certainly benefits from sound above the viewer.
Where to Put the Sound
I believe there still is a reluctance to push the sound envelope (experience), which may be a hangover from uninspiring 5.1 surround sound production and strict loudness compliance; but clearly more atmosphere behind and above the viewer does not add much to the interest of the soundfield. Many sound practitioners approach immersive sound as another horizontal layer over the existing surround, ear-level layer. No doubt this is a limited creative approach to immersive sound.
Additionally horizontal layers of sound are hard for most soundbars to reproduce so by pushing the sound into the upper front space, the sound design also takes advantage of the mechanical sound reproduction of a soundbar using up-firing speakers. Immersive sound design for soundbars is more about creating a sound experience over reproduction of an accurate soundfield.
Basketball on television is perfect for advanced immersive sound applications because the visual coverage beyond a wide shot of the action on the court is close-up, head-to-toe of the athlete under the baskets. With the basket perceptibly above the athlete there is a clear motivation to put the basket sound above the viewer and create a front vertical soundfield of sports specific sounds. The front soundfield consists of the Left, Right, Left Height and Right Height sound zones acting as a coherent, controllable sound space where sound can be steered anywhere vertically in front of the listener.
Basketball sound is perfect for immersive soundbars and application of the principles of Front Vertical Soundfield Reinforcement (FVSR). What sounds go above the viewer? With film there is “room tone” ambiance as well as specific effects, and increasingly, film sound production uses electronic digital spatial enhancements.
Broadcasters and content streamers of dramatic and serial content produce their sound similarly to film, but with sports, the sonic palate is undeveloped beyond the absolute consumer-expected sounds like the crack of the bat. Imagine head-to-toe sound of the athlete or putting the viewer inside a bobsled at 80 mph.
What About Field Sports?
With basketball the benefits of advanced immersion applications such as FVSR is obvious, but look at the majority of televised sports, particularly field sports such as baseball as well as football and soccer. These sports are heavy ambiance with a center focus on the sport’s specific sounds.
With advanced immersive sound practices such as trajectory mapping it would be possible to recreate the dimensional movement of the ball in the soundfield—think a moving asteroid! Think AI-rendered soundfields. Or simply by elevating mono and stereo sounds, elements between the ear-level soundfield and the above soundfield can create a more integrated soundfield with clear correlation between the front and front height speakers.
Sound design for immersive sound is about reproducing a dimensional experience, which to me is all about creating and mixing for soundbars. Since the production sound mixer will probably not mix over a soundbar and the vast selection of consumer soundbars are so different in design it is hard to expect a sound practitioner to mix for soundbars.
However, the sound practitioner should be considering the reproduction side of sound. The sound mixer needs to be familiar with the physics of soundbars and have a willingness to experiment. After a little practice the sonic outcomes for the sound practitioner will quickly improve. Additionally, I have published a practical guide to immersive sound production with case studies and examples to share my experience and ideas and help jump start the immersive sound initiative.
The creative possibility for immersive sound is an open proposition although a challenge because the ATSC 3.0 did not mandate advanced audio standards for immersive sound. Some in our industry don’t really see the need for immersive sound, but its time has arrived, and it is up to the audio practitioners and the entertainment industry to stretch the boundaries and promote this truly innovative entertainment experience.
