It was 100 years ago that innovations in sound-on-film led to the first commercial screenings of short motion pictures that incorporated sound. Though it would take another four years for the technology to be employed in the full-length feature The Jazz Singer, it was a revolutionary step forward.
Fast forward and today’s Main Stage session, “Immersive Storytelling: Expanding Audiences With XR in Games, Education and Location-Based Entertainment,” could preview the next step, which may promise to be as much evolutionary as revolutionary in video production. The session will explore what is next on the horizon for virtual reality (VR), augmented reality (AR) and extended reality (XR). From headsets at home and in schools to location-based entertainment venues, consumers are embracing innovative ways to tap into their favorite stories.
“Technology tends to be an evolutionary force for what we can do with the experience,” said session speaker Ted Schilowitz, futurist at Paramount Global. “It is an interesting perspective compared to what sound brought to the experience. Now we can look forward to what this simulated technology can bring — as it provides new ways to trick the brain, the eyes and other senses.”
Truly Immersive: Game On
Hardware will certainly play a key role. Immersive technology can add much value to the player/viewer and provides a unique experience only achievable through a headset.
“Take the PlayStation VR2, for example,” said session speaker Jake Zim, senior vice president of virtual reality at Sony Pictures Entertainment. “It has impressive new features like eye tracking, haptic feedback and foveated rendering, providing game developers with more ways to create unforgettable moments, blurring the lines between real life and the digital experience.”
Virtual worlds, including those found in VR, AR and XR through a headset, are now the most immersive form of entertainment as these allow gamers to take center stage and star in their favorite stories alongside their favorite characters.
“Even though we’ve already seen many significant hardware advancements, this technology has so much potential to grow, for games and many other use cases,” said Zim. “But the content needs to be there, which is where Sony Pictures VR steps in, leading the way forward with best-in-class developers to deliver new games that bring players directly into the universe of beloved franchises like Ghostbusters and Zombieland. Incredible content will bring players to VR and make way for innovations, shaping how we play and interact with one another.”
Challenges and Opportunities Ahead
As this technology moves to other platforms, it will come with new challenges and opportunities — including the compatibility between different hardware platforms, addressing latency concerns and even making sure the player/viewer is looking in the right direction.
Session speaker Aaron Grosky, president and COO of Dreamscape Immersive, said, “There are so many issues to consider. How do you control the frame rate? How do you boil down the essence of the story? And what is its narrative arc? We can control the hardware ecosystem in that the target devices are utilizing the highest frame rates, poly counts and fidelity. When you go to a broader space of devices, then you have to design for the most common targets. We expect to see a cloud-based 5G future playing a role where there isn’t a need for local compute. But as long as you have that local compute you could have many of these challenges.”
Presenting a linear experience could present its own challenges, as well as opportunities. As virtual worlds become more open — and thus all the more real-seeming — ways will be required to get the viewer to stay on track.
“This could take cues from the theme park experience,” said Schilowitz. “You must convince people to look in a particular direction. That is the main tenet of magic and really good theater. We’re just putting a layer of technology on top — but all the best tenets can still be employed.”
We are also entering a new period where people will want a truly magical experience that could be greater than what current technology can deliver. Hence, we may still see evolutionary steps forward — so the production world needs to ensure that it is still overwhelming instead of underwhelming with what it is able to deliver.
“This is very much about managing expectations,” said Grosky. “We expect early adopters to be the most vocal critics. We started looking at VR in 2014 and 2015 and were put off as we didn’t think we could quite get past that moment of ‘suspension of disbelief.’ But now we’re seeing that the technology is making the experience richer and we’re finally at that ‘ah ha’ moment.”
Why This Matters
Immersive storytelling could be the next great leap forward in terms of audience experience. It could be as great as the introduction of sound was 100 years ago.
Key Takeaways:
- Virtual worlds found in VR, AR and XR through a headset are the most immersive form of entertainment we have right now.
- The adoption of advanced technology is revolutionizing the immersive storytelling experience.
- We are on the cusp of a convergence between traditional entertainment and advanced technology.
Copyright NAB 2023.
