Shortly before the start of the SMPTE 2017 Technical Conference & Exhibition, TV Technology spoke with Bill Redmann, a Technicolor Fellow and director of standards for Immersive Media Technologies at Technicolor, about the session track he chaired on “Advances in Immersive Storytelling VR/AR/MR & 360° Video Pt1: Understanding Immersive Storytelling.”
TV TECHNONOGY: How do you see immersive storytelling evolve?
BILL REDMAN: So much about the evolution of storytelling follows a pattern that might start with an improvement in technology, but really depends on its adoption by avant garde storytellers. These are the storytellers willing to experiment, take a risk, stretch their imaginations and almost surely make some mistakes, but these are the ones who ultimately discover a new path, develop a new kind of relationship with their audience and demonstrate it for others to take up and improve on in turn. Very quickly, the technical innovation is taxed and limitations, perhaps flaws, are discovered and railed against, beginning the cycle again, perhaps even before it is completed.
Today, we see technology that can represent whole worlds that storytellers build and offer for us to inhabit—as evocative as you can make with a brushstroke or as realistic and detailed as you choose to afford.
In surveying these advances, our speakers stand in multiple roles. Some represent the avant garde themselves, showing and critiquing their own work. Some are scholars, surveying the breadth of these creations, searching for generalizations and looking for rules, useful for informing subsequent creators or the next iteration. Some represent the cycle coming full circle—technologists measuring success or reacting to discovered shortcomings, and proposing new improvements.
TVT: What are the most interesting applications/projects you've seen?
REDMANN: To me, immersive storytelling is most interesting when it's social. Looking back, visits to the multiplex or drive-in theaters were not typically solo activities. These were things you did with family or friends. My friends and I use to sit around a single Apple II, shouting out ideas while someone typed commands into some adventure game. My kids grew up playing games online, talking into headsets, and sharing adventures with good friends that they might not meet in person until years later. I just saw “Blade Runner 2049” in IMAX with my wife and four friends. We talked about it waiting in queue, and we hung out talking about it for 45 minutes afterward. Immersion works best when its shared.
In some degree, modern immersive technology lives on or near set—where actors inhabit worlds and share them with us, the audience, as they always have. But now we're being invited up on stage, onto the set. It's an advance build upon techniques that reach back past Méliès to Sophocles and their respective audiences. What we've seen developed in multi-player games is a big hint as to what is coming—when you look at what is happening with The Void or Nomadic, immersive media that you share with family and friends is upon us.
TVT: What technical challenges or constraints are most impacting creators?
REDMANN: The biggest constraint impacting creators is education. So much is so new. We're just past the point in the cycle where only the avant garde had firsthand experience. The knowledge is out there; it's spreading; and that is important. SMPTE is helping that mission through this event, as are we, with the Technicolor Experience Center. Once creators have a sense of what they want to do and how they want to do it, the biggest technical challenge becomes a lack of standards.
So many capture devices, production tools, distribution platforms and display equipment are effectively isolated from each other by mismatched formats, unfamiliar adjustments or ill-matched theories of operation. When choice of a particular camera or display is made, the ordinary difficulties of moviemaking are handled by the usual means, but that choice can constrain the reach of the work to the corresponding walled garden. Standards are going to help break down those walls and extend the artists’ skills and communicative reach, so that they apply more broadly. Or, technology is going to build doors through those walls—that can happen, too.
TVT: What are three things you want content creators to understand about using these technologies/tools?
REDMANN: Content creators are craft masters. While new media technologies are just tools, in the right hands, they are really high-quality tools. These tools aid their craft in three ways. First, there are technologies that enhance their abilities. A simple example is how a higher-resolution camera enables them to capture better images, but what about a way to meet that higher-resolution camera's increased demand for correct lens settings?
Second, there are technologies that allow the creator to operate at a higher level of abstraction. For example, rather than editing many different versions of a film to fit every screen format to be supported, capture instead a description of what's important in a scene, to describe how cropping decisions should be made, so that those decisions can be applied as needed when a specific format is needed. This extends the creative's reach and speed, yet maintains the integrity and quality of the work.
Third, there are technologies that will let them do more. Any given day only has so many hours, and those hours only allow for so many decisions to be made, so many edits, so many adjustments, so many retries. To be able to increase those limits, to try something more quickly, to have a system propose a better starting point, one that might be closer because it is informed by the artist's earlier decisions, or systems that might be able to retroactively apply a change across an entire workflow.
TVT: What advances have been made to improve the immersive user experience?
REDMANN: You ask, "What are the biggest advances for improving the immersive user experience?" Hah! Were you expecting a technology answer? Nope. The biggest advances in user experience come from the experience gained by creatives working in the medium. Throughout the history of video game platforms, you'd see that even though hardware wouldn't change at all over a console's first five years, the games themselves got better and better, year after year. All of that improvement was due to improvements in the creative's ideas on how to use the platform and their recognition of what about their tools and workflows needed improvement.
So today, with respect to immersive user experience, the biggest improvements come from creatives applying their experience, not only to their next work, but also to their tools, driving improvements that are a response to their earlier experiences. Oh, and more, better pixels.
