Wednesday's SMPTE sessions devoted to the burgeoning field of virtual reality/augmented reality (VR/AR) production and delivery focused primarily on strategies to ensure end users a high-quality consumer experience.
Regardless of the massive technical, creative, manufacturing and marketing resources pushing for mass adoption, presenters warned, VR/AR's future depends on consumers' positive overall experiences, which, in turn, require high standards for all the mediums' individual components.
Many people who have experienced VR or AR have quickly developed an almost messianic belief that they've seen "the future." But, according to studies cited, as many as half in groups of test subjects have become physically ill from the experience.
Mary-Luc Champel The presentation "VR/AR: Assuring Quality in Virtual Reality" covered many of the issues that could dissuade potential adopters, as well as some new solutions. "We don't want this to have the same fate as 3DTV," emphasized Mary-Luc Champel, standards director MPEG ATSC and contributor to multiple standards organizations.
The industry, he said, "must offer unique and novel experiences to users and solve issues that could cause rejection." Those include motion sickness ("If it makes the user sick, they won't go back for more," Champel noted) and issues such as refresh rate and flicker, vection—a wide field of view with fast moving objects in the near field—and duration of exposure by the user.
"Some people can experience it for hours," with no detrimental effects, Champel said, "while others can't even go 15 minutes." He shared that the data he's been privy to is far from being able to determine trends about who will respond what way, although the younger audiences seem better able to adapt to the experience overall than older ones and there is a difference between males and females (although he refused to elaborate on this, despite prodding from the audience).
He touched on sensory conflict: when your memory holds one set of data about where you're located in space and your sensory input tells you something different. And he discussed postural instability: when sense of balance tells us our environment has properties other than what we're "seeing" in the headset.
Champel also spoke about methods of compression that allow for massive amount of bandwidth required with minimal detriment to the experience. Because of motion-to-photon latency, it isn't possible to just render imagery to the viewport (the area the viewer happens to be looking towards), and bandwidth constraints can prevent sending full quality to 360 space. As a result, he advocated adaptive streaming that renders everything at highest quality in the viewport and the lowest quality 180-degrees from the viewport.
Champel believes VR/AR is definitely "coming fast" and that it's vital now to develop interoperable formats and standards for compression and delivery that can cope with the enormous bandwidth the mediums require without destroying the delicate effect.
Pierre Routhier Pierre (Pete) Routhier, president of Digital Troublemaker, began his session, "The Special Challenges of Offering High Quality Experience for VR Video," picking up on Champel's key warning (wise for any business) to not distribute products that "make consumers sick." As Routhier pointed out, virtual reality "is not recreating reality. It is just playing an optical trick. You have to know how this trick works to make it work."
He went into detail about a number of areas that must work perfectly in order for the "trick" to succeed. Among these is the way the multiple images are stitched together such that the seams are invisible and differences among capture devices don't destroy the illusion. "Master your lenses!" he stressed.
Different focal lengths, settings or optical characteristics can create images that can never be seamlessly stitched together. Different capture technologies that render images as well as artifacts differently will also destroy the sense of reality for the user when the resulting frames are stitched to one another.
Cameras must be synched, of course, or movement will appear unnatural from one image to the next. He warned of "blind spots," when objects close to the photographic rig get too close and seem to disappear in between the stitched images. In short, the worlds VR/AR creators are trying to build can easily fall apart with what Routhier referred to as "Frankenframes."
Likewise, Routhier expressed great concern that the VR/AR experience have consistency regardless of HMD (head-mounted device), which is completely absent today. Content from the past Olympics, he pointed out, looked quite different depending on HMD used.
"We need consensual standards," he sad, "not just lists of what everybody's doing. It was hard enough in 3DTV where everybody was using different kinds of glasses. The challenges here are increased by 10 times."
