DAMASCUS, SYRIA—Imagine being ABC News reporter Alexander Marquardt, reporting from a war zone in the Syrian capital of Damascus, apprising viewers of what he sees around him. Basically, that’s what TV reporters do.
But is it, really?
ABC News Reporter Alex Marquardt shoots VR video with the Jaunt ONE on location in Syria. That question has arisen more in the past year or so, because not only can a reporter working with a virtual reality (VR) camera still report about their surroundings, they can capture the 360-degree survey of the scene and bring the viewer along while doing so.
That’s where companies like Palo Alto, Calif.-based Jaunt enter the picture. Jaunt has manufactured a 24-lens camera and end-to-end solution that heightens the scale and presentation of professional grade 360-degree stereoscopic VR content—and has raised $100 million in venture capital, including a recent $65 million infusion that was led by Disney.
Now that the technology has already been employed in a war and numerous other scenarios, it is hoped that it will become less expensive, and therefore more often visible on a screen near you.
Jaunt chief technical officer and Co-founder Arthur van Hoff with the Jaunt ONE camera
LOOKING AROUND
Jaunt was co-founded in mid-2013 by three Silicon Valley veterans, including CEO Jens Christensen, and Chief Technology Officer Arthur van Hoff. At that point, VR was getting on the radar, “but in a subtle way,” said Scott Broock, vice president of content for the Jaunt.
“Oculus was developing the Rift, a tethered, head-mounted display that was intended for VR game development, but that was not the background or focus of our founders,” said Broock. “They saw VR was a very interesting trend that used computational photographic principals to capture the real world; the catch was to capture it in true 3-D, so you would get stereoscopic 3-D no matter where you looked.”
That’s tough to do with traditional cameras, he said, “Because the focal point is, greatly simplifying, where the sensor meets the lens. If you arrange a number of off-the-shelf cameras in a ring, individually or in pairs, a line drawn through each of the focal points lies on the outside of a sphere — which is the wrong place for an array and causes distortion. With our camera, the Jaunt ONE (formerly known as Jaunt NEO), every point in space has at least three lenses on it and we mathematically compute the correct common focal point on the ‘inside’ of the array, where it needs to be. We calculate a virtual camera with all the physics of a real camera.”
What the virtual camera creates is “a computational left eye and a computational right eye,” said Broock. “The viewer turns within these recorded spheres.”
The other key to the VR experience is the audio, which is captured using a tetrahedral microphone with four capsules that garner sound impulses from every direction. “That’s important when you record life-like audio in the 3-D,” he said. “It has to have the same direction as the visuals or you don’t get the immersive effect.”
FIFTH TIME’S A CHARM
Of equal importance is the stitching of the content, to ensure that each feed is in sync in the final edit. While that task has gotten easier in the past year, “It can still be tedious,” said Broock. “Jaunt’s computational algorithm automates this process in the cloud, avoiding the tedious, time-consuming and expensive hand stitching that almost all other approaches require.” He noted that Jaunt created five prototypes before presenting the Jaunt ONE, the first professional grade camera for shooting high-end cinematic VR.
“The first prototypes were based on GoPro Video cameras, but we moved away of them … For instance, GoPro lenses are not as refined as [those in a] pro system, since they are manufactured for the mass market. We’ve also switched over to a much larger sensor with a global shutter, which dramatically improves resolution and sync.”
How to Watch Jaunt VR For those who want to experience Jaunt VR themselves, all they need is a smartphone. Users with smartphones built in the last year and a half can watch Jaunt’s content in stereoscopic VR with either the Jaunt VR iOS app or the Inside Syria VR app on Google Play.
Google Cardboard supports any smartphone with a screen up to 6 inche
To get the fully immersive VR experience, users will need the Google Cardboard viewer, or other VR viewing devices, like the Oculus. Google’s new cardboard viewer can be assembled in three steps and supports any phone with a screen up to 6 inches. If users don’t have a Google Cardboard viewer, they can still access a monoscopic, 360 degree view by pinching the screen; this 360 view is also available online at abcnews.go.com.
Also, the sensor is larger for the Jaunt ONE because it performs much better in low light and has much better range, as well as a global shutter (which is key for fast-moving objects), as opposed to a rolling shutter. “That makes for a much better system that is specifically designed for VR,” he said.
After the video and audio are process in the cloud, the resulting file can be edited using traditional post processing tools.
“There are two globes, one for each eye, as rectilinear images in each frame,” Broock said. “From that point, edits, transitions, color correction and other editing functions are completed on Final Cut Pro X, or on a recent version of Avid or Premiere, with ProTools or Dolby Atmos for sound.”
From the standpoint of ABC News, the 360-degree camera is a welcome new tool in its arsenal that gives the network “the ability to tell great stories,” said Dan Silver, executive producer.
“In [the case of the Syria story], it’s a little more than just, ‘Wow,’” Silver said. “[The story marked] the first time we’ve used it, but we can already see that it gives us a much broader canvas to paint a picture on with a news report or a short documentary.”
The operation of the camera, he said, “is really simple. You just hit play or record. Framing is fairly simple, too, because you just frame from point zero and shoot all around you. What we mainly noticed about the camera was that it was smaller than we thought it would be. We joked around that it looks like something out of a sci-fi movie, like the head of R2D2 or a Dr. Who robot.”
Other companies are also involved in the VR fray. The Associated Press, for instance, uses Matterport, a Mountain View, Calif.- based company that specializes in immersive 3D models via their camera.
The AP has also used VR technology on one project so far, what it called, “The Suite Life,” for a virtual reality tour. Viewers can visit the suite class on Singapore Airlines, the Grand Duplex suite aboard Cunard’s Queen Mary 2 oceanliner and the Ty Warner Penthouse suite in the Four Seasons Hotel New York.
Like Silver, Paul Cheung, interactive director with the AP, also of New York, is finding himself looking forward to what’s next. “We’re also experimenting with creating a 360-degree video using Cinema 4-D and 3D software,” he said Cheung, adding that the AP is planning to purchase the Theta 360 camera, “so we’re going to experiment with this trend.
To him, it’s all about that heightened access. “[It allows people to go] anyplace that they can’t usually go,” he said. “For instance, we just did a story about the tunnel that [Mexican drug lord] El Chapo used to escape,” said Cheung. “Can you imagine if we’d been able to present that coverage as a VR experience?”
HARVEST OF CHANGE
Gannett has also gone the VR route. The corporation produced a special report in September 2014, “Harvest of Change,” which was set on an Iowa farm, and won an Edward R. Murrow Award. “That was a breakthrough, as far as large media organizations using 360 VR to tell a story,” said Mitch Gelman, vice president of product for the corporation.
Since then, Gannett has produced documentary-style features about the Kentucky Derby, Cincinnati Reds, U.S. National Ski team and, recently, used an array of six GoPro cameras to broadcast live, in 360-degrees, the speeches of about 20 presidential candidates at the Iowa State Fair.
He agrees that the movement of VR to wider acceptance will follow the improvement of the technology and the lessening of the expense. “In addition to the cameras, what has also already improved is the autostitching software. A year ago, if you wanted to do a full spherical video, you had to hand stitch it yourself; now, there are more efficient ways to create 360-degree experiences with less post production.”
On that note, Gelman thinks much more journalism will be presented in VR— “especially when HTC, Oculus and Samsung come up with their new, less expensive and more immersive headsets,” he said.
