ATLANTA—On June 20, 2018 The Weather Channel blew away its viewers by apparently bringing a tornado into its studio. With the help of 3D augmented reality graphics rendered by The Future Group’s Frontier graphics creation system (processed and played out by Ross Video’s UX control system) and Mo-Sys’ camera tracking system, TWC meteorologist Jim Cantore appeared to experience the real-time ravages of a tornado in-studio as it progressed from an EF0 (Enhanced Fujita scale; winds 65-85 mph) to an EF5 (winds over 200 mph).
As the twister intensified, TWC viewers saw Cantore jump back from a falling electrical pole pulsing with electricity inside the studio. He was subsequently menaced by a flying 2x4 wooden beam that blasted through a studio TV monitor, and narrowly missed by a mangled red sedan that crashed through the studio ceiling; the left front headlight still flashing and clicking as Cantore soldiered on with his commentary. (A hyper-realistic outdoor camera view was interspersed with the studio shots; allowing Cantore to explain the tornado’s evolution from EF0 to EF5 throughout the broadcast.)
Eventually the lights flickered as the TWC studio began to fall apart under the storm’s ferocity. Jim Cantore took refuge in a “safe room” before the signal was apparently “lost” as TWC went to commercial. After the commercial break, Cantore was seen standing outside in the apocalyptic chaos left by the EF5 tornado; until the TWC’s studio outline appeared around him in grid form, and then rematerialized to its original undamaged appearance. (For the record, Cantore’s introduction and after-commercial conclusion were done live; the rest was prerecorded.)
As convincing as the tornado vignette was to watch—the SFX was worthy of a Hollywood disaster movie—none of the mayhem was real. Which was precisely the point: TWC is adopting the wow factor of augmented and virtual reality graphics to make its TV content incredibly compelling.
“This is our bold first step into immersive weather presentation,” said Michael Potts, vice president of design for the Weather Group. Referring to the channel’s approach as “immersive mixed reality,” Potts said “we’re dropping the boundaries between what is real and what isn’t real, so we can take our talent and put them into the weather, or bring the weather into our studio. The goal for doing so is to create a much more meaningful viewing experience for the audience.”
'EXPLAINER VIDEO'
Creating this complex “explainer video” (as Potts calls it) of TWC’s in-studio tornado was a massive effort, requiring “lots of brainstorming sessions between The Weather Channel and The Future Group,” said Lawrence Jones; head of North America Operations for The Future Group.
“After we had all these brainstorming sessions together, TWC sent us a script that we used to create movie-style storyboards of the tornado video,” Jones said. “Once we agreed upon those boards, we created concept art to depict what these graphics would look like; including the flying 2x4 and the photo-realistic car. For this to work, every graphic element and overlay had to be planned out perfectly, so that it could be done within the limits of the technology.”
It took two months for The Future Group’s teams in Bulgaria, Norway, and the United States to create the AR graphics for the video. The graphics included the virtual TWC studio that rematerialized around Cantore after the commercial break. Once reconstructed in full onscreen, the virtual studio was indistinguishable from the real studio.
These graphics were designed to run on The Future Group’s Frontier graphics engine, which is based on the Unreal Engine gaming graphics system created by Epic Games. Ross began partnering with The Future Group in 2015 when it announced a seven-figure investment in the company, followed by a technology partnership a year later.
“The original Unreal Engine wasn’t designed for broadcast,” said Gideon Ferber, director of product management and business development for Ross Video’s Virtual Solution division. “So The Future Group built their own version of the Unreal Engine that included broadcast features such as genlock, frame buffers, and Fill/Key output. Ross then added the tracking integration, control interface, and everything else you need to operate Frontier on-air in a broadcast setting.”
[Read: NAB Show: Epic Games Integrates Unreal Engine With Strategic Partners]
With The Future Group’s equipment in place at TWC’s Atlanta production center, the crew held rehearsals to iron out the bugs and ready the segment for broadcast. These rehearsals included an enthusiastic Cantore, who memorized theatrical-style blocking (knowing where he physically had to be at what time) and learning his lines, alongside his regular broadcast duties.
Cantore’s hard work resulted in a realistic, apparently spontaneous on-air performance as he endured (and educationally exploited) the tornado’s wrath.
“Jim brought an amazing amount of life to this,” said Potts. “Yes, the graphics are amazing, and the technology underneath it makes it possible. But our talent drives this thing. With a wonderful narrative to work with, Jim was the hero of this project.”
MORE CHAOS TO COME?
TWC’s success in blending real video and AR graphics in Cantore’s tornado segment has won the network lots of positive attention and justified its push into AR/VR-enhanced content. Right now, TWC hopes to produce 80 percent of its programming using AR/VR by 2020.
TWC will air its second AR-enhanced explainer video on July 31.
“We’re going to look at the power of lightning,” said Potts. This time Cantore can relax. Rather than TWC zapping him just weeks after dropping a car close to his head, “we’re going to spread the love and bring some other talent into this,” Potts said.
