Multiple challenges give rise to creative fixes
DENVER
Consumers today can see video on diverse devices from analog TV sets to plasma HDTV screens to computer screens to cellphones to iPods. For video producers, the goal is to create content that can be easily converted for each of these platforms. But the myriad formats create myriad challenges.
"A number of problems need to be resolved before we achieve the 'create once and play anywhere' model," said Brett Jenkins, vice president of engineering for Thales Broadcast and Multimedia in Southwick, Mass., recently acquired by Thomson Grass Valley. "Content creators usually produce for a particular format, such as HD in ATSC, so they don't want the aspect ratios changed, or they like the look of a particular frame rate, such as 30 frames per second. These are just some the barriers to creating content once that plays anywhere."
The chief technical barrier is generating video in sufficient quality in real time, said Jed Deame, vice president and general manager of the Teranex Division in Silicon Optix in Orlando, Fla.
"Software encoders on platforms like the Macintosh and PC can do the best encoding because they encode the video in at least two passes, but they cannot handle the compression in real time like the hardware encoders," he said.
The challenge at low bit-rates is avoiding noise, he said. If a film or video was shot in low light or turns out grainy, for instance, how does the codec know not to encode all those little specs of noise? Teranex has developed noise reduction systems for mastering film to tape, and these are useful for iPod and IPTV encoding, "but it's not a perfect solution."
"A fundamental issue is frame rate judder," said Steve Sherlock, director of technology and marketing for Brick House Video in Hampshire, U.K.. "Jumpy multiple edges are common when converting from 50 Hz to 60 Hz, from PAL to NTSC, and these problems persists when converting to other platforms.
Motion compensation is a solution, he said.
"Snell and Wilcox has shown how you can get far fewer artifacts than with linear standards conversion, yet a lot of work still needs to be done to get the clean video that would maker all customers happy on every platform."
ASPECT RATIO
Another traditional standards conversion issue that affects cross-platform repurposing is aspect ratio conversion, Sherlock said. Part of the problem is deciding how much of the 16:9 image stays in the 4:3 frame.
"How can you convert an HDTV image intended for a 43-inch plasma screen to play cleanly on a tiny mobile phone screen?" he said.
Michel Proulx, chief technical officer at Miranda Technologies in Montreal said, "you can't just downsize the logo graphics as part of downconverting the video. You need to separately reduce and simplify the graphics, then overlay these fresh images over the shrunken video."
Copyright issues also pose a barrier, he said.
"Specialty channels like Home & Garden and the Food Network often don't own the content they distribute, so they need to negotiate the rights to repurpose the content for multiple platforms."
Deame agreed that the biggest barrier to cross-platform repurposing is not technical but legal. "The studios and content providers have lacked business models and agreements that allow it to happen. We are seeing more content on Apple's iTunes Web site, and that says they are starting to come around."
Said Proulx, "It's not a technical breakthrough so much as an industry breakthrough. Since Apple first launched the video iPod last November, we're seeing a serious shift in the industry culture. The pace of change over the past six months has been truly staggering."
USERS SELECT DISPLAY
Jenkins confirmed that the industry is adapting quickly. For example, Thomson Grass Valley has written an application called "EyePhone" that is being used today by some of its customers to test and validate mobile media delivery systems using DVB-H and H-264 technologies.
"We won't see one format or one standard rule the world," Jenkins said. "It's hard to get everyone to agree on the same standard, and the trend is toward flexibility that allows multiple formats on multiple platforms, so these multiple formats must coexist."
Deame said the best hope lay in improved compression technology that can deliver enough quality at a small enough bit-rate. He noted that iPod video bit-rates need to get down to 600 kbps or lower. Compression based on the MPEG-4 Part 10 video teleconferencing standards supports a variable bit-rate more than MPEG-2. "This let's you cram as much as you can into the data stream, so even 700 kbps looks close to DVD quality."
AVC codecs can get down to 100 to 200 kbps and still have acceptable video quality for iPods and the hand-held mobile devices with small screens. "Compression technology is the key," Deame said.
Deame is hesitant to say when software encoders will match hardware encoders for real-time quality. "I'd like to think it would be 12 to 18 months, but don't hold me to it."
"I see the biggest hope is the use of file-based formats for content distribution," said Jenkins. Instead of real-time satellite or cable distribution on an MPEG-2 transport stream, "we're seeing non-real time files being downloaded by the end user from the closest possible server using IP distribution technology. Even iPods are using file-based delivery of MPEG-3 or MPEG-4 content. That's a big paradigm shift," he said.
Proulx identified another vital trend. "Streaming video over the same technology used for wireless telephony is not practical because there's not enough bandwidth. An alternative is digital RF transmission, such as COFDM. We're already seeing this in Japan, where new phones have both types of receivers, microwave and RF, so video does not need to compete for bandwidth with voice and messaging."
To achieve the optimum viewer experience, content creators need to produce the content that supports various platforms, according to John Delay, director of strategic management for the Broadcast Communications division of Harris Corp. in Melbourne, Fla. "But it's not practical or affordable to produce a different version for every platform," he said.
Delay said better transcoding needs to be combined with better form manipulation, so the transcoder knows whether to show a close up of the character's face or the glass of wine in his hand or to intercut between the two.
Wavelet compression might be the answer here, Delay speculated.
"You take multiple slices of the image within each domain, so recoverable data is planted in each slice, and then you tell the transcoder how to pick and choose among those elements when reconstructing each frame. MPEG-4 supports object frameworks, and that may be where the opportunities await us, but the industry is not yet far enough along that someone has solved these problems," he said.
"When the industry feels enough pain from having to produce content more than once for multiple platforms," Delay said, "that's when we will demand that someone solve these problems. We already have a lot of pieces of the puzzle on the table. We just have to figure out how to put the pieces together."
