BERLIN, GERMANY—When it comes to advances in telecommunications technology—and specifically video codecs—Germany’s Fraunhofer Heinrich Hertz Institute has been at the forefront. The organization is an integral member of the experts groups, which developed the H.264/AVC video codec nearly 20 years ago, as well as its successor, HEVC. It was also among the team that received numerous Technical Emmys including one in
2017 for the High Efficiency Video Codec.
Thomas Wiegand, executive director, Fraunhofer Heinrich Hertz Institute HHI; Benjamin Bross, project manager at the Video Coding & Analytics department with the Fraunhofer HHI and Detlev Marpe, head of Department Video Coding & Analytics at Fraunhofer HHI
Now celebrating its 90th year, TV Technology caught up with Benjamin Bross, project manager at the Video Coding & Analytics department with the Fraunhofer Heinrich Hertz Institute to discuss the evolution of HEVC and what’s next in video compression.
TV Technology:What are the latest developments on video compression standards work?
Benjamin Bross: The ITU Video Coding Expert Group (VCEG) and the ISO/IEC Moving Pictures Expert Group (MPEG) have already started working together on a successor to HEVC by forming the so-called “Joint Video Exploration Team” or JVET. We currently have a joint exploration codec model (JEM) already in place; this isn’t necessarily the successor codec, but it’s new technology that was proposed and investigated after the standardization of HEVC. This JEM codec provides about 30 percent bit rate reduction for the same objective quality compared to HEVC. So this is clear public evidence that it’s possible to further improve the compression performance compared to HEVC. We have started to work on the successor with a call for proposals and it is possible that we might soon be seeing another factor of two in coding efficiency relative to HEVC.
TVT:What does the call for proposals entail?
BB: Last October, JVET issued a call for proposals for video coding technology beyond HEVC and several companies including us are submitting technology. Each proposal will be independently tested and compared to HEVC and the results will be evaluated at the next JVET meeting in April. That’s when the official standardization process will begin with the goal to finish the new standard by 2020. Besides submitted coding technology, a lot of other influences will be taken into account. One of the main concerns from hardware manufacturers for example is complexity. Even if you find a nice technique that significantly reduces the bit rate, but it’s super complex, chipset vendors will say ‘that’s pretty nice, but there’s no way we can build a chip using that technology at a reasonable cost.’
[HEVC Advances Enhance Cell-Based ENG]
TVT:Is it the JVET that’s doing this?
BB: Yes, the JVET, ITU and ISO are teaming up to generate a codec that is both efficient and simple enough in order to be able to put the decoder on a chip that can be integrated into TV set-top boxes, mobile phones and so on. The goal is to reduce the bit rate by up to 50 percent of the previous standard for the same perceived subjective quality.
TVT:Can you elaborate on ‘subjective’ vs. ‘objective’ quality?
BB: You cannot perform subjective evaluation for every new thing that you add during standardization—the logistics to perform subjective testing for every new incremental step is just too high. That’s why you have an “objective error measurement” that you calculate, like some kind of mean squared error-based measure that gives you an indication of the quality; but usually these measurements are a little bit more pessimistic compared to subjective quality.
For example, in HEVC, we ended up with 38 percent bit-rate reduction over H.264/AVC for the same objective quality. When we did perform subjective evaluation and verification of that in independent test laboratories, we ended up with 50 percent bit-rate reduction for the same subjective quality perceived by the individuals involved in the test.
TVT:Would you say HEVC has attained its goal, has it been a success in terms of adoption?
BB: From a technical perspective, I would say totally. The support is very broad. Apple introduced it last year, so now every new iPhone is able to decode HEVC up to UHD—almost every Android phone is able to do that—and all of the new TV sets have HEVC built in, as well as new set-top boxes and other media devices. Then there’s the question of whether business-wise it makes sense to employ HEVC. Apple’s support was extremely important, so now we will see how many services will jump on that and actually use it.
For the broadcast community HEVC is pretty well set, especially. When it comes to UHD and new improvements like HDR and high frame rate, HEVC is the codec to use. In Germany, for example, they started the new terrestrial TV service with HDTV that uses HEVC and it’s working pretty well.
TVT:What about the future development of HEVC? Has it fully matured as a standard and is that why you’re working on a new compression standard?
BB: For compression efficiency, when it comes to encoders, there is a steady improvement still to come. When you look at MPEG-2, H.264/AVC, even over the last five years—and H.264/AVC is already quite old—encoders have improved in efficiency; improvements that are also being carried over to HEVC. I don’t think we’re at the end of the line efficiencywise for HEVC, but the encoders that are out there are already mature enough to provide the gain that HEVC promised at the beginning.
TVT:What is Fraunhofer HHI is doing in regards to HEVC?
BB: After we contributed core compression techniques to the standard, we started to develop products for HEVC encoding and decoding to the industry. We have a software encoder that is used in broadcast to encode HD and UHD content live, for example, for satellite and terrestrial television in Germany. Here, we do not sell the appliance, we provide the software encoding component.
TVT:Where do you see advances in video codecs going?
BB: Right now, everyone is following the same concept of hybrid video coding. They’re adding and improving a lot of parts of it, but the general concept remains the same.
TVT:How important is it for Silicon Valley companies like Google, Amazon, Facebook, Netflix and Apple to participate in the development of the successor to HEVC?
BB: It is very important as compressed video data are the by far highest percentage of bits on the Internet and mobile phones. These companies and everybody else who is serious about video compression will highly benefit from the coding efficiency improvements that the JVET process will surely deliver. They and many others are interested in video compression technology improvements that will help their businesses to flourish.
For example, Netflix has provided content to the JVET already. When we develop a standard we need test sequences that are characteristic of the content that is out there; we cannot test or develop a new standard using sequences captured with cameras from the ’90s.
TVT:Are you saying then that it’s to their benefit to participate in this process rather than try and go their own way?
BB: Sure, and they will be looking at the new video codec and realize that the achieved coding efficiency improvements are a big deal. But, if you create such a superior video codec as a result of a joint effort with a lot of players in the industry, there are patents involved. Hence, there needs to be a reasonable way to get a license.
