Video compression has been available in a standardized form for about 20 years. Over that time, there have been considerable advances in the technology. To get a better appreciation of how it will continue to evolve, it's useful to look at how it has developed so far. While there are several video compression systems in use today, some of them proprietary, the progress made by the MPEG working group forms a good basis for this analysis.
A look back
Work on MPEG-1 video and audio was started in 1988, with the first standards being released in the early 1990s. The system was designed to operate at compressed bit rates of up to about 1.5Mb/s and was primarily aimed at delivering digital video on storage media such as compact disc. One typical use was on the video CD (VCD). MPEG-1 also specifies audio coding, including the ubiquitous MP3 format (so-called because it is defined in MPEG-1 Layer III).
MPEG-1 described a standard method for defining picture types and groups of pictures, called intra, predictive and bipredictive. It broke the pictures down into slices of macroblocks, which are further divided into constituent blocks. These blocks then undergo transform coding, which converts the spatial video information into frequency information that can then be perceptually compressed by means of quantization. Interpicture redundancy is then removed by means of residuals (differences between pictures) and motion vectors (which predict the motion of objects between macroblocks). Finally, entropy coding is used to reduce the transmitted information by exploiting the statistical redundancy in the generated code words. (See Figure 1.)
MPEG-2, which was introduced in the mid-'90s, went further by allowing good video quality at higher bit rates, with higher resolutions, including HDTV, as well as an optimized mechanism to support interlaced video. (HDTV support was originally envisioned for an MPEG-3 standard; that effort was soon rolled into MPEG-2, when it was found that the HD resolutions could be easily supported therein.) MPEG-2 also introduced the concept of the transport stream, which allowed for simultaneous multiple program streams, each with its own synchronization information. The target then became transmission and broadcast systems, which are inherently one-way, and therefore need a fault-tolerant synchronization process. In addition, the adoption of MPEG-2 for DVDs provided the quality and convenience needed to quickly establish the medium as a popular consumer format.
The growing need for more bandwidth efficiency drove working groups to develop MPEG-4 in the late '90s. By adding the concepts of video objects and video object planes, MPEG-4 coding allows a foreground video object plane to be coded over a separately coded background image, in contrast to coding an entire picture. The technique also introduced methods for coding still textures and synthetic images. A still-growing application of MPEG-4 at low bit rates is for 3G/4G mobile-phone video.
While a target of lower bit rates initially drove the development of MPEG-4, new techniques developed into the early 2000s enabled the improved coding of video at higher bit rates as well. This led to the development of MPEG-4 Part 10, a.k.a. AVC, for Advanced Video Coding (equivalent to H.264), which enabled about a 50-percent improvement in coding efficiency for even HDTV video. AVC support is now required for all Blu-ray disc players and is suitable for use in various transmission systems, including extensions to DVB-T and ATSC, as well as for IPTV.
AVC provides a much-enhanced toolkit over its predecessors; some of the key features include picture-adaptive frame/field coding, intra estimation (within blocks), 4 x 4 transform blocks, lossless macroblock coding, macroblock adaptive field-frame coding, 4 x 4 (and other size) motion compensation blocks, quarter-pel motion vector precision, multiple-reference P- and B-frames, de-blocking filters, various entropy coding algorithms and Fidelity Range Extension. These features enable better video quality at lower bit rates and reduce the visibility of artifacts; different implementations can also offer lower delay, lower decoder processor load and better error resilience. Higher supported resolutions also include those for digital cinema.
MPEG-4 Part 10 offers various levels of scalability, using Scalable Video Coding (SVC), so that simpler receivers decode only the elements needed for that level of performance. SVC has been incorporated into the new ATSC M/H mobile TV standard, allowing for simultaneous coding of different resolutions and frame rates.
MPEG-4 also defined an advanced audio coding (AAC) standard (based on a likewise-named portion of an earlier MPEG-2 specification), which has since come to be used in the Apple iPod and various video game systems. The “high-efficiency” AAC codec also has been incorporated into various transmission systems, such as ATSC M/H and DVB-H.
Where is compression going?
The MPEG working groups continue to refine aspects of the MPEG-4 standard. In 2008, an extension to AVC was released, called Multiview Video Coding (MVC), which enables the simultaneous coding of different views of the same scene. This allows for a standard method of coding 3-D video, which has since been adopted for use on Blu-ray discs. MPEG-4/MVC compresses both left- and right-eye views and can provide full 1080p-resolution backward-compatibility with 2-D decoders. It works by exploiting the similarities between multiple-camera video captures of a scene. By eliminating redundant information across camera views, MVC achieves a reduction in bit rate of approximately 20 percent to 25 percent on average, when compared to encoding each view separately.
Work also has been started on the feasibility of High-performance Video Coding (HVC), which is mainly intended for high-quality applications. The goals of the project include performance improvements at higher resolutions, such as for home cinema and ultra-HDTV (U-HDTV); the benefits include lower noise level, extended color gamut and higher dynamic range. The work will generally involve the modification (extension and refinement) of existing coding tools.
What's next? While there is no “MPEG-5” in the works, we can expect incremental improvements to MPEG-4 Part-10 — perhaps a few more coding tools — that continue to offer better quality at lower bit rates; this could yield perhaps another 10 percent to 15 percent in efficiency. In addition, encoding techniques, while constrained to generating compatible compressed video, can improve the quality/bit rate efficiency through novel search techniques and distortion optimization. Expect about a 5-percent to 10-percent improvement. There is no limit to technological ingenuity, especially when bandwidth and cost are driving innovation.
Aldo Cugnini is a consultant in the digital television industry.
Send questions and comments to:aldo.cugnini@penton.com
