MPEG-4 AVC offers many tools for coding video and higher coding efficiency than MPEG-2. The growing number of AVC applications is facilitated by a broad set of profiles, which group together different feature sets. A recent extension to AVC is Multiview Video Coding (MVC), which offers content producers the ability to code different views of the same production, directly supporting 3-D presentation. First, let's look at some of the various profiles.
Profiles
MPEG profiles are defined so that a higher profile is a superset of a lower one. (See Table 1.) Within these profiles, levels are defined that describe a set of constraints on decoder performance (processing power and the memory size), essentially limiting the maximum pixel rate. A decoder that supports a particular profile and level is only required to support the corresponding subset of the full syntax and a set of parameter constraints.
It is important to realize that MPEG coding was developed to enable a wide range of applications, supporting both real-time (streaming) and non-real-time (storage) applications. The use of the different profiles essentially concentrated on videoconferencing, storage and broadcast. The Constrained Baseline Profile and Baseline Profile were developed for low-cost applications; videoconferencing and mobile uses gravitated towards the former, being cheaper to implement and not needing the additional error resilience tools of the latter. Both profiles specify that every coded picture of the coded video sequence is a coded frame containing only frame macroblocks, i.e., coded pictures of the coded video sequence may not be coded as fields.
The Main Profile adds support for B-frames (bidirectionally predictive), which were left out of the simpler profiles to lower both memory requirements and computational complexity. The Extended Profile adds some coding tools, including support for up to eight slice groups per picture. (Pixels are arranged in groups called blocks, and these are further arranged into macroblocks.) Macroblocks of a picture can be mapped into slice groups, which are further partitioned into slices, adding compression efficiency.
The High Profile improves compression quality (up to 12 percent for progressive HD video and film sources) while adding almost no computational complexity (but a slight memory increase) over Main Profile. The profile adds switchable transform and scaling matrices to achieve the improvement. Devices such as the Sony PlayStation and all current Blu-ray disc players support the High Profile.
The “derivative” High Profiles add additional tools: High 10 adds 10 bits/channel coding, High 4:2:2 adds 4:2:2 chroma sampling, and High 4:4:4 Predictive adds 4:4:4 chroma at 12 bits/channel. An earlier High 4:4:4 Profile was later removed.
3-D
Stereoscopic video is receiving growing interest by consumers, CE manufacturers and broadcasters, and existing MPEG codecs can handle certain 3-D formats, but not necessarily in a backwards-compatible way. Frame-compatible 3-D can be transmitted over either MPEG-2 or AVC, because the left and right views are vertically decimated by a factor of two and arranged into formats such as side-by-side or top-and-bottom. Because these frame-compatible formats fit into a conventional 30Hz frame period, no additional baseband bandwidth is needed, and conventional baseband video equipment can handle the signals. Frame-compatible video can thus be encoded using a conventional MPEG-2 or AVC codec, but cannot be displayed on a 2-D display, which cannot separate or properly integrate the left and right views.
MVC
Newer specifications, however, can deal with these issues and provide bandwidth savings, as well. Multiview Video Coding (MVC) was developed by MPEG to support multiple simultaneous views of a subject, and in 2008, an MVC extension to AVC was released. In general, an MVC encoder receives n temporally synchronized video streams and generates one output bit stream. The decoder receives the bit stream, and decodes and outputs the n video signals.
MVC works by exploiting the similarities between multiple-camera video captures of a scene. By eliminating redundant information across camera views, MVC achieves a reduction in bit rate of about 20 to 25 percent on average when compared to encoding each view separately. This is accomplished by encoding a full-resolution 2-D view and the difference information between the left and the right views. This difference is coded in the video stream in a format that updated decoders and 3-D displays can play back in any 3-D format at the highest quality possible, while legacy 2-D televisions and decoders play the stream in 2-D. The 2-D view can be decoded from the base-layer AVC stream by using a High Profile decoder, which could then be output on a conventional 2-D display.
An MVC decoder would generate the stereo views from the base layer and enhancement layers, and deliver them to a stereoscopic display. In this way, MVC is fully backward-compatible and display-independent. Of course, encoding 3-D in a fashion that is intended to yield a compatible 2-D representation will not always be the production intent; the left or right view is not always intended to be viewed alone.
The Multiview High Profile, using the same coding tools as supported by the High Profile, has been specified so that fixed decoder resources of single-view AVC decoders, such as memory, can be repurposed for decoding stereo and multiview video bit streams. The Stereo High Profile was developed to support coding left- and right-eye stereo views as the multiple views of MVC. Also added to AVC was Frame Packing Arrangement Supplemental Enhancement Information (SEI) messaging, which signals the decoder that the left- and right-eye stereo views are packed into a single high-resolution video frame either in a top-and-bottom, side-by-side, checkerboard or other arrangement.
The Blu-ray 3-D specification calls for encoding 3-D video using the MVC codec. (AVC is currently supported by all Blu-ray Disc players.) The Multiview High Profile as defined in the MPEG-MVC Amendment uses the same coding tools as supported by the earlier High Profile of the MPEG-4 AVC standard.
Intra
As an aid to editing and content retrieval, several of the advanced profiles include intra versions, in which there are no predictive frames, i.e., every frame is intra coded. Although this carries a bandwidth premium, it generates streams (and files) where each frame is an intact element that does not depend on any other frame for its reconstruction. The CAVLC 4:4:4 Intra and High 4:4:4 Intra Profile provide the highest capability in the production environment, with the latter supporting both CABAC and CAVLC stream coding. CABAC compresses data more efficiently than CAVLC but requires considerably more processing power to encode and decode.
AVC SVC Intra Constrained baseline profile Baseline profile Scalable baseline profile Main profile Extended profile High profile Scalable high profile Scalable high Intra profile High 10 profile High 10 Intra profile High 4:2:2 profile High 4:2:2 Intra profile High 4:4:4 predictive profile High 4:4:4 Intra profile CAVLC 4:4:4 Intra profile Stereo high profile Multiview high profile
Table 1. Various MPEG profiles
Aldo Cugnini is a consultant in the digital television industry.
Send questions and comments to: aldo.cugnini@penton.com
