微精选

3.1.2-1 Motion Vector Coding.

Motion vectors (MVs) are predicted using a spatial neighborhood of three MVs and the resulting prediction error is variable length coded. Motion vectors are transmitted only for P-VOPs and B-VOPs. MPEG-4 Part 2 uses a variety of motion compensation techniques, such as the use of unrestricted MVs (motion vectors that are allowed to point outside the coded area of a reference VOP), and the use of four MVs per macroblock. In addition, version 2 of MPEG-4 Part 2 supports global motion compensation and quarter-sample motion vector accuracy.

3.1.2-2 Texture Coding.

3.1.2-3 Shape Coding.

Besides H.263+ [4], which provides some limited shape coding support via its chroma-keying coding technique, MPEG-4 Parts 2 and 10 are the only other video coding standards that support shape coding. As opposed to H.263+, MPEG-4 Part 2 adopted a bitmap-based shape coding method because of its ability to clearly distinguish between shape and texture information while retaining high compression performance and reasonable complexity. In bitmap-based shape coding, the shape and transparency of a VOP are defined by their binary or grayscale (respectively) alpha planes. A binary alpha plane indicates whether or not a sample belongs to a VOP. A gray-scale alpha plane indicates the transparency of each sample within a VOP. Transparency of samples can take values from 0 (transparent) to 255 (opaque). If all of the samples in a VOP block are indicated to be opaque or transparent, then no additional transparency information is transmitted for that block.

In inter shape coding using binary alpha planes, the shape of the current block is first predicted from the shape of the temporally previous VOP by performing motion estimation and compensation using integer sample accuracy. The shape motion vector is then coded predictively. Next, the difference between the current and the predicted shape block is arithmetically coded. The context for an inter coded shape block is computed using a template of nine samples from both the current and temporally previous VOP shape blocks, as shown in Fig. 6(b).

In both intra and inter shape coding, lossy coding of the binary shape is achieved by either not transmitting the difference between the current and the predicted shape block or subsampling the binary alpha plane by a factor of two or four prior to arithmetic encoding. To reduce the blocky appearance of the decoded shape caused by lossy coding, an upsampling filter is employed during the reconstruction.

3.1.2-4 Sprite Coding.

3.1.4-1 Face Animation.

It is required that every MPEG-4 Part 2 decoder that supports face object decoding have a default face model, which can be replaced by downloading a new face model. Either model can be customized to have a different visual appearance by transmitting facial definition parameters (FDPs). FDPs can determine the shape (i.e., head geometry) and texture of the face model.

The FAPs are coded by applying quantization followed by arithmetic coding. The quantization is performed by taking into consideration the limited movements of the facial features. Alternatively, DCT coding can be applied to a vector of 16 temporal instances of the FAP. This solution improves compression efficiency at the expense of a higher delay.

3.1.4-2 Body Animation.

Body animation was standardized in version 2 of MPEG-4 Part 2. Similar to the case of a face object, two sets of parameters are defined for a body object: body definition parameters (BDPs), which define the body through its dimensions, surface and texture, and body animation parameters (BAPs), which define the posture and animation of a given body model.

3.2.1-1 Slices and Slice Groups.

In contrast with previous standards in which the picture type (I, P, or B) determined the macroblock prediction modes available, it is the slice type in H.264/AVC that determines which prediction modes are available for the macroblocks. For example, I slices contain only intra predicted macroblocks, and P slices can contain inter predicted (motion-compensated) macroblocks in addition to the types allowed in I slices. Slices of different types can be mixed within a single picture.

3.2.1-2 Spatial Directional Intra Prediction.

The compression performance of a block-based video codec depends fundamentally on the effective use of prediction of sample blocks to minimize the residual prediction error. H.264/AVC provides a very powerful and flexible model for the prediction of each block of samples from previously encoded and reconstructed sample values. This includes spatial directional prediction within the same picture (intra prediction), and flexible multiple reference picture motion compensation from a set of previously reconstructed and stored pictures (inter prediction). As stated above, the prediction modes that are available are dependent upon the type of each slice (intra, predictive, or bipredictive, for I, P, and B slices, respectively).

The 16 × 16 intra prediction mode operates similarly, except that the entire luma macroblock is predicted at once, based on the samples above and to the left of the macroblock. Also, in this mode there are only 4 modes available for prediction: DC, vertical, horizontal and planar. The 16 × 16 intra mode is most useful in relatively smooth picture areas.

3.2.1-3 Enhanced Motion-Compensation Prediction Model.

Motion-compensated prediction (inter prediction) in H.264/AVC, similar to that in the previous standards, is primarily based on the translational block-based motion model, in which blocks of samples from previously reconstructed reference pictures are used to predict current blocks through transmission of motion vectors. However, the motion-compensation model defined in H.264/AVC [25, 26] is more powerful and flexible than those defined in earlier standards, and provides a much larger number of options in the search for block matches to minimize the residual error. The H.264/AVC motion model includes seven partition sizes for motion compensation, quarter-sample accurate motion vectors, a generalized multiple reference picture buffer, and weighted prediction. For an encoder to take full advantage of the larger number of prediction options, the prediction selection in H.264/AVC is more computationally intense than in some earlier standards that have simpler models, such as MPEG-2.

All motion vectors in H.264/AVC are transmitted with quarter-luma sample accuracy, providing improved opportunities for prediction compared to most previous standards, which allowed only half-sample accurate motion compensation. For motion vectors at fractional-sample locations, the sample values are interpolated from the reference picture samples at integer-sample locations. Luma component predictions at half-sample locations are interpolated using a 6-tap finite impulse response (FIR) filter. Predictions at quarter-sample luma locations are computed via a bilinear interpolation of the values for two neighboring integer- and half-sample locations. Chroma fractional-sample location values are computed by linear interpolation between integer-location values.

In P slices, each motion-compensated luma prediction block is derived by quarter-sample interpolation of a prediction from a single block within the set of reference pictures. In B slices, an additional option is provided, allowing each prediction block to be derived from a pair of locations within the set of reference pictures. By default, the final prediction in a bipredictive block is computed using a sample-wise ordinary average of the two interpolated reference blocks. However, H.264/AVC also includes a weighted prediction feature, in which each prediction block can be assigned a relative weighting value rather than using an ordinary average value, and an offset can be added to the prediction. These parameters can either be transmitted explicitly by the encoder, or computed implicitly, based on a time-related relationship between the current picture and the reference picture(s). Weighted prediction can provide substantial coding gains in scenes containing fades or cross fades, which are very difficult to code efficiently with previous video standards that do not include this feature.

3.2.1-4 In-Loop Deblocking Filter.

Block-based prediction and transform coding, including quantization of transform coefficients, can lead to visible and subjectively objectionable changes in intensity at coded block boundaries, referred to as blocking artifacts. In previous video codecs, such as MPEG-2, the visibility of these artifacts could optionally be reduced to improve subjective quality by applying a deblocking filter after decoding and prior to display. The goal of such a filter is to reduce the visibility of subjectively annoying artifacts while avoiding excessive smoothing that would result in loss of detail in the image. However, because the process was optional for decoders and was not normatively specified in these standards, the filtering was required to take place outside of the motion-compensation loop, and large blocking artifacts would still exist in reference pictures that would be used for predicting subsequent pictures. This reduced the effectiveness of the motion-compensation process and allowed blocking artifacts to be propagated into the interior of subsequent motion-compensated blocks, causing the subsequent picture predictions to be less effective and making the removal of the artifacts by filtering more challenging.

3.2.1-5 Transform, Quantization, and Scanning.

As in earlier standards, H.264/AVC uses block transform coding to efficiently represent the prediction residual signal. However, unlike previous standards, the H.264/AVC transform is primarily based on blocks of 4 × 4 samples, instead of 8 × 8 blocks. Furthermore, instead of performing an approximation of the floating-point DCT (i.e., by specifying that each implementation must choose its own approximation of the ideal IDCT equations), the transform employed has properties similar to that of the 4 × 4 DCT, but it is completely specified in integer operations. This eliminates the problem of mismatch between the inverse transforms performed in the encoder and decoder, enabling an exact specification of the decoded sample values and significantly reducing the complexity of the IDCT computations. In particular, the IDCT can be computed easily using only 16-bit fixed-point computations (including intermediate values), which would have been very difficult for the IDCT defined in all previous standards. Additionally, the basis functions of the transform are extended by using additional second-stage transforms that are applied to the 2 × 2 chroma DC coefficients of a macroblock, and the 4 × 4 luma DC coefficients of a macroblock predicted in the 16 × 16 intra mode.

Scalar quantization is applied to the transform coefficients [27], but without as wide a dead-zone as used in all other video standards. The 52 quantization parameter values are designed so that the quantization step size increases by approximately 12.2% for each increment of one in the quantization parameter (such that the step size exactly doubles if the quantization parameter is increased by 6). As in earlier standards, each block of quantized coefficients is zig-zag-scanned for ordering the quantized coefficient representations in the entropy coding process. The decoder performs an approximate inversion of the quantization process by multiplying the quantized coefficient values by the step size that was used in their quantization. This inverse quantization process is referred to as scaling in the H.264/AVC standard, since it consists essentially of just a multiplication by a scale factor. In slice types other than SP/SI, the decoder then performs an inverse transform of the scaled coefficients and adds this approximate residual block to the spatial-domain prediction block to form the final picture reconstruction prior to the deblocking filter process.

In SP and SI synchronization slices, the decoder performs an additional forward transform on the prediction block and then quantizes the transform-domain sum of the prediction and the scaled residual coefficients. This additional forward transform and quantization operation can be used by the encoder to discard any differences between the details of the prediction values obtained when using somewhat different reference pictures in the prediction process (for example, differences arising from the reference pictures being from an encoding of the same video content at a different bit rate). The reconstructed result in this case is then formed by an inverse transform of the quantized transform-domain sum, rather than the ordinary case where the reconstructed result is the sum of a spatial-domain prediction and a spatial-domain scaled residual.

3.2.1-6 Entropy Coding.

Two methods of entropy coding are specified in H.264/AVC. The simpler variable-length coding method is supported in all profiles. In both cases, many syntax elements except for the quantized transform coefficients are coded using a regular, infinite-extent variable-length codeword set. Here, the same set of Exp-Golomb codewords is used for each syntax element, but the mapping of codewords to decoded values is selected to fit the syntax element.

In the simpler entropy coding method, scans of quantized coefficients are coded using a context-adaptive variable-length coding (CAVLC) scheme. In this method, one of a number of variable-length coding tables is selected for each symbol, depending on the contextual information. This includes statistics from previously coded neighboring blocks, as well as statistics from previously coded data for the current block.

For improved coding efficiency, a more complex context-adaptive binary arithmetic coding (CABAC) method can be used [28]. When CABAC is in use, scans of transform coefficients and other syntax elements of the macroblock level and below (such as reference picture indices and motion vector values) are encoded differently. In this method, each symbol is binarized (i.e., converted to a binary code), and then the value of each bin (bit of the binary code) is arithmetically coded. To adapt the coding to non-stationary symbol statistics, context modeling is used to select one of several probability models for each bin, based on the statistics of previously coded symbols. The use of arithmetic coding allows for a non-integer number of bits to be used to code each symbol, which is highly beneficial in the case of very skewed probability distributions. Probability models are updated following the coding of each bit, and this high degree of adaptivity improves the coding efficiency even more than the ability to use a fractional number of bits for each symbol. Due to its bit-serial nature, the CABAC method entails greater computational complexity than the VLC-based method.

3.2.1-7 Frame/Field Adaptive Coding.

H.264/AVC includes tools for efficiently handling the special properties of interlaced video, since the two fields that compose an interlaced frame are captured at different instances of time. In areas of high motion, this can lead to less statistical correlation between adjacent rows within the frame, and greater correlation within individual fields, making field coding (where lines from only a single field compose a macroblock) a more efficient option. In addition to regular frame coding in which lines from both fields are included in each macro-block, H.264/AVC provides two options for special handling of interlaced video: field picture coding and macroblock-adaptive field/frame coding (MB-AFF).

Field coding provides the option of coding pictures containing lines from only a single video field (i.e., a picture composed of only top field lines or only bottom field lines). Depending on the characteristics of the frame, an encoder can choose to code each input picture as two field pictures, or as a complete frame picture. In the case of field coding, an alternate zig-zag coefficient scan pattern is used, and individual reference fields are selected for motion compensated prediction. Thus, frame/field adaptivity can occur at the picture level. Field coding is most effective when there is significant motion throughout the video scene for interlaced input video, as in the case of camera panning.