MC2000 - Chapter 8 - Natural Video Coding

This chapter focuses on details of coding of arbitrary-shaped video objects and related issues.

MPEG4 Video coding

An input video sequence consists of pictures separated in time. Each picture represent instance of objects that undergo changes such as:
- translation
- rotation
- scaling
- brightness variations
- color variations

New objects may come, other may go, or a whole scene might need to be changed.

Video Object Planes and Video Objects

A video Object Plane can be considered as a temporal instance of video objects.

VOPs may be describe using texture variation (set of luminance and chrominance values)

In natural scenes, VOPs are obtained by segmentation.

Video object may be represented as Video Objects Layers for scalable representation.

A picture is therefore separated in many VOPs containing video objects.

The Video Objects are coded separately and multiplexed to form a manipulable  bitstream.

MPEG-4 extends the concept of I, P and B frames to I-VOP, P-VOP and B-VOP.

The basic MPEG-4 coding employs motion compensation and DCT-based coding.

Intra coding is improved in MPEG-4 to include
- Prediction of the DC coefficient
- Prediction of a subset of AC coefficient
- Specialized coefficient scanning based on the coefficient prediction
- Variable length coding (VLC) table selection
- Non-linear inverse DC quantization
- Downloadable quantization matrices.

P-VOP coding
Coded using motion-compensated block matching technique to determine prediction error. Blocks being of arbitrary sizes, this involves use of padding. It also supports an overlapped block motion compensation, where each pixel in a luminence block of the macroblock is computed a as weighted sum of 3 prediction values using 3 motion vectors.

B-VOP coding
Four modes are used:
- Forward/backward/bidirectional modes,one single motion vector is used
- Direct mode,uses motion vectors on 8x8 blocks with bidirectional motion compensation

MPEG-4 is compared to other standards in various tables. Obviously beats the cr*p out of them regarding bit rate and SNR...

Interlaced video coding

MPEG-4 provides tool to do interlaced encoding, dvs encode half of the scan lines at the time while still exploiting spatial and temporal redundancy. Also:
- Adaptive field/frame DCT - reorder lines in 8x8 luminance blocks
- Alternate and Adaptive Scans of DCT coefficient - alternate vertical scan
- Adaptive Field/Frame Prediction
- Motion vector coding - differential coding of motion vector using spatial neighborhood of motion vectors already transmitted.

Interlaced Direct Mode Coding in B-VOPs
This is an algorithm to construct forward and backward motion vectors for B-VOP MB.

Field-based Padding
Using padding techniques to adjust object boundaries allow for more efficient coding.

Performance of interlaced tools
Again - diagrams to show that MPEG4 has a lower average bit per VOP is shown.

Error resilience
4 tools have been incorporated to make MPEG-4 video more error resilient
- Video packet resynchronization (resynch after loss of marker)
- Data partioning (DP) (use of the erroneous part by guessing where the error is)
- Header extension code (HEC) (allows repeting important header information - similar to FEC include)
- Reversible variable length codes (RVLCs) (enables decoder to better isolate the erroneous part)

Scalable video coding
Property that allow a decoder to decoder a portion of the stream only and achieve quality corresponding to the amount of data used. Allows to adjust on both processing power, terminal capacities and available bandwidth.
Temporal scalability and spatial scalability are used.