6. Decoding Process
This section preserves the RFC text for APV, including bitstream syntax, syntax element processing, decoding and parsing processes, metadata, profiles, levels, bands, raw bitstream format, and implementation references.
Original RFC Text
6. Decoding Process
This process is invoked to obtain a decoded frame from a bitstream.
Input to this process is a bitstream of a coded frame. Output of
this process is a decoded frame.
The decoding process operates as follows for the current frame:
* The syntax structure for a coded frame is parsed to obtain the
parsed syntax structures.
* The processes in Sections 6.1, 6.2, and 6.3 specify the decoding
processes using syntax elements in all syntax structures. For
bitstreams conforming to this document, the coded tiles of the
frame MUST contain tile data for every MB of the frame, such that
the division of the frame into tiles and the division of the tiles
into MBs form a partitioning of the frame.
* After all the tiles in the current frame have been decoded, the
decoded frame is cropped using the cropping rectangle if
FrameWidthInSamplesY is not equal to FrameWidthInMbsY * MbWidth or
FrameHeightInSamplesY is not equal to FrameHeightInMbsY *
MbHeight.
* The cropping rectangle, which specifies the samples of a frame
that are output, is derived as follows:
- The cropping rectangle contains the luma samples with
horizontal frame coordinates from 0 to FrameWidthInSampleY - 1
and vertical frame coordinates from 0 to FrameHeightInSamplesY
- 1, inclusive.
- The cropping rectangle contains the two chroma arrays having
frame coordinates (x//SubWidthC, y//SubHeightC), where (x,y)
are the frame coordinates of the specified luma samples.
6.1. MB Decoding Process
This process is invoked for each MB.
Input to this process is a luma location (xMb, yMb) specifying the
top-left sample of the current luma MB relative to the top-left luma
sample of the current frame. Outputs of this process are the
reconstructed samples of all color components. The total number of
color components is indicated by the value of NumComps for the
current MB. For example, when chroma_format_idc is equal to 2 or 3,
the value of NumComps is equal to 3 and three components, Y
component, Cb component, and Cr component, are reconstructed
The following steps apply:
* Let recSamples[0] be a (MbWidth)x(MbHeight) array of the
reconstructed samples of the first color component (when
chroma_format_idc is equal to 2 or 3, Y).
* The block reconstruction process as specified in Section 6.2 is
invoked with the luma location (xMb, yMb), the variable nBlkW set
equal to MbWidth, the variable nBlkH set equal to MbHeight, the
variable cIdx set equal to 0, and the (MbWidth)x(MbHeight) array
recSamples[0] as inputs. The output is a modified version of the
(MbWidth)x(MbHeight) array recSamples[0], which is the
reconstructed samples of the first color component for the current
MB.
* When chroma_format_idc is not equal to 0, let recSamples[1] be a
(MbWidthC)x(MbHeightC) array of the reconstructed samples of the
second color component. For example, when chroma_format_idc is
equal to 2 or 3, recSamples[1] is the Cb color component.
* When chroma_format_idc is not equal to 0, the block reconstruction
process as specified in Section 6.2 is invoked with the luma
location (xMb, yMb), the variable nBlkW set equal to MbWidthC, the
variable nBlkH set equal to MbHeightC, the variable cIdx set equal
to 1, and the (MbWidthC)x(MbHeightC) array recSamples[1] as
inputs. The output is a modified version of the
(MbWidthC)x(MbHeightC) array recSamples[1], which is the
reconstructed samples of the second color component for the
current MB.
* When chroma_format_idc is not equal to 0, let recSamples[2] be a
(MbWidthC)x(MbHeightC) array of the reconstructed samples of the
third color component. For example, when chroma_format_idc is
equal to 2 or 3, recSamples[2] is the Cr color component.
* When chroma_format_idc is not equal to 0, the block reconstruction
process as specified in Section 6.2 is invoked with the luma
location (xMb, yMb), the variable nBlkW set equal to MbWidthC, the
variable nBlkH set equal to MbHeightC, the variable cIdx set equal
to 2, and the (MbWidthC)x(MbHeightC) array recSamples[2] as
inputs. The output is a modified version of the
(MbWidthC)x(MbHeightC) array recSamples[2], which is the
reconstructed samples of the third color component for the current
MB.
* When chroma_format_idc is equal to 4, let recSamples[3] be a
(MbWidthC)x(MbHeightC) array of the reconstructed samples of the
fourth color component.
* When chroma_format_idc is equal to 4, the block reconstruction
process as specified in Section 6.2 is invoked with the luma
location (xMb, yMb), the variable nBlkW set equal to MbWidthC, the
variable nBlkH set equal to MbHeightC, the variable cIdx set equal
to 3, and the (MbWidthC)x(MbHeightC) array recSamples[3] as
inputs. The output is a modified version of the
(MbWidthC)x(MbHeightC) array recSamples[3], which is the
reconstructed samples of the fourth color component for the
current MB.
6.2. Block Reconstruction Process
Inputs to this process are:
* a luma location (xMb, yMb) specifying the top-left sample of the
current MB relative to the top-left luma sample of the current
frame,
* two variables nBlkW and nBlkH specifying the width and the height
of the current block,
* a variable cIdx specifying the color component of the current
block, and
* an (nBlkW)x(nBlkH) array of recSamples of a reconstructed block.
Output of this process is a modified version of the (nBlkW)x(nBlkH)
array recSamples of reconstructed samples.
The following applies:
* The variables numBlkX and numBlkY are derived as follows:
- numBlkX = nBlkW // TrSize
- numBlkY = nBlkH // TrSize
* For yIdx = 0..numBlkY - 1, the following applies:
- For xIdx = 0..numBlkX - 1, the following applies:
o The variables xBlk and yBlk are derived as follows:
+ xBlk = xMb // (cIdx==0? 1: SubWidthC) + xIdx*TrSize
+ yBlk = yMb // (cIdx==0? 1: SubHeightC) + yIdx*TrSize
o The scaling and transformation process as specified in
Section 6.3 is invoked with the location (xBlk, yBlk), the
variable cIdx set equal to cIdx, the transform width nBlkW
set equal to TrSize, and the transform height nBlkH set
equal to TrSize as inputs. The output is a
(TrSize)x(TrSize) array r of a reconstructed block.
o The (TrSize)x(TrSize) array recSamples is modified as
follows:
+ recSamples[(xIdx * TrSize) + i, (yIdx * TrSize) + j] =
r[i,j], with i=0..TrSize-1, j=0..TrSize-1
6.3. Scaling and Transformation Process
Inputs to this process are:
* a location (xBlkY, yBlkY) of the current color component
specifying the top-left sample of the current block relative to
the top-left sample of the current frame,
* a variable cIdx specifying the color component of the current
block,
* a variable nBlkW specifying the width of the current block, and
* a variable nBlkH specifying the height of the current block.
Output of this process is the (nBlkW)x(nBlkH) array of reconstructed
samples r with elements r[x][y].
The quantization parameter qP is derived as follows:
* qP = Qp[cIdx] + QpBdOffset
The (nBlKW)x(nBlkH) array of reconstructed samples r is derived as
follows:
* The scaling process for transform coefficients as specified in
Section 6.3.1 is invoked with the block location (xBlkY, yBlkY),
the block width nBlkW and the block height nBlkH, the color
component variable cIdx, and the quantization parameter qP as
inputs. The output is an (nBlkW)x(nBlkH) array of scaled
transform coefficients d.
* The transformation process for scaled transform coefficients as
specified in Section 6.3.2 is invoked with the block location
(xBlkY, yBlkY), the block width nBlkW and the block height nBlkH,
the color component variable cIdx, and the (nBlkW)x(nBlkH) array
of scaled transform coefficients d as inputs. The output is an
(nBlkW)x(nBlkH) array of reconstructed samples r.
* The variable bdShift is derived as follows:
- bdShift = 20 - BitDepth
* The reconstructed sample values r[x][y] with x = 0..nBlkW - 1, y =
0..nBlkH - 1 are modified as follows:
- r[x][y] = clip(0, (1 << BitDepth)-1, ((r[x][y]+(1 << (bdShift-
1)))>>bdShift) + (1 << (BitDepth-1)))
6.3.1. Scaling Process for Transform Coefficients
Inputs to this process are:
* a location (xBlkY, yBlkY) of the current color component
specifying the top-left sample of the current block relative to
the top-left sample of the current frame,
* a variable nBlkW specifying the width of the current block,
* a variable nBlkH specifying the height of the current block,
* a variable cIdx specifying the color component of the current
block, and
* a variable qP specifying the quantization parameter.
Output of this process is the (nBlkW)x(nBlkH) array d of scaled
transform coefficients with elements d[x][y].
The variable bdShift is derived as follows:
* bdShift = BitDepth + ((log2(nBlkW) + log2(nBlkH)) // 2) - 5
The list levelScale[] is specified as follows:
* levelScale[k] = {40, 45, 51, 57, 64, 71} with k = 0..5.
For the derivation of the scaled transform coefficients d[x][y] with
x = 0..nBlkW - 1, y = 0..nBlkH - 1, the following applies:
* The scaled transform coefficient d[x][y] is derived as follows:
- d[x][y] = clip(-32768, 32767, ((TransCoeff[cIdx][xBlkY][yBlkY]
* QMatrix[cIdx][x][y] * levelScale[qP % 6] << (qP//6)) + (1 <<
(bdShift-1)) >> bdShift))
6.3.2. Process for Scaled Transform Coefficients
6.3.2.1. General
Inputs to this process are:
* a location (xBlkY, yBlkY) of the current color component
specifying the top-left sample of the current block relative to
the top-left sample of the current frame,
* a variable nBlkW specifying the width of the current block,
* a variable nBlkH specifying the height of the current block, and
* an (nBlkW)x(nBlkH) array d of scaled transform coefficients with
elements d[x][y].
Output of this process is the (nBlkW)x(nBlkH) array r of
reconstructed samples with elements r[x][y].
The (nBlkW)x(nBlkH) array r of reconstructed samples is derived as
follows:
* Each (vertical) column of scaled transform coefficients d[x][y]
with x = 0..nBlkW - 1, y = 0..nBlkH - 1 is transformed to e[x][y]
with x = 0..nBlkW - 1, y = 0..nBlkH - 1 by invoking the one-
dimensional transformation process as specified in Section 6.3.2.2
for each column x = 0..nBlkW - 1 with the size of the transform
block nBlkH, and the list d[x][y] with y = 0..nBlkH - 1 as inputs.
The output is the list e[x][y] with y = 0..nBlkH - 1.
* The following applies:
- g[x][y] = (e[x][y] + 64) >> 7
* Each (horizontal) row of the resulting array g[x][y] with x =
0..nBlkW - 1, y = 0..nBlkH - 1 is transformed to r[x][y] with x =
0..nBlkW - 1, y = 0..nBlkH - 1 by invoking the one-dimensional
transformation process as specified in Section 6.3.2.2 for each
row y = 0..nBlkH - 1 with the size of the transform block nBlkW,
and the list g[x][y] with x = 0..nBlkW - 1 as inputs. The output
is the list r[x][y] with x = 0..nBlkW - 1.
6.3.2.2. Transformation Process
Inputs to this process are:
* a variable nTbS specifying the sample size of scaled transform
coefficients, and
* a list of scaled transform coefficients x with elements x[j], with
j = 0..(nTbS - 1).
Output of this process is the list of transformed samples y with
elements y[i], with i = 0..(nTbS - 1).
The transformation matrix derivation process as specified in
Section 6.3.2.3 is invoked with the transform size nTbS as input, and
the transformation matrix transMatrix as output.
The list of transformed samples y[i] with i = 0..(nTbS - 1) is
derived as follows:
* y[i] = sum(j = 0, nTbS - 1, transMatrix[i][j] * x[j])
6.3.2.3. Transformation Matrix Derivation Process
Input to this process is a variable nTbS specifying the horizontal
sample size of scaled transform coefficients.
Output of this process is the transformation matrix transMatrix.
The transformation matrix transMatrix is derived based on nTbs as
follows:
* If nTbS is equal to 8, the following applies:
transMatrix[m][n] =
{
{ 64, 64, 64, 64, 64, 64, 64, 64 }
{ 89, 75, 50, 18, -18, -50, -75, -89 }
{ 84, 35, -35, -84, -84, -35, 35, 84 }
{ 75, -18, -89, -50, 50, 89, 18, -75 }
{ 64, -64, -64, 64, 64, -64, -64, 64 }
{ 50, -89, 18, 75, -75, -18, 89, -50 }
{ 35, -84, 84, -35, -35, 84, -84, 35 }
{ 18, -50, 75, -89, 89, -75, 50, -18 }
}
Figure 25: Transform matrix for nTbS == 8