WO2026050279A1 - Block-level control flag coding in cross-component sample offset - Google Patents

Abstract

A probability distribution is selected for coding a flag indicating whether a filter is applied to a filter processing unit of a reconstructed frame, wherein the probability distribution is selected based on whether the filter is applied to neighboring filter processing units of the filter processing unit. The flag indicating whether the filter is applied to the filter processing unit is entropy coded. The filter is selectively applied to the filter processing unit based on the flag. The filter can be a Cross-Component Sample Offset (CCSO) filter. The neighboring filter processing units can be identified based on a predefined search order of sub-blocks within the neighboring filter processing units.

Description

Atty. Doc. No. GOGL-2267-A-WO PATENT

BLOCK-LEVEL CONTROL FLAG CODING IN CROSS-COMPONENT SAMPLE

OFFSET

CROSS REFERENCE TO RELATED APPLICATION

[0001] This application claims priority to and the benefit of U.S. Provisional Patent Application Serial No. 63/687,681, filed August 27, 2024, the entire disclosure of which is incorporated herein by reference.

BACKGROUND

[0002] Digital video streams may represent video using a sequence of frames or still images. Digital video can be used for various applications including, for example, video conferencing, high definition video entertainment, video advertisements, or sharing of usergenerated videos. A digital video stream can contain a large amount of data and consume a significant amount of computing or communication resources of a computing device for processing, transmission, or storage of the video data. Various approaches have been proposed to reduce the amount of data in video streams, including encoding or decoding techniques.

SUMMARY

[0003] One aspect of the disclosed implementations relates to a method that includes selecting a probability distribution for coding a flag indicating whether a filter is applied to a filter processing unit of a reconstructed frame, where the probability distribution is selected based on whether the filter is applied to neighboring filter processing units of the filter processing unit; entropy coding the flag indicating whether the filter is applied to the filter processing unit; and selectively applying the filter to the filter processing unit based on the flag. The filter can be a Cross -Component Sample Offset (CCSO) filter.

[0004] In some implementations, the method includes identifying the neighboring filter processing units based on a predefined search order of sub-blocks within the neighboring filter processing units.

[0005] In some implementations, identifying the neighboring filter processing units based on the predefined search order includes: identifying a first sub-block of a first neighboring filter processing unit; and identifying a second sub-block of a second neighboring filter processing unit. [0006] In some implementations, selecting the probability distribution for coding the flag includes: selecting a first probability distribution in a case that the first neighboring filter processing unit and the second neighboring filter processing unit exist, the first neighboring filter processing unit is different from the second neighboring filter processing unit, the filter is applied to the first neighboring filter processing unit, and the filter is applied to the second neighboring filter processing unit.

[0007] In some implementations, selecting the probability distribution for coding the flag further includes, at least one of: selecting a second probability distribution in a case that the first neighboring filter processing unit and the second neighboring filter processing unit exist, the first neighboring filter processing unit and the second neighboring filter processing unit are a same neighboring filter processing unit, and the filter is applied to the first neighboring filter processing unit; or selecting a second probability distribution in a case that only one of the first neighboring filter processing unit or the second neighboring filter processing unit exists and the filter is applied to the neighboring filter processing unit that exists.

[0008] In some implementations, selecting the probability distribution for coding the flag further includes: selecting a third probability distribution in a case that the first neighboring filter processing unit and the second neighboring filter processing unit exist, the first neighboring filter processing unit is different from the second neighboring filter processing unit, and the filter is applied to only one of the first neighboring filter processing unit or the second neighboring filter processing unit.

[0009] In some implementations, selecting the probability distribution for coding the flag further includes: selecting a fourth probability distribution in a first case where neither the first neighboring filter processing unit nor the second neighboring filter processing unit exists, or in a second case where neither the first neighboring filter processing unit nor the second neighboring filter processing unit has the filter applied.

[0010] In some implementations, the method includes: storing the coded flag for use in filtering subsequent filter processing units within a same frame as the filter processing unit. [0011] In some implementations, selecting the probability distribution includes selecting from a plurality of probability distributions based on states of the neighboring filter processing units.

[0012] In some implementations, wherein the plurality of probability distributions includes a first probability distribution when both neighboring filter processing units exist, are different units, and both have the filter applied; a second probability distribution when both neighboring filter processing units are a same unit and have the filter applied, or when only one neighboring unit exists and has the filter applied; a third probability distribution when both neighboring filter processing units exist, are different units, and only one has the filter applied; and a fourth probability distribution when neither neighboring unit exists or neither has the filter applied.

[0013] Another aspect of the disclosed implementations relates to a device that includes a processor that is configured to perform the method.

[0014] Another aspect of the disclosed implementations relates to a device that includes a memory and a processor. The processor can be configured to execute instructions stored in the memory to perform the method.

[0015] Another aspect of the disclosed implementations relates to a non-transitory computer-readable storage medium that includes executable instructions that, when executed by a processor, cause the performance of the method.

[0016] Another aspect of the disclosed implementations relates to a non-transitory computer-readable storage medium having stored thereon an encoded bitstream that is configured for decoding by the method.

[0017] Another aspect of the disclosed implementations relates to a non-transitory computer-readable storage medium having stored thereon an encoded bitstream that is generated by an encoder performing the method.

[0018] These and other aspects of the present disclosure are disclosed in the following detailed description of the embodiments, the appended claims and the accompanying figures. It will be appreciated that aspects can be implemented in any convenient form. For example, aspects may be implemented by appropriate computer programs which may be carried on appropriate carrier media which may be tangible carrier media (e.g. disks) or intangible carrier media (e.g. communications signals). Aspects may also be implemented using suitable apparatus which may take the form of programmable computers running computer programs arranged to implement the methods and/or techniques disclosed herein. Aspects can be combined such that features described in the context of one aspect may be implemented in another aspect.

BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The description herein makes reference to the accompanying drawings described below, wherein like reference numerals refer to like parts throughout the several views. [0020] FIG. 1 is a schematic of a video encoding and decoding system.

[0021] FIG. 2 is a block diagram of an example of a computing device that can implement a transmitting station or a receiving station. [0022] FIG. 3 is a diagram of a typical video stream to be encoded and subsequently decoded.

[0023] FIG. 4 is a block diagram of an encoder according to implementations of this disclosure.

[0024] FIG. 5 is a block diagram of a decoder according to implementations of this disclosure.

[0025] FIG. 6 is an illustration of examples of portions of a video frame.

[0026] FIG. 7 is a block diagram of an example of a video frame filtering stage.

[0027] FIG. 8 is a block diagram that illustrates the operations of a Cross -Component

Sample Offset (CCSO) filter.

[0028] FIG. 9 illustrates syntax elements that may be used to signal aspects of the CCSO filter.

[0029] FIGS. 10A and 10B provide an implementation of a function for identifying probability distributions for entropy coding block-level syntax elements described with respect to FIG. 9.

[0030] FIG. 11 illustrates neighboring CCSO units of a CCSO unit used as context in deriving probability distributions for entropy coding the CCSO control flags.

[0031] FIG. 12 is a flowchart of a technique for entropy coding block-level control flags associated with CCSO.

DETAILED DESCRIPTION

[0032] Video compression schemes may include breaking respective images, or frames, of a video stream into smaller portions, such as blocks, or coding tree units (CTUs), and generating an encoded bitstream using techniques to limit the information included for respective CTUs thereof. The bitstream can be decoded to recreate the source frames from the limited information. Encoding CTUs to or decoding CTUs from a bitstream can include predicting the values of pixels or CTUs based on similarities with other pixels or CTUs in the same frame which have already been coded. Those similarities can be determined using intra prediction, which attempts to predict the pixel values of a coding unit (CU) of a CTU using pixels peripheral to the CU (e.g., pixels that are in the same frame as the CU, but which are outside the CU). During encoding, the result of an intra-prediction mode performed against a CU is a prediction unit (PU). A prediction residual can be determined based on a difference between the pixel values of the CU and the pixel values of the PU. The prediction residual and the intra prediction mode used to ultimately obtain that prediction residual can then be encoded to a bitstream. During decoding, the prediction residual is reconstructed into a CU using a PU produced based on the intra prediction mode and is thereafter included in an output video stream.

[0033] A CU includes a luminance, also referred to as luma, component and two chrominance, also referred to as chroma, components. These luma and chroma components may in some cases be referred to as a luma block and chroma blocks. The luma component of a CU may, for example, be expressed within a Y component (also referred to as “plane”) of the CU and the chroma components may be expressed either within U and V components or Cr and Cb components of the CU. The luma component is understood to include some number of luma samples and each chroma component is understood to include some number of chroma samples. Generally, the luma samples provide measures of brightness throughout a subject CU and thus represent the structural qualities of the video content of the subject CU, whereas the chroma samples provide measures of color throughout the subject CU. Because of this, conventional video compression schemes often use finer prediction approaches for predicting luma components of CUs than chroma components thereof. Such schemes may also use approaches directed to predicting those chroma components from the predicted luma components.

[0034] The process of video compression and decompression can introduce artifacts and distortions in the reconstructed video. To mitigate these issues, various filtering techniques are often employed during the decoding process. These filters aim to improve the visual quality of the reconstructed video, such as by smoothing out blocky artifacts, reducing noise, and enhancing details. Depending on the specific codec and configuration, zero or more filters may be applied to a reconstructed block. One such filter, known as CCSO, leverages the correlation between different color components (luma and chroma) to enhance the visual quality of the reconstructed video.

[0035] Briefly, the CCSO filter operates by adjusting at least one of luma or chroma samples based on the characteristics of corresponding and neighboring samples. CCSO utilizes a lookup table (LUT) that maps quantized differences between neighboring samples to offset values applied to current samples. The offset values are determined by the encoder and transmitted to the decoder. The encoder determines these offset values based on the source video data.

[0036] The behavior of the CCSO filter can be controlled through various syntax elements signaled by an encoder to a decoder in a compressed video bitstream. Among these elements are block-level control flags (i.e., CCSO unit-level control flags), which direct the decoder on whether to apply the CCSO filter to a specific color component (i.e., color plane) of a CCSO unit. A block-level control flag associated with a color component is used by the decoder to determine whether the CCSO filter should be applied to that color component of the particular CCSO unit. The block-level control flags are applied at the CCSO unit level. A reconstructed frame is partitioned into CCSO units (i.e., filter processing units), and the application of the CCSO filter to a CCSO unit is determined by the corresponding block-level control flag. [0037] A CCSO unit, which may also be referred to as a filter processing unit, is a processing unit for CCSO filtering. A CCSO unit may contain multiple variable-size coding blocks used for prediction and transform coding, and operates independently of the coding block partitioning. The size of a CCSO unit can be that of the largest coding block, which may also be referred to as a macroblock or superblock. For some codecs, the largest coding block size can be 256x256 pixels for a luma component or 128x128 pixels for a corresponding chroma component; however, other sizes for the CCSO unit are possible. The CCSO filter is further described with respect to FIG. 8.

[0038] These block-level control flags are typically entropy coded to achieve efficient compression. Entropy coding is a lossless compression technique that reduces the number of bits required to represent data, such as video, by using probability distributions (e.g., models) to estimate the distribution of values within a region of the image. By accurately modeling these probabilities, entropy coding can assign shorter codewords to more frequent symbols and longer codewords to less frequent symbols, thereby approaching the theoretical minimum bit rate. Common entropy coding methods include arithmetic coding and Huffman coding, both of which perform better when the probability distribution closely matches the actual data distribution. The effectiveness of entropy coding heavily relies on the context, which captures the statistical dependencies between symbols.

[0039] Implementations according to this disclosure utilize context-dependent entropy coding for a block-level (i.e., a CCSO unit-level) control flag associated with a current filter processing unit in a Cross-Component Sample Offset (CCSO) filtering process. This approach involves selecting a probability distribution for coding the flag based on data derived from neighboring filter processing units. The accuracy of the context modeling is enhanced by considering the spatial correlation between adjacent filter processing units. By selecting contexts that account for the state of neighboring CCSO units, the entropy coding of the blocklevel control flag can be improved.

[0040] In some implementations, a probability distribution is selected for coding a flag indicating whether a filter is applied to a filter processing unit of a reconstructed frame. The probability distribution is selected based on whether the filter is applied to neighboring filter processing units of the filter processing unit. The flag indicating whether the filter is applied to the filter processing unit is entropy coded. The filter is selectively applied to the filter processing unit based on the flag.

[0041] Further details of techniques for block-level control flag coding in crosscomponent sample offset are described herein with initial reference to a system in which they can be implemented. FIG. 1 is a schematic of a video encoding and decoding system 100. A transmitting station 102 can be, for example, a computer having an internal configuration of hardware such as that described in FIG. 2. However, other implementations of the transmitting station 102 are possible. For example, the processing of the transmitting station 102 can be distributed among multiple devices.

[0042] A network 104 can connect the transmitting station 102 and a receiving station 106 for encoding and decoding of the video stream. Specifically, the video stream can be encoded in the transmitting station 102, and the encoded video stream can be decoded in the receiving station 106. The network 104 can be, for example, the Internet. The network 104 can also be a local area network (LAN), wide area network (WAN), virtual private network (VPN), cellular telephone network, or any other means of transferring the video stream from the transmitting station 102 to, in this example, the receiving station 106.

[0043] The receiving station 106, in one example, can be a computer having an internal configuration of hardware such as that described in FIG. 2. However, other suitable implementations of the receiving station 106 are possible. For example, the processing of the receiving station 106 can be distributed among multiple devices.

[0044] Other implementations of the video encoding and decoding system 100 are possible. For example, an implementation can omit the network 104. In another implementation, a video stream can be encoded and then stored for transmission at a later time to the receiving station 106 or any other device having memory. In one implementation, the receiving station 106 receives (e.g., via the network 104, a computer bus, and/or some communication pathway) the encoded video stream and stores the video stream for later decoding. In an example implementation, a real-time transport protocol (RTP) is used for transmission of the encoded video over the network 104. In another implementation, a transport protocol other than RTP may be used (e.g., a Hypertext Transfer Protocol-based (HTTP-based) video streaming protocol).

[0045] When used in a video conferencing system, for example, the transmitting station 102 and/or the receiving station 106 may include the ability to both encode and decode a video stream as described below. For example, the receiving station 106 could be a video conference participant who receives an encoded video bitstream from a video conference server (e.g., the transmitting station 102) to decode and view and further encodes and transmits his or her own video bitstream to the video conference server for decoding and viewing by other participants.

[0046] FIG. 2 is a block diagram of an example of a computing device 200 that can implement a transmitting station or a receiving station. For example, the computing device 200 can implement one or both of the transmitting station 102 and the receiving station 106 of FIG. 1. The computing device 200 can be in the form of a computing system including multiple computing devices, or in the form of one computing device, for example, a mobile phone, a tablet computer, a laptop computer, a notebook computer, a desktop computer, and the like.

[0047] A processor 202 in the computing device 200 can be a conventional central processing unit. Alternatively, the processor 202 can be another type of device, or multiple devices, capable of manipulating or processing information now existing or hereafter developed. For example, although the disclosed implementations can be practiced with one processor as shown (e.g., the processor 202), advantages in speed and efficiency can be achieved by using more than one processor.

[0048] A memory 204 in computing device 200 can be a read-only memory (ROM) device or a random-access memory (RAM) device. However, other suitable types of storage device can be used as the memory 204. The memory 204 can include code and data 206 that is accessed by the processor 202 using a bus 212. The memory 204 can further include an operating system 208 and application programs 210, the application programs 210 including at least one program that permits the processor 202 to perform the techniques described herein. For example, the application programs 210 can include applications 1 through N, which further include a video coding application that performs the techniques described herein. The computing device 200 can also include a secondary storage 214, which can, for example, be a memory card used with a mobile computing device. Because the video communication sessions may contain a significant amount of information, they can be stored in whole or in part in the secondary storage 214 and loaded into the memory 204 as needed for processing.

[0049] In some implementations, the code and data 206, the operating system 208, and the application programs 210 may be stored on a non-transitory computer-readable storage medium. The term “non-transitory” excludes transitory signals and refers to media such as hard drives, flash memory, ROM, and other physical storage devices capable of storing executable instructions. Such a non-transitory computer-readable storage medium may contain instructions that, when executed by the processor 202, cause the computing device 200 to perform any of the methods or processes described herein, including reference frame motion field selection for wedge mode blocks. The secondary storage 214 is a non-transitory computer-readable storage medium that can store code and data, including machine-readable instructions that, when executed by the processor 202, cause the computing device 200 to perform one or more of the techniques described herein.

[0050] The computing device 200 can also include one or more output devices, such as a display 218. The display 218 may be, in one example, a touch sensitive display that combines a display with a touch sensitive element that is operable to sense touch inputs. The display 218 can be coupled to the processor 202 via the bus 212. Other output devices that permit a user to program or otherwise use the computing device 200 can be provided in addition to or as an alternative to the display 218. When the output device is or includes a display, the display can be implemented in various ways, including by a liquid crystal display (LCD), a cathode-ray tube (CRT) display, or a light emitting diode (LED) display, such as an organic LED (OLED) display.

[0051] The computing device 200 can also include or be in communication with an image-sensing device 220, for example, a camera, or any other image-sensing device 220 now existing or hereafter developed that can sense an image such as the image of a user operating the computing device 200. The image-sensing device 220 can be positioned such that it is directed toward the user operating the computing device 200. In an example, the position and optical axis of the image-sensing device 220 can be configured such that the field of vision includes an area that is directly adjacent to the display 218 and from which the display 218 is visible.

[0052] The computing device 200 can also include or be in communication with a soundsensing device 222, for example, a microphone, or any other sound-sensing device now existing or hereafter developed that can sense sounds near the computing device 200. The sound-sensing device 222 can be positioned such that it is directed toward the user operating the computing device 200 and can be configured to receive sounds, for example, speech or other utterances, made by the user while the user operates the computing device 200.

[0053] Although FIG. 2 depicts the processor 202 and the memory 204 of the computing device 200 as being integrated into one unit, other configurations can be utilized. The operations of the processor 202 can be distributed across multiple machines (wherein individual machines can have one or more processors) that can be coupled directly or across a local area or other network. The memory 204 can be distributed across multiple machines such as a network-based memory or memory in multiple machines performing the operations of the computing device 200. Although depicted here as one bus, the bus 212 of the computing device 200 can be composed of multiple buses. Further, the secondary storage 214 can be directly coupled to the other components of the computing device 200 or can be accessed via a network and can comprise an integrated unit such as a memory card or multiple units such as multiple memory cards. The computing device 200 can thus be implemented in a wide variety of configurations.

[0054] FIG. 3 is a diagram of an example of a video stream 300 to be encoded and subsequently decoded. The video stream 300 includes a video sequence 302. At the next level, the video sequence 302 includes a number of adjacent frames 304. While three frames are depicted as the adjacent frames 304, the video sequence 302 can include any number of adjacent frames 304. The adjacent frames 304 can then be further subdivided into individual frames, for example, a frame 306. At the next level, the frame 306 can be divided into a series of planes or segments 308. The segments 308 can be subsets of frames that permit parallel processing, for example. The segments 308 can also be subsets of frames that can separate the video data into separate colors. For example, a frame 306 of color video data can include a luminance component and two chrominance components. The segments 308 may be sampled at different resolutions.

[0055] Whether or not the frame 306 is divided into segments 308, the frame 306 may be further subdivided into blocks 310, which can contain data corresponding to, for example, 16x16 pixels in the frame 306. The blocks 310 can also be arranged to include data from one or more segments 308 of pixel data. The blocks 310 can also be of any other suitable size such as 4x4 pixels, 8x8 pixels, 16x8 pixels, 8x16 pixels, 16x16 pixels, or larger.

[0056] FIG. 4 is a block diagram of an encoder 400 according to implementations of this disclosure. The encoder 400 can be implemented, as described above, in the transmitting station 102, such as by providing a computer software program stored in memory, for example, the memory 204. The computer software program can include machine instructions that, when executed by a processor such as the processor 202, cause the transmitting station 102 to encode video data in the manner described in FIG. 4. The encoder 400 can also be implemented as specialized hardware included in, for example, the transmitting station 102. In one particularly desirable implementation, the encoder 400 is a hardware encoder.

[0057] The encoder 400 has the following stages to perform the various functions in a forward path (shown by the solid connection lines) to produce an encoded or compressed bitstream 420 using the video stream 300 as input: an intra/inter prediction stage 402, a transform stage 404, a quantization stage 406, and an entropy encoding stage 408. The encoder 400 may also include a reconstruction path (shown by the dotted connection lines) to reconstruct a frame for encoding of future blocks. In FIG. 4, the encoder 400 has the following stages to perform the various functions in the reconstruction path: a dequantization stage 410, an inverse transform stage 412, a reconstruction stage 414, and a loop filtering stage 416. Other structural variations of the encoder 400 can be used to encode the video stream 300.

[0058] When the video stream 300 is presented for encoding, respective adjacent frames 304, such as the frame 306, can be processed in units of blocks. At the intra/inter prediction stage 402, respective blocks can be encoded using intra-frame prediction (also called intraprediction) or inter- frame prediction (also called inter-prediction). In any case, a prediction block can be formed. In the case of intra-prediction, a prediction block may be formed from samples in the current frame that have been previously encoded and reconstructed. In the case of inter-prediction, a prediction block may be formed from samples in one or more previously constructed reference frames.

[0059] Next, the prediction block can be subtracted from the current block at the intra/inter prediction stage 402 to produce a residual block (also called a residual). The transform stage 404 transforms the residual into transform coefficients in, for example, the frequency domain using block-based transforms. The quantization stage 406 converts the transform coefficients into discrete quantum values, which are referred to as quantized transform coefficients, using a quantizer value or a quantization level. For example, the transform coefficients may be divided by the quantizer value and truncated.

[0060] The quantized transform coefficients are then entropy encoded by the entropy encoding stage 408. The entropy-encoded coefficients, together with other information used to decode the block (which may include, for example, syntax elements such as used to indicate the type of prediction used, transform type, motion vectors, a quantizer value, or the like), are then output to the compressed bitstream 420. The compressed bitstream 420 can be formatted using various techniques, such as variable length coding (VLC) or arithmetic coding. The compressed bitstream 420 can also be referred to as an encoded video stream or encoded video bitstream, and the terms will be used interchangeably herein.

[0061] The reconstruction path (shown by the dotted connection lines) can be used to ensure that the encoder 400 and a decoder 500 (described below with respect to FIG. 5) use the same reference frames to decode the compressed bitstream 420. The reconstruction path performs functions that are similar to functions that take place during the decoding process (described below with respect to FIG. 5), including dequantizing the quantized transform coefficients at the dequantization stage 410 and inverse transforming the dequantized transform coefficients at the inverse transform stage 412 to produce a derivative residual block (also called a derivative residual). At the reconstruction stage 414, the prediction block that was predicted at the intra/inter prediction stage 402 can be added to the derivative residual to create a reconstructed block. The loop filtering stage 416 can be applied to the reconstructed block to reduce distortion such as blocking artifacts.

[0062] Other variations of the encoder 400 can be used to encode the compressed bitstream 420. In some implementations, a non-transform based encoder can quantize the residual signal directly without the transform stage 404 for certain blocks or frames. In some implementations, an encoder can have the quantization stage 406 and the dequantization stage 410 combined in a common stage.

[0063] FIG. 5 is a block diagram of a decoder 500 according to implementations of this disclosure. The decoder 500 can be implemented in the receiving station 106, for example, by providing a computer software program stored in the memory 204. The computer software program can include machine instructions that, when executed by a processor such as the processor 202, cause the receiving station 106 to decode video data in the manner described in FIG. 5. The decoder 500 can also be implemented in hardware included in, for example, the transmitting station 102 or the receiving station 106.

[0064] The decoder 500, similar to the reconstruction path of the encoder 400 discussed above, includes in one example the following stages to perform various functions to produce an output video stream 516 from the compressed bitstream 420: an entropy decoding stage 502, a dequantization stage 504, an inverse transform stage 506, an intra/inter prediction stage 508, a reconstruction stage 510, a loop filtering stage 512, and a deblocking filtering stage 514. Other structural variations of the decoder 500 can be used to decode the compressed bitstream 420.

[0065] When the compressed bitstream 420 is presented for decoding, the data elements within the compressed bitstream 420 can be decoded by the entropy decoding stage 502 to produce a set of quantized transform coefficients. The dequantization stage 504 dequantizes the quantized transform coefficients (e.g., by multiplying the quantized transform coefficients by the quantizer value), and the inverse transform stage 506 inverse transforms the dequantized transform coefficients to produce a derivative residual that can be identical to that created by the inverse transform stage 412 in the encoder 400. Using header information decoded from the compressed bitstream 420, the decoder 500 can use the intra/inter prediction stage 508 to create the same prediction block as was created in the encoder 400 (e.g., at the intra/inter prediction stage 402).

[0066] At the reconstruction stage 510, the prediction block can be added to the derivative residual to create a reconstructed block. The loop filtering stage 512 can be applied to the reconstructed block to reduce blocking artifacts. Other filtering can be applied to the reconstructed block. In this example, the deblocking filtering stage 514 is applied to the reconstructed block to reduce blocking distortion, and the result is output as the output video stream 516. The output video stream 516 can also be referred to as a decoded video stream, and the terms will be used interchangeably herein. Other variations of the decoder 500 can be used to decode the compressed bitstream 420. In some implementations, the decoder 500 can produce the output video stream 516 without the deblocking filtering stage 514.

[0067] FIG. 6 is an illustration of examples of portions of a video frame 600, which may, for example, be the frame 306 shown in FIG. 3. The video frame 600 includes a number of 64x64 CTUs, such as four 64x64 CTUs 610 in two rows and two columns in a matrix or Cartesian component, as shown. Each 64x64 CTU 610 may include up to four 32x32 CUs 620. Each 32x32 CU 620 may include up to four 16x16 CUs 630. Each 16x16 CU 630 may include up to four 8x8 CUs 640. Each 8x8 CU 640 may include up to four 4x4 CUs 650. Each 4x4 CU 650 may include 16 pixels, which may be represented in four rows and four columns in each respective CU in the Cartesian plane or matrix.

[0068] In some implementations, the video frame 600 may include CTUs larger than 64x64 and/or CUs smaller than 4x4. Subject to features within the video frame 600 and/or other criteria, the video frame 600 may be partitioned into various arrangements. Although one arrangement of CUs is shown, any arrangement may be used. Although FIG. 6 shows NxN CTUs and CUs, in some implementations, NxM CTUs and/or CUs may be used, wherein N and M are different numbers. For example, 32x64 CTUs, 64x32 CTUs, 16x32 CUs, 32x16 CUs, or any other size may be used. In some implementations, Nx2N CTUs or CUs, 2NxN CTUs or CUs, or a combination thereof, may be used.

[0069] The pixels may include information representing an image captured in the video frame 600, such as luminance information, color information, and location information. In some implementations, a block, such as a 16x16 pixel block as shown, may include a luminance block 660, which may include luminance pixels 662; and two chrominance blocks 670, 680, such as a U or Cb chrominance block 670, and a V or Cr chrominance block 680. The chrominance blocks 670, 680 may include chrominance pixels 690. For example, the luminance block 660 may include 16x16 luminance pixels 662 and each chrominance block 670, 680 may include 8x8 chrominance pixels 690 as shown.

[0070] In some implementations, coding the video frame 600 may include ordered blocklevel coding. Ordered block-level coding may include coding CUs of the video frame 600 in an order, such as raster-scan order, wherein CUs may be identified and processed starting with a CTU in the upper left comer of the video frame 600, or portion of the video frame 600, and proceeding along rows from left to right and from the top row to the bottom row, identifying each CU in turn for processing. For example, the 64x64 CTU in the top row and left column of the video frame 600 may be the first CTU coded and the 64x64 CTU immediately to the right of the first CTU may be the second CTU coded. The second row from the top may be the second row coded, such that the 64x64 CTU in the left column of the second row may be coded after the 64x64 CTU in the rightmost column of the first row. [0071] In some implementations, coding a CTU of the video frame 600 may include using quad-tree coding, which may include coding smaller CUs within a CTU in raster-scan order. For example, the 64x64 CTU shown in the bottom left corner of the portion of the video frame 600 may be coded using quad-tree coding wherein the top left 32x32 CU may be coded, then the top right 32x32 CU may be coded, then the bottom left 32x32 CU may be coded, and then the bottom right 32x32 CU may be coded. Each 32x32 CU may be coded using quad-tree coding wherein the top left 16x16 CU may be coded, then the top right 16x16 CU may be coded, then the bottom left 16x16 CU may be coded, and then the bottom right 16x16 CU may be coded. Each 16x16 CU may be coded using quad-tree coding wherein the top left 8x8 CU may be coded, then the top right 8x8 CU may be coded, then the bottom left 8x8 CU may be coded, and then the bottom right 8x8 CU may be coded. Each 8x8 CU may be coded using quad-tree coding wherein the top left 4x4 CU may be coded, then the top right 4x4 CU may be coded, then the bottom left 4x4 CU may be coded, and then the bottom right 4x4 CU may be coded. In some implementations, 8x8 CUs may be omitted for a 16x16 CU, and the 16x16 CU may be coded using quad-tree coding wherein the top left 4x4 CU may be coded, then the other 4x4 CUs in the 16x16 CU may be coded in raster- scan order.

[0072] In some implementations, coding the video frame 600 may include encoding the information included in the original version of the image or video frame by, for example, omitting some of the information from that original version of the image or video frame from a corresponding encoded image or encoded video frame. For example, the coding may include reducing spectral redundancy, reducing spatial redundancy, or a combination thereof. Reducing spectral redundancy may include using a color model based on a luminance component (Y) and two chrominance components (U and V or Cb and Cr), which may be referred to as the YUV or YCbCr color model, or color space. Using the YUV color model may include using a relatively large amount of information to represent the luminance component of a portion of the video frame 600, and using a relatively small amount of information to represent each corresponding chrominance component for the portion of the video frame 600. For example, a portion of the video frame 600 may be represented by a high-resolution luminance component, which may include a 16x16 block of luma samples, and by two lower resolution chrominance components, each of which represents the portion of the image as an 8x8 block of chroma samples. A sample may indicate a value, for example, a value in the range from 0 to 255, and may be stored or transmitted using, for example, eight bits. Although this disclosure is described in reference to the YUV color model, another color model may be used. Reducing spatial redundancy may include transforming a CU into the frequency domain using, for example, a discrete cosine transform. For example, a unit of an encoder may perform a discrete cosine transform using transform coefficient values based on spatial frequency.

Although described herein with reference to matrix or Cartesian representation of the video frame 600 for clarity, the video frame 600 may be stored, transmitted, processed, or a combination thereof, in a data structure such that pixel values and/or luma and chroma samples may be efficiently represented for the video frame 600. For example, the video frame 600 may be stored, transmitted, processed, or any combination thereof, in a two-dimensional data structure such as a matrix as shown, or in a one-dimensional data structure, such as a vector array. Furthermore, although described herein as showing a chrominance subsampled image where U and V have half the resolution of Y, the video frame 600 may have different configurations for the color channels thereof. For example, referring still to the YUV color space, full resolution may be used for all color channels of the video frame 600. In another example, a color space other than the YUV color space may be used to represent the resolution of color channels of the video frame 600.

[0073] FIG. 7 is a block diagram of an example of a video frame filtering stage 700. The video frame filtering stage 700 performs filtering on a reconstructed frame 702 to obtain an enhanced frame 704 to prepare the reconstructed frame 702 for display or storage. During encoding, the video frame filtering stage 700 may be the loop filtering stage 416 of the encoder 400 shown in FIG. 4 or a stage that performs some, but not all, of the operations performed by the loop filtering stage 416. During decoding, the video frame filtering stage 700 may be the loop filtering stage 512 of the decoder 500 shown in FIG. 5 or a stage that performs some, but not all, of the operations performed by the loop filtering stage 512.

[0074] The reconstructed frame 702 is a video frame output from a reconstruction stage, which may be the reconstruction stage 414 of the encoder 400 or the reconstruction stage 510 of the decoder 500. After the filtering performed by the video frame filtering stage 700, the enhanced frame 704 is sent as output for display or storage 706. The display or storage 706 may represent or include operations for storing the enhanced frame 704 in a reference frame buffer of the encoder 400 or of the decoder 500. Alternatively, the display or storage 706 may represent or include operations for outputting the enhanced frame 704 within an output video stream for display at a device that receives the output video stream.

[0075] The video frame filtering stage 700 receives the reconstructed frame 702 after the video frame is output from the reconstruction stage and prepares the reconstructed frame 702 for sending as output for the display or storage 706. The video frame filtering stage 700 may apply one or more filters to the reconstructed frame 702 to obtain the enhanced frame 704. The encoder selects and signals in the compressed bitstream which of the available filters are to be applied to the reconstructed frame 702. The encoder can encode (e.g., signal) parameters (e.g., configuration characteristics) for the filters. In an example, a deblocking filter 708 is always applied and, as such, the encoder does not signal whether the deblocking filter 708 is to be applied.

[0076] The video frame filtering stage 700 uses processing units to process individual regions of the reconstructed frame 702 one at a time. A processing unit is a coding structure of size MxN, where M and N may be the same or different numbers. The size of the processing units may be based on the size of the largest block within the reconstructed frame 702. For example, if the largest block within the reconstructed frame 702 is 128x128, the processing units used by the video frame filtering stage 700 may be of size 128x128 or larger. In an example, each filter of the video frame filtering stage 700 may use different processing unit sizes.

[0077] The processing units are typically square in shape. However, as the processing units may be of size MxN, the processing units may be square or rectangular in shape. The processing units are typically all of the same size and shape. However, in some cases, the processing units may be variably sized and/or variably shaped. For example, a variable size and/or variable shape processing unit partitioning scheme can be used to divide the reconstructed frame 702 into a plurality of processing units. [0078] Each of the processing units includes pixel values from a region of the reconstructed frame 702. Each of the pixel values of the video frame is included in a single processing unit. As such, the video frame filtering stage 700 filters each of the pixel values of the reconstructed frame 702 by processing each of the processing units. The video frame filtering stage 700 sequentially processes the processing units one at a time. The order for processing the processing units at the video frame filtering stage 700 may depend upon a scan order or other order for the encoding or decoding of the reconstructed frame 702. For example, where a raster order is used, the video frame filtering stage 700 first processes a processing unit that includes top-left-most pixel values of the reconstructed frame 702.

[0079] The video frame filtering stage 700 is shown as including the deblocking filter 708, a constrained directional enhancement filter (CDEF) 710, a loop restoration (LR) filter 712, and a CCSO filter 714. The video frame filtering stage 700 may include more or fewer filters. The CCSO filter 714 can be performed in parallel with the CDEF 710. That is, the input to the CCSO filter 714 is the same as that provided to the CDEF 710 and the output is applied to the CDEF -filtered samples. The CCSO fiber 714 is further described with respect to FIG, 8,

[0080] The deblocking filter 708 can be applied across transform block boundaries to remove block artifacts caused by quantization errors. The CDEF 710 performs edge direction searching at an 8x8 block-level. In CDEF, eight edge directions can be identified within blocks according to edge templates. A primary filter processes reconstruction samples along the edge direction while a secondary filter processes reconstruction samples along a direction 45 degrees from the edge direction.

[0081] The LR filter 712 is applied to units of either 64x64, 128x128, or 256x256-pixel blocks, named loop restoration units (LRU). Bypass filtering, a Wiener filter, or a self-guided filter can be independently selected for each LRU. The self-guided filter scheme applies simple filters to reconstructed pixels, X, to generate two denoised versions, Xi and X2; their differences from the reconstructed pixels, (Xi-X) and (X2- X), are used to span a sub-space, upon which the differences between the reconstructed pixels and the original pixels, (X_s - X) are projected.

[0082] The Wiener filter can be a 7x7 separable filter that includes a 7-tap vertical filter and a 7-tap horizontal filter. Filtering of the reconstruction samples of a block can be performed by applying the vertical and horizontal filters sequentially. After applying the vertical and horizontal filters, the final filtered reconstruction samples are generated. The decoded frame pixel values (at (p, q) and a k X k neighborhood of the pixel) are used to filter to obtain a filtered frame value at corresponding pixels. The process can be formulated as shown in equation (1)

[0083] In equation (1), (p, q) indicates a location of a pixel of an image or video frame, and are the filter coefficients for its k X k neighborhood. The filter coefficients can be derived by an encoder and signaled into a compressed bitstream. Alternatively, a set of filters may be predefined and stored at both an encoder and a decoder, and a predefined logic can be used as the filter for a pixel or block at both the encoder and the decoder. Some other shape of the neighborhood, such as a diamond shape, may be used instead of the rectangle or square (i.e., k X fc) shape.

[0084] In some implementations, symmetric Wiener filters can be used to reduce the bit overhead associated with filter coefficient signaling and also to reduce the computational complexity of the filtering process. As such, only three coefficients may be signaled for a 7- tap filter, with the three mirrored coefficients derived as the same values. That is, in the symmetric filter, the value of f^(p,qj(m> ri) is equal to f^(p,qj(—m, — ri).

[0085] FIG. 8 is a block diagram 800 that illustrates the operations of the CCSO filter. The CCSO filter may adjust reconstructed samples of a video frame to enhance the visual quality of the reconstructed video frame, such as the reconstructed frame 702 of FIG. 7. The CCSO filter corrects luma and/or co-located chroma reconstruction samples with offsets. The CCSO filter operates on CCSO units (also referred to herein as “filter processing units”) of the reconstructed video frame. As mentioned above, a CCSO unit can have the size of the largest possible coding block.

[0086] A filter 802 illustrates that the CCSO filter uses a 3-tap filter applied to luma pixels. The CCSO filter may be applied to either or both of the luma or chroma pixels of a current CCSO unit. In CCSO, a set of 3-tap filters are used. The input luma reconstructed samples located at the three filter taps include a current luma pixel 804 (i.e., denoted r/) in the center, and two symmetrically neighboring luma samples 808 and 810, denoted po and pi, respectively. In this context, the current luma pixel 804 has a co-located chroma pixel 806, denoted rd. When the CCSO filter is applied to a luma sample, the current luma pixel 804 is an actual luma sample of reconstructed video frame; and when the CCSO filter is applied to a chroma sample, the current luma pixel 804 is a co-located luma pixel to a chroma sample. The co-located luma pixel may be obtained as described herein and may be, in some implementations, an actual luma sample of reconstructed video frame.

[0087] To illustrate the concept of a luma pixel being co-located with a chroma pixel, a 4:2:0 chroma subsampling scheme is assumed. In this scheme, each chroma pixel corresponds to four luma pixels. For example, a luma block 812 corresponds to a chroma block 814. In this scheme, each group of four luma pixels in the luma block 812 (e.g., luma pixels 818 numbered 0, 1, 4, and 5) corresponds to one chroma pixel (e.g., chroma pixel 816 numbered 0) in the chroma block 814. The luma co-located pixel for the chroma pixel 816 can be derived from the corresponding luma pixels 818. The co-located luma pixel may be the average, the median, or some other function of the luma pixels 818. The co-located luma pixel may be one of the luma pixels 818, such as the top-left luma pixel (e.g., the luma pixel numbered 0). Other ways of obtaining the co-located luma pixel are also possible.

[0088] The differences between these luma samples used in the filtering are computed, and these differences are quantized into discrete levels denoted as do and di. The quantized values are then used to determine a combination index from a combination lookup table (LUT) 820.

[0089] That is, given pi and r/, where i = 0, 1, the following steps are applied to process the input samples:

1) the delta values, denoted rm, between pi and n are computed (i.e., nn=ri-pi), then

2) the delta values, rm, are quantized as di based on a quantization step size Qccso, using the following quantization process: a) di is set equal to -1, if mi is less than -Qccso b) di is set equal to 0, if mi is in the range [-Qccso, Qccso], and c) di is set equal to 1, if mi is greater than Qccso

[0090] The quantization step size, Qccso, can be 8, 16, 32, or 64. After do and di are calculated, an offset value (denoted 5) is derived using the LUT 820. Each combination of do and di is used to identify a row in the LUT 820 to retrieve the offset value, namely a gradient offset. The offset values can be integers including 0, 1, -1, 3, -3, 7, -7, and -10. Finally, the derived offset 5 of CCSO is applied to a chroma color component using equation (2): r_c' = clip(r_c + s) (2)

[0091] In equation (2), r_c is the reconstructed sample to be filtered by CCSO, and 5 is the derived offset value retrieved from the LUT 820; the filtered sample value r_c' is further clipped into the range specified by the bit depth. [0092] The same CCSO process can be applied to the luma components with the exception that the output is applied to the luma reconstruction samples themselves and a colocated luma pixel is not used (e.g., calculated or selected).

[0093] In CCSO, there are six optional filter shapes, denoted as/], i = 1...6, as shown in filters 822. These six filter shapes are switchable (e.g., selectable or set) at the video frame level, and the selection can be signaled by a syntax element, exlj"dler_supporl, using a 3-bit fixed length code.

[0094] With respect to the LUT 820, the offset values so through ss may not be fixed and may vary from video frame to video frame. As such, the offset values so through ss can be calculated by an encoder and transmitted to the decoder in the compressed bitstream. The offset values so through ss can be transmitted as nine, 3-bit offset values.

[0095] FIG. 9 illustrates syntax elements that may be used to signal aspects of the CCSO filter. The signaling of CCSO can be categorized into frame-level and block-level syntax elements. A table 900 illustrates the frame-level syntax elements; and a table 910 illustrates the block-level (i.e., CCSO unit-level) syntax elements. In this context, the syntax elements refer to CCSO units (i.e., filter processing units). For example, with respect to a luma component, the CCSO unit may be 256x256 pixels, and the corresponding chroma CCSO units may each be 128x128 pixels in a 4:2:0 subsampling format. The block- level control flags enable CCSO to be selectively applied within these CCSO units, depending on the content and the desired level of filtering.

[0096] The table 900 is shown as including syntax elements 902 through 908. The syntax elements of the table 900 can be included in a compressed bitstream of frame header for each color component of a video frame. For purposes of this description, the syntax elements of the table 900 are described with respect to the luma component. The syntax element 902 (e.g., FLAG) is a one-bit flag indicating whether CCSO is applied (e.g., enabled) for the luma component of the frame. That is, the syntax element 902 indicates whether the CCSO filter is applied to at least one luma block of the frame. If CCSO is not enabled for the luma component of the frame, then the other syntax elements (e.g., the syntax elements 902 through 908) are not included in the compressed bitstream.

[0097] The syntax element 904 (e.g., EXT_FILTER_SUPPORT) is a three-bit field indicating the filter shape to be used during the CCSO process. The filter shapes can be as described with respect to FIG. 8. The syntax element 906 (e.g., Q_STEP) is a two-bit field indicating the selection of the quantization step (e.g., Qccso described above). This field defines the level of quantization applied to the calculated differences, mi, which in turn affects the offsets used in the CCSO process. Syntax elements 908 (e.g., OFFSETS) include nine three-bit values that are used in the LUT (e.g., the LUT 820 of FIG. 8) for determining the offset values applied during the CCSO process.

[0098] Again, the frame-level syntax elements 902 through 908 may be included in the frame header for each of the Y (luma), U (chroma U), and V (chroma V) color components. In some implementations, the U and V syntax elements may be assumed to be the same, allowing for a more efficient bitstream. That is, the frame header would not include separate syntax elements for the chroma U and chroma V color components.

[0099] The table 910 is shown as including flags 912 through 916. The flag 912 (e.g., LUMA_FLAG) is a one-bit flag indicating whether CCSO is applied to the current luma CCSO unit. This flag provides fine control at the CCSO unit level, enabling or disabling CCSO for specific luma CCSO units within the frame. The flag 914 (e.g., CHROMA_U_FLAG) is a one-bit flag indicating whether CCSO is applied to the current chroma U CCSO unit. Similar to the luma flag, this flag allows selective application of CCSO to chroma U CCSO units. The flag 916 (e.g., CHROMA_V_FLAG) is a one-bit flag indicating whether CCSO is applied to the current chroma V CCSO unit. This flag enables or disables CCSO for specific chroma V CCSO units, allowing for precise control over the application of CCSO across different color components.

[0100] FIGS. 10A and 10B provide an implementation of a function 1000 for identifying probability distributions for entropy coding block-level syntax elements described with respect to FIG. 9, such as the flags shown in the table 910 of FIG. 9. For each of the flags 912 through 916, a probability distribution may be selected from a table (e.g., a set) of probability distributions ccso_cdf[3][CCSO_CONTEXT][CDF_SIZE(2)], which represents a three- dimensional array that stores the cumulative distribution functions (CDFs) used for entropy coding the block-level syntax elements.

[0101] The first dimension, [3], corresponds to the three color components (Y, U, and V). The second dimension, [CCSO_CONTEXT], represents four possible contexts (0 to 3) derived from neighboring CCSO units' control flags (described below). The third dimension, [CDF_SIZE(2)], stores the probabilities for the two possible values of the flag (0 or 1, indicating whether CCSO is enabled or disabled for the current block). The CCSO_CONTEXT values are determined based on the CCSO control flags of neighboring CCSO units, allowing the encoder to leverage the spatial correlation between CCSO units and improve the accuracy of probability estimations for the current CCSO unit's flag. [0102] At code block 1002, a function avl_get_ccso_context is designed to determine the context value for the entropy coding of a block-level on/off flag based on the state of neighboring CCSO units. The function takes three parameters: a pointer to a structure containing common data for the AVI codec (AV1_COMMON *cm), a pointer to the macroblock data being processed (MACROBLOCKD *xd), and an integer representing the color component (i.e., int plane). The function begins by initializing two pointers, neighborO and neighbor 1, which point to 4x4 neighboring blocks above and to the left of the current block (the CCSO unit). Specifically, neighborO and neighborl correspond to the neighboring blocks located at xd-> neighbors [0] and xd->neighbors[l], respectively, where xd is a pointer to the macroblock data structure containing neighboring block information. As further described, at least one of the neighborO and neighborl may not be available; and the neighborO and neighborl may or may not belong to the same CCSO unit.

[0103] At code block 1004, the CCSO unit size for the chroma planes is computed by adjusting for any subsampling. The logarithmic size of the block in the y and x dimensions is computed and stored in log2_blk_size_y and log2_blk_size_x, respectively. The actual size of the block in pixels, derived from these logarithmic values, is stored in blk_size_y and blk_size_x.

[0104] At code block 1006, the function then checks if both neighboring CCSO units (above and left) exist by evaluating neighborO and neighborl. If both neighbors exist, the starting coordinates of the CCSO unit (a large block that may contain multiple coding units) for each neighbor are calculated and stored in neighbor0_sb_y, neighbor0_sb_x, neighbor l_sb_y, and neighbor l_sb_x.

[0105] At code block 1008, the superblock references for neighborO and neighborl are then determined and stored in neighbor0_sb and neighborl_sb, respectively. Two variables, is_neighbor0_ccso and is_neighborl_ccso, are initialized to store whether CCSO is enabled for each neighbor.

[0106] At code block 1010, the function checks the plane value to determine which color component is being processed (luma, chroma U, or chroma V). Depending on the plane, the CCSO flag values from the superblock for each neighbor are assigned to is_neighbor0_ccso and is_neighborl_ccso. Specifically, when the variable plane indicates the luma component, the function checks the ccso_blk_y flag; when plane indicates the chroma U component, the function checks the ccso_blk_u flag; and when plane indicates the chroma V component, the function checks the ccso_blk_v flag of the neighboring CCSO units. This determines whether CCSO is applied to the corresponding CCSO units in each plane. [0107] At code block 1012 of FIG. 10B, the function determines the context based on the status of the neighboring superblocks. The context values (0, 1, 2, 3) correspond to different combinations of CCSO being enabled or disabled in the neighboring blocks. The context logic can be summarized using the shorthand notation shown in Table I, where “ccso_true” indicates CCSO is enabled, “ccso_false” indicates CCSO is disabled, and ” indicates the neighbor is not available:

TABLE I

[0108] More specifically, the context value is determined based on the states of neighboring CCSO units, including whether they are available, whether they are the same CCSO unit or different CCSO units, and whether CCSO is enabled for those units. The context value is 0 when: both neighboring CCSO units exist and both have CCSO disabled, only one neighboring CCSO unit exists and has CCSO disabled, or no neighboring CCSO units exist. The context value is 1 when both neighbors exist, belong to different CCSO units, and exactly one has CCSO enabled. The context value is 2 when: both neighbors exist and belong to the same CCSO unit with CCSO enabled, or only one neighbor exists and has CCSO enabled. The context value is 3 when both neighbors exist, belong to different CCSO units, and both have CCSO enabled.

[0109] The shorthand notation used in Table I corresponds to the actual implementation variables as follows: the terms “neighbor0_ccso_true” and “neighbor0_ccso_false” represent the boolean states of the variable is_neighbor0_ccso, where “neighbor0_ccso_true” indicates is_neighborO_ccso == true and “neighbor0_ccso_false” indicates is_neighborO_ccso == false. Similarly, “neighborl_ccso_true” and “neighborl_ccso_false” represent the boolean states of the variable is_neighborl_ccso. The terms “neighborO_ccso_unit” and “neighborl_ccso_unit” refer to the CCSO unit identities determined by the neighborO_sb_y, neighborO_sb_x, neighbor l_sb_y, and neighborl_sb_x coordinate values.

[0110] For example, Context 1 (neighborO_ccso_true/neighborl_ccso_false, neighborO_ccso_false/neighborl_ccso_true) translates to the implementation condition where (is_neighborO_ccso == true && is_neighborl_ccso == false) || (is_neighborO_ccso == false && is_neighborl_ccso == true), representing cases where exactly one of the two neighbors has CCSO enabled. Context 3 (neighborO_ccso_true/neighborl_ccso_true && neighborO_ccso_unit!=neighborl_ccso_unit) corresponds to the implementation condition where (is_neighborO_ccso == true && is_neighborl_ccso == true) and the neighbors belong to different CCSO units as determined by comparing their coordinate values.

[0111] At code block 1014, if only one neighbor exists (either neighborO or neighborl), the function processes the existing neighbor. The starting coordinates of the superblock for the available neighbor are calculated and stored in neighbor_sb_y and neighbor_sb_x.

[0112] At code block 1016, the superblock reference for the single existing neighbor is then determined and stored in neighbor_sb. The context value is set based on whether CCSO is enabled for the neighbor’s corresponding plane (luma, chroma U, or chroma V). The function returns 2 if CCSO is enabled for the neighbor, otherwise 0.

[0113] At code block 1018, if neither neighborO nor neighborl exists, the function returns 0, indicating no CCSO context from neighbors.

[0114] The context value can be determined as follows: when both neighboring CCSO units exist and are different CCSO units, the value 3 is returned if CCSO is enabled for both neighbors; otherwise, the value 1 is returned if CCSO is enabled for only one of them. If both neighboring CCSO units exist and are the same CCSO unit, the value 2 is returned if CCSO is enabled for that CCSO unit; otherwise, the value 0 is returned. When only one neighboring CCSO unit exists, the value 2 is returned if CCSO is enabled for that CCSO unit; otherwise, the value 0 is returned. If neither of the neighboring CCSO units exists, the value 0 is returned.

[0115] FIG. 11 illustrates neighboring CCSO units of a CCSO unit used as context in deriving probability distributions for entropy coding the CCSO control flags. FIG. 11 includes a CCSO unit 1100 along with its neighboring blocks 1102, 1104, 1106, and 1108, which may each be a 4x4 block. The neighboring blocks 1102, 1104, 1106, and 1108 are used to identify neighborO and neighborl described with respect to FIGS. 10A-10B. The neighboring blocks 1102, 1104, 1106, and 1108 are sub-blocks of neighboring CCSO (filter processing) units.

[0116] Identifying neighborO and neighborl follows a specific search order. According to this search order, neighborO is set to the first identified neighboring block in the sequence, and neighborl is set to the second identified neighboring block, if available. Each of these neighbors corresponds to a 4x4 block adjacent to the current CCSO unit 1100. The identified neighborO and neighborl are then used to determine the CCSO units (neighbor0_ccso and neighborl_ccso) that include these neighbors.

[0117] For instance, if the search order identifies the neighboring block 1102 first, then neighborO is set to the neighboring block 1102, and neighbor0_ccso is the CCSO unit that includes block 1102. In this case, neighbor0_ccso would be set to a neighboring CCSO unit 1110. If the neighboring block 1104 is identified next, neighborl is set to the neighboring block 1104, and neighbor l_cc so is set to a neighboring CCSO unit 1112 that includes the neighboring block 1104. In another example, if the search order identifies the neighboring block 1102 first, then neighborO is set to the neighboring block 1102, and neighbor0_ccso is set to the neighboring CCSO unit 1110. If the neighboring block 1106 is identified next, neighborl is set to the neighboring block 1106, and neighbor l_cc so is also set to the neighboring CCSO unit 1110 that includes the neighboring block 1106.

[0118] These identified CCSO units (neighbor0_ccso and neighborl_ccso) are then used to derive the context (CCSO_CONTEXT), which is a key input for selecting the appropriate probability distribution from the ccso_cdf[3][CCSO_CONTEXT][CDF_SIZE(2)] table. This context-dependent entropy coding improves the efficiency of the compression process by leveraging the spatial correlation between neighboring blocks.

[0119] The variables is_neighbor0_ccso and is_neighborl_ccso are derived during the execution of the function described in FIGS. 10A-10B. Specifically, after neighborO and neighborl are identified and their respective CCSO units (neighbor0_ccso and neighborl_ccso) are determined, the function checks whether CCSO is enabled for these units for the particular color component (i.e., plane) being processed. This check is stored in the variables is_neighbor0_ccso and is_neighborl_ccso. These flags indicate whether CCSO is active in the neighboring units.

[0120] If only one neighbor is identified, such as neighborO, the variable is_neighbor_ccso is used to store whether CCSO is enabled for the CCSO unit associated with neighborO (i.e., for the particular color component being processed). This may be done when neighborl is not available or does not exist. The is_neighbor_ccso flag is then used to determine the context for CCSO entropy coding in cases where only one neighboring block is involved.

[0121] FIG. 12 is a flowchart of a technique 1200 for entropy coding block-level (i.e., CCSO unit-level) control flags associated with CCSO. That is, the technique 1200 can be used to entropy code one or more of the flags 912 through 916 of FIG. 9. The technique 1200 can be implemented, for example, as a software program that may be executed by computing devices such as transmitting station 102 or receiving station 106. The software program can include machine-readable instructions that may be stored in a memory such as the memory 204 or the secondary storage 214, and that, when executed by a processor, such as processor 202, may cause the computing device to perform the technique 1200. The technique 1200 can be implemented using specialized hardware or firmware. Multiple processors, memories, or both, may be used.

[0122] The technique 1200 may be implemented in whole or in part in the entropy encoding stage 408 of the encoder 400 of FIG. 4 and/or the entropy decoding stage 502 of the decoder 500 of FIG. 5. When implemented by an encoder, entropy coding means entropy encoding; and when implemented by a decoder, entropy coding means entropy decoding. [0123] At 1202, a probability distribution is selected for coding a flag that indicates whether a filter is applied to a filter processing unit of a reconstructed frame. The probability distribution is selected based on whether the filter is applied to neighboring filter processing units of the filter processing unit. In some implementations, the filter is a Cross-Component Sample Offset (CCSO) filter. As such, the filter processing unit can be the CCSO unit 1100, and the neighboring filter processing units can be or include the neighboring CCSO units 1110 and 1112 of FIG. 11. The flag can be one of the flags 912 to 916 of FIG. 9. As such, flag is a block-level control flag that is applied at the CCSO unit level, as described herein. [0124] The identification of neighboring filter processing units may be based on a predefined search order of sub-blocks within those neighboring filter processing units, such as described with respect to FIG. 11. The search order may include identifying a first subblock of a first neighboring filter processing unit and identifying a second sub-block of a second neighboring filter processing unit.

[0125] The selected probability distribution may vary depending on the relationship between the neighboring filter processing units. For instance, a first probability distribution may be selected when the first and second neighboring filter processing units exist, are different filter processing units, and the filter is applied to both. A second probability distribution may be selected if the neighboring filter processing units belong to the same filter processing unit and have the filter applied; or if only one neighboring unit exists and has the filter applied. A third probability distribution may be selected if the neighboring filter processing units exist but belong to different filter processing units, with the filter applied to only one of them. A fourth probability distribution may be selected if neither neighboring filter processing unit exists or if neither has the filter applied.

[0126] At 1204, the flag indicating whether the filter is applied to the filter processing unit is entropy coded using the selected probability distribution. The entropy coding process compresses the flag for efficient transmission or storage within the compressed bitstream. At 1206, the filter is selectively applied to the filter processing unit based on the flag. In some implementations, the coded flag may be stored for use in filtering subsequent filter processing units within the same frame as the current filter processing unit.

[0127] In some implementations, the CCSO filter may be applied in the deblocking filtering stage 514 of the decoder. As such, the CCSO is not applied at the encoder.

Therefore, in such an implementation, the filter is not applied by the encoder. That is, the step 1206 would not be performed at the encoder. In some other implementations, the CCSO filter may be applied in the loop filtering stage 512 of the decoder. In such implementations, the step 1206 would be performed at the encoder.

[0128] To reiterate, the probability distribution can be selected from a plurality of probability distributions based on states of the neighboring filter processing units. The plurality of probability distributions includes a first probability distribution when both neighboring filter processing units exist, are different units, and both have the filter applied. The plurality of probability distributions includes a second probability distribution when both neighboring filter processing units are a same unit and have the filter applied, or when only one neighboring unit exists and has the filter applied. The plurality of probability distributions includes a third probability distribution when both neighboring filter processing units exist, are different units, and only one has the filter applied. The plurality of probability distributions includes a fourth probability distribution when neither neighboring unit exists or neither has the filter applied.

[0129] For simplicity of explanation, the technique 1200 of FIG. 12 is depicted and described as a series of steps or operations. However, the steps or operations in accordance with this disclosure can occur in various orders and/or concurrently. Additionally, other steps or operations not presented and described herein may be used. Furthermore, not all illustrated steps or operations may be required to implement a technique in accordance with the disclosed subject matter.

- l- [0130] The aspects of encoding and decoding described above illustrate some examples of encoding and decoding techniques. However, it is to be understood that encoding and decoding, as those terms are used in the claims, could mean compression, decompression, transformation, or any other processing or change of data.

[0131] The word “example” is used herein to mean serving as an example, instance, or illustration. Any aspect or design described herein as “example” is not necessarily to be construed as being preferred or advantageous over other aspects or designs. Rather, use of the word “example” is intended to present concepts in a concrete fashion. As used in this application, the term “or” is intended to mean an inclusive “or” rather than an exclusive “or.” That is, unless specified otherwise or clearly indicated otherwise by the context, the statement “X includes A or B” is intended to mean any of the natural inclusive permutations thereof. That is, if X includes A; X includes B; or X includes both A and B, then “X includes A or B” is satisfied under any of the foregoing instances. In addition, the articles “a” and “an” as used in this application and the appended claims should generally be construed to mean “one or more,” unless specified otherwise or clearly indicated by the context to be directed to a singular form. Moreover, use of the term “an implementation” or the term “one implementation” throughout this disclosure is not intended to mean the same embodiment or implementation unless described as such.

[0132] Implementations of the transmitting station 102 and/or the receiving station 106 (and the algorithms, methods, instructions, etc., stored thereon and/or executed thereby, including by the encoder 400 and the decoder 500) can be realized in hardware, software, or any combination thereof. The hardware can include, for example, computers, intellectual property (IP) cores, application- specific integrated circuits (ASICs), programmable logic arrays, optical processors, programmable logic controllers, microcode, microcontrollers, servers, microprocessors, digital signal processors, or any other suitable circuit. In the claims, the term “processor” should be understood as encompassing any of the foregoing hardware, either singly or in combination. The terms “signal” and “data” are used interchangeably. Further, portions of the transmitting station 102 and the receiving station 106 do not necessarily have to be implemented in the same manner.

[0133] Further, in one aspect, for example, the transmitting station 102 or the receiving station 106 can be implemented using a general-purpose computer or general-purpose processor with a computer program that, when executed, carries out any of the respective methods, algorithms, and/or instructions described herein. In addition, or alternatively, for example, a special-purpose computer/processor can be utilized which can contain other hardware for carrying out any of the methods, algorithms, or instructions described herein. [0134] The transmitting station 102 and the receiving station 106 can, for example, be implemented on computers in a video conferencing system. Alternatively, the transmitting station 102 can be implemented on a server, and the receiving station 106 can be implemented on a device separate from the server, such as a handheld communications device. In this instance, the transmitting station 102, using an encoder 400, can encode content into an encoded video signal and transmit the encoded video signal to the communications device. In turn, the communications device can then decode the encoded video signal using a decoder 500. Alternatively, the communications device can decode content stored locally on the communications device, for example, content that was not transmitted by the transmitting station 102. Other suitable transmitting and receiving implementation schemes are available. For example, the receiving station 106 can be a generally stationary personal computer rather than a portable communications device, and/or a device including an encoder 400 may also include a decoder 500.

[0135] Further, all or a portion of implementations of the present disclosure can take the form of a computer program product accessible from, for example, a computer-usable or computer-readable medium. A computer-usable or computer-readable medium can be any device that can, for example, tangibly contain, store, communicate, or transport the program for use by or in connection with any processor. A medium can be, for example, an electronic, magnetic, optical, electromagnetic, or semiconductor device. Other suitable mediums are also available.

[0136] The above-described embodiments, implementations, and aspects have been described in order to facilitate easy understanding of this disclosure and do not limit this disclosure. On the contrary, this disclosure is intended to cover various modifications and equivalent arrangements included within the scope of the appended claims, which scope is to be accorded the broadest interpretation as is permitted under the law so as to encompass all such modifications and equivalent arrangements.

Claims

What is claimed is:

1. A method, comprising: selecting a probability distribution for coding a flag indicating whether a filter is applied to a filter processing unit of a reconstructed frame, wherein the probability distribution is selected based on whether the filter is applied to neighboring filter processing units of the filter processing unit; entropy coding the flag indicating whether the filter is applied to the filter processing unit; and selectively applying the filter to the filter processing unit based on the flag.

2. The method of claim 1, wherein the filter is a Cross -Component Sample Offset (CCSO) filter.

3. The method of claim 1, further comprising: identifying the neighboring filter processing units based on a predefined search order of sub-blocks within the neighboring filter processing units.

4. The method of claim 3, wherein identifying the neighboring filter processing units based on the predefined search order comprises: identifying a first sub-block of a first neighboring filter processing unit; and identifying a second sub-block of a second neighboring filter processing unit.

5. The method of claim 4, wherein selecting the probability distribution for coding the flag comprises: selecting a first probability distribution in a case that the first neighboring filter processing unit and the second neighboring filter processing unit exist, the first neighboring filter processing unit is different from the second neighboring filter processing unit, the filter is applied to the first neighboring filter processing unit, and the filter is applied to the second neighboring filter processing unit.

6. The method of claim 5, wherein selecting the probability distribution for coding the flag further comprises, at least one of: selecting a second probability distribution in a case that the first neighboring filter processing unit and the second neighboring filter processing unit exist, the first neighboring filter processing unit and the second neighboring filter processing unit are a same neighboring filter processing unit, and the filter is applied to the first neighboring filter processing unit; or selecting a second probability distribution in a case that only one of the first neighboring filter processing unit or the second neighboring filter processing unit exists and the filter is applied to the neighboring filter processing unit that exists.

7. The method of claim 6, wherein selecting the probability distribution for coding the flag further comprises: selecting a third probability distribution in a case that the first neighboring filter processing unit and the second neighboring filter processing unit exist, the first neighboring filter processing unit is different from the second neighboring filter processing unit, and the filter is applied to only one of the first neighboring filter processing unit or the second neighboring filter processing unit.

8. The method of claim 7, wherein selecting the probability distribution for coding the flag further comprises: selecting a fourth probability distribution in a first case where neither the first neighboring filter processing unit nor the second neighboring filter processing unit exists, or in a second case where neither the first neighboring filter processing unit nor the second neighboring filter processing unit has the filter applied.

9. The method of any one of claim 1 to claim 8, further comprising: storing the coded flag for use in filtering subsequent filter processing units within a same frame as the filter processing unit.

10. The method of claim 1, wherein selecting the probability distribution comprises: selecting from a plurality of probability distributions based on states of the neighboring filter processing units.

11. The method of claim 10, wherein the plurality of probability distributions includes: a first probability distribution when both neighboring filter processing units exist, are different units, and both have the filter applied; a second probability distribution when both neighboring filter processing units are a same unit and have the filter applied, or when only one neighboring unit exists and has the filter applied; a third probability distribution when both neighboring filter processing units exist, are different units, and only one has the filter applied; and a fourth probability distribution when neither neighboring unit exists or neither has the filter applied.

12. A device, comprising: a processor that is configured to perform the method of any one of claims 1 to 11.

13. A device, comprising: a memory; and a processor, the processor configured to execute instructions stored in the memory to perform the method of any one of claims 1 to 11.

14. A non-transitory computer-readable storage medium, comprising executable instructions that, when executed by a processor, facilitate performance of operations, comprising operations that perform the method of any one of claims 1 to 11.

15. A non-transitory computer-readable storage medium having stored thereon an encoded bitstream, wherein the encoded bitstream is configured for decoding by the method of any one of claims 1 to 11.

16. A non-transitory computer-readable storage medium having stored thereon an encoded bitstream, wherein the encoded bitstream is generated by an encoder performing the method of any one of claims 1 to 11.