KR102897092B1 - Method for video encoding and decoding based on intra prediction and appartus thereof - Google Patents

Abstract

The present invention relates to an improved encoding/decoding method and device based on IBC (intra block copy) among intra-screen prediction encoding/decoding methods. A method for decoding an encoded video bitstream according to an embodiment of the present invention may include the steps of: reading prediction mode information for prediction decoding of a current block of a picture currently being decoded from the bitstream; determining whether the prediction mode information indicates a mode based on intra block copy (IBC); reading, from the bitstream, at least one prediction vector designating a position of at least one reference block from at least one picture including the picture currently being decoded based on the prediction mode information; obtaining at least one reference block based on the prediction vector; generating a prediction block based on the at least one reference block; and performing prediction decoding on the current block based on the prediction block.

Description

Method and apparatus for video encoding and decoding based on intra-screen prediction

The present invention relates to the field of digital video encoding and decoding, and more particularly, to a method for encoding and decoding digital video, a method for recording such data, and components, devices, and systems for implementing such methods. Specifically, the present invention relates to an improved encoding/decoding method and device based on IBC (intra block copy) among intra-screen prediction encoding/decoding methods.

The present invention may be in the same technical field as at least one of the digital video compression technology standards known by the names of standards such as MPEG-2, MPEG-4 Video, H.263, H.264/AVC, H.265/HEVC, H.266/VVC, VC-1, AV1, QuickTime, VP-9, VP-10, and Motion JPEG, or may be in the technical field for improving the inherent efficiency of the standards, or may be in the technical field for improving or replacing the standards.

Digital video encoding and decoding are widely used in various digital video applications. For example, digital television broadcasting, video transmission over communication networks, video calls/video conversations/video chats, recording and providing video content using optical media including video compact discs (VCDs), digital versatile discs (DVDs), and Blu-Rays, all processes for producing, editing, collecting, and distributing video content, and devices such as video recording devices and camcorders for various purposes, including personal, commercial, industrial, and security purposes, all depend on video encoding and decoding technologies.

Accordingly, implementations that may be referred to as digital video encoders and decoders may form part of a wide range of devices related to the generation, recording, and provision of digital video, including digital televisions, digital broadcasting systems, wireless broadcasting systems, computers in the form of notebooks/desktops/tablets, e-book readers, digital cameras, digital recording devices, digital multimedia playback devices, video game devices/terminals/consoles, mobile phones (including smartphones) with multimedia playback capabilities, equipment for video conferencing, and other devices.

The above digital video encoders and decoders can be implemented by a digital video compression standard that is widely used and understood by those skilled in the art. The digital video compression standard may include at least one of compression standards known by a standard name such as MPEG-2, MPEG-4 Video, H.263, H.264/AVC, H.265/HEVC, H.266/VVC, VC-1, AV1, QuickTime, VP-9, VP-10, and Motion JPEG.

Video encoders and decoders can be implemented to more efficiently encode or decode digital video information while complying with the above standards, or by improving or modifying the above standards. Attempts to modify the above standards can also lead to the development of new standards. A well-known example is the so-called enhanced compression model (ECM), an attempt to improve and replace the existing H.266/VVC standard, currently being developed by the Joint Video Experts Team (JVET), a joint international standardization group of ISO, IEC, and ITU-T.

Versatile Video Coding (VVC), an international standard for video compression, is an intra-frame predictive encoding/decoding method that derives block information from a target block to use reference blocks within the same frame as prediction information for that block. This technology is considered a useful compression tool for computer graphics-processed screen content and is collectively known as intra-block copy (IBC).

Digital video is known to require a significant amount of information to describe its content in its uncompressed state. Therefore, recording or transmitting this information in its original form can be inefficient. Therefore, digital video is compressed using various methods prior to recording or transmission. These compression methods include lossy and lossless encoding. Lossy encoding sacrifices some image quality to achieve high compression performance, while lossless encoding sacrifices some compression performance to prevent image quality degradation. Regardless of the encoding method, there is a need to implement a technology that achieves a high compression ratio while minimizing image quality degradation to meet the growing demand for high-quality digital video within the constraints of limited memory storage capacity and communication transmission bandwidth.

As described above, the encoding process for compression requires various operations, such as spatial segmentation of digital video, segmentation and/or processing in color channels, removal of spatial redundancy, removal of temporal redundancy, tracking of motion vectors within the video, encoding of differential images, quantization, coefficient scan, run-length coding, entropy coding, and loop filtering. These encoding operations generally consume computing resources and take a certain amount of time to complete. Similarly, the decoding operation for the encoding operation also requires certain computing resources and a certain amount of time. The main goal of video encoding and decoding technology is to ensure that the above resource consumption and time consumption do not interfere with the production, recording, distribution, and viewing of digital videos.

Accordingly, the present invention provides a new technology that can contribute to at least one of the following: improvement of encoding efficiency, improvement of decoding efficiency, improvement of video quality, reduction of computational amount, reduction of software size, reduction of hardware size, and improvement of other performances related to encoding and decoding in the technical tasks in the field of video encoding and decoding.

Specifically, the present invention aims to improve the efficiency of an IBC-based encoding/decoding method. In particular, the present invention aims to improve the precision of a prediction vector, for example, a block vector (BV), provide a method for refining a prediction vector through template matching (TM), and enhance prediction accuracy by providing a bi-predictive mode and a combined mode of intra- and inter-screen prediction.

The present invention is one of the intra-screen prediction methods that can be used in the above-described video encoding and decoding areas, and proposes a method of obtaining prediction information of a target block from a position indicated by a block vector within the same screen.

According to an embodiment of the present invention for solving the above-described technical problem, a method for decoding an encoded video bitstream may include the steps of: reading prediction mode information for prediction decoding of a current block of a picture currently being decoded from the bitstream; determining whether the prediction mode information indicates a mode based on intra block copy (IBC); reading, from the bitstream, at least one prediction vector designating a position of at least one reference block from at least one picture including the picture currently being decoded based on the prediction mode information; obtaining at least one reference block based on the prediction vector; generating a prediction block based on the at least one reference block; and performing prediction decoding on the current block based on the prediction block.

The step of reading the prediction vector may include the step of obtaining an adaptive block vector resolution (ABVR) flag value from the bit string, the step of reading precision representation information of the block vector when the ABVR flag value falls within a first range, the step of determining the precision of the prediction vector based on the ABVR flag information and the precision representation information, and the step of reading the prediction vector based on the resolution.

The precision of the above prediction vector may be characterized in that it is determined by at least one precision unit including a sub-pixel unit.

The precision of the above prediction vector may be characterized in that it is determined in units of 1/4 pixels when the ABVR flag value does not fall within the first range, in units of 1 pixel when the ABVR flag value falls within the first range, in units of 1 pixel when the value of the precision representation information is the first value, and in units of 4 pixels when the value of the precision representation information is the second value.

The above first range may be characterized by meaning a case where the ABVR flag value is “1”, and the precision expression information may be characterized by being an index value having at least two cases.

The method further includes a step of determining whether the prediction mode information indicates a mode that uses refinement of a prediction vector, and a step of redefining the prediction vector by compensation if the mode uses refinement, and the step of obtaining the reference block can operate based on the refined prediction vector.

The method may further include a step of dividing the current block into two or more sub-blocks, wherein the step of redefining the prediction vector may be characterized by individually compensating and correcting the prediction vector for each sub-block based on the prediction vector, and the step of obtaining the reference block may be characterized by obtaining two or more sub-reference blocks obtained based on the individually redefined prediction vector for each sub-block, and combining the respective sub-reference blocks to obtain the reference block.

The step of redefining the above prediction vector may be characterized in that it operates based on a template matching method having a search range based on a point pointed to by the above prediction vector.

The step of redefining the above prediction vector may be characterized by performing compensation correction with a higher precision than the precision of the above prediction vector.

The step of redefining the above prediction vector may be characterized by compensating and correcting the prediction vector read in integer pixel units in subpixel units.

The method may further include a step of determining whether the prediction mode information indicates a mode that allows two or more prediction vectors, wherein the step of reading the prediction vector includes a step of reading a first prediction vector and a step of reading a second prediction vector, wherein the step of obtaining the reference block includes a step of obtaining a first reference block based on the first prediction vector and a step of obtaining a second reference block based on the second prediction vector, and wherein the step of generating the prediction block may be characterized in that the step of generating the prediction block generates the prediction block through prediction fusion based on two or more reference blocks including the first reference block and the second reference block.

The first prediction vector and the second prediction vector are block vectors for intra-screen prediction, and the first prediction vector and the second prediction vector may be configured to indicate the positions of different reference blocks within an already decoded area of the current picture.

The first prediction vector may be a block vector for intra-screen prediction and configured to indicate the position of the first reference block within an already decoded area of the current picture, and the second prediction vector may be a motion vector for inter-screen prediction and configured to indicate the position of the second reference block within a previously decoded picture.

The above prediction fusion may be performed by a weighted sum or weighted average performed by a combination of weights, and information about the weights may be obtained from the bit string.

According to an embodiment of the present invention for solving the above-described technical problem, a method for encoding a moving picture to generate a bit stream may include the steps of: determining at least one prediction vector that designates a position of at least one reference block from at least one picture including the picture currently being decoded, for a current block of the picture currently being encoded; obtaining at least one reference block based on the prediction vector; generating a prediction block based on the at least one reference block; performing prediction encoding on the current block based on the prediction block; determining a prediction mode for the current block based on a result of the prediction encoding as a mode based on intra block copy (IBC); generating a bit stream syntax representing the prediction mode and the at least one prediction vector based on the prediction encoding mode; and recording the bit stream syntax in the bit stream.

The above prediction vector is determined by at least one precision including a sub-pixel unit, and the step of generating the bit string syntax may include the step of generating a bit string syntax representing a unit of the precision based on an adaptive block vector resolution (ABVR) flag value and precision representation information of the block vector, and the step of generating a bit string syntax representing the prediction vector based on the unit of the precision.

The method may further include a step of compensating the prediction vector, wherein the step of obtaining the reference block is characterized by operating based on the compensated-corrected prediction vector, the step of determining the prediction mode is characterized by determining the prediction mode as a mode that uses refinement of the prediction vector, and the step of generating the bit string syntax may be characterized by generating a bit string syntax representing the prediction vector before the compensation correction.

The step of determining the prediction vector may include a step of determining a first prediction vector and a step of determining a second prediction vector, the step of obtaining the reference block may include a step of obtaining a first reference block based on the first prediction vector, and a step of obtaining a second reference block based on the second prediction vector, the step of generating the prediction block may be characterized in that the prediction block is generated through prediction fusion based on two or more reference blocks including the first reference block and the second reference block, and the step of determining the prediction mode may be characterized in that the prediction mode is determined as a mode that allows two or more prediction vectors.

The first prediction vector may be a block vector for intra-screen prediction and configured to indicate the position of the first reference block within an already encoded region of the current picture, and the second prediction vector may be a motion vector for inter-screen prediction and configured to indicate the position of the second reference block within a picture whose encoding has been completed previously.

According to an embodiment of the present invention for solving the above-described technical problem, a decoder device configured to decode a video bit stream encoded by a computer device comprises: a processor, a memory, an input unit for inputting the bit stream, an output unit for outputting a decoded video, a reference buffer for storing information of at least one decoded picture including a picture currently being decoded, a bit stream parsing unit having a function of reading prediction mode information for prediction decoding of a current block of the picture currently being decoded from the bit stream, determining whether the prediction mode information indicates a mode based on intra block copy (IBC), and reading at least one prediction vector designating a position of at least one reference block from at least one picture based on the prediction mode information, and obtaining at least one reference block from at least one picture stored in the reference buffer based on the prediction vector. The apparatus may include a prediction decoding unit, which includes a function of generating a prediction block based on at least one reference block, and performing prediction decoding on the current block based on the prediction block, and a video decoding unit configured to decode the bit string based on a result of the prediction decoding.

The present invention provides an improved invention for an encoding/decoding method based on IBC, which is one of the intra-screen prediction methods, and is a method of obtaining prediction information of a target block from a position indicated by a block vector within the same screen. According to the present invention, through a method of calculating and transmitting block vector information of a block to be decoded and a method of redefining the corresponding block vector during the decoding process, more accurate motion information can be calculated, thereby obtaining the effect of improving the compression performance of the target block.

More specifically, according to the present invention, by refining the resolution of a prediction vector, for example, a block vector (BV), to a sub-pixel level, it has the effect of enabling more precise prediction than the existing integer pixel level. In addition, by applying a template matching-based prediction vector redefinition method to both the IBC skip/merge mode and the IBC AMVP mode, it has the effect of improving the accuracy of the prediction vector during the decoding process, thereby enhancing the prediction performance. In addition, through a sub-block level prediction vector redefinition method, it has the effect of enabling more precise prediction that reflects local characteristics within a block. In addition, by introducing the concept of bi-predictive prediction to IBC, it has the effect of providing improved prediction performance than uni-predictive prediction through weighted prediction using two reference blocks. In addition, through an extended bi-predictive method that combines IBC-based intra-picture prediction and inter-picture prediction, it has the effect of enabling more effective prediction by simultaneously utilizing information from the current picture and reference pictures.

According to the present invention, based on the technical effects described above, there is an effect of improving the overall efficiency of the IBC-based encoding/decoding method, and achieving more effective encoding, especially for special images such as screen contents.

Figure 1 is a conceptual diagram of a video communication system according to one embodiment of the present invention;
FIG. 2 is a conceptual diagram of the arrangement of an encoder and decoder in a real-time video streaming environment according to one embodiment of the present invention.
Figure 3 is a functional unit conceptual diagram of a video decoder according to one embodiment of the present invention;
Figure 4 is a functional unit conceptual diagram of a video encoder according to one embodiment of the present invention;
Figure 5 is a conceptual diagram of a frame type according to one embodiment of the present invention;
Figure 6 is a conceptual diagram showing the structure of a video encoder according to another embodiment of the present invention.
Figure 7 is a conceptual diagram of IBC (intra block copy) according to one embodiment of the present invention.
FIG. 8 is a conceptual diagram for redefining a prediction vector based on template matching according to one embodiment of the present invention.
FIG. 9 is a flowchart showing a process of redefining a prediction vector according to one embodiment of the present invention;
FIG. 10 is a conceptual diagram for sub-block-based prediction vector redefinition according to one embodiment of the present invention;
Figure 11 is a conceptual diagram showing a prediction method using bidirectional prediction according to one embodiment of the present invention.
FIG. 12 is a flowchart for processing directional flag information according to one embodiment of the present invention, and
Fig. 13 is a conceptual diagram illustrating a prediction method using extended bidirectional prediction according to one embodiment of the present invention.

The present invention is susceptible to various modifications and embodiments. Specific embodiments are illustrated and described in detail in the drawings. However, this is not intended to limit the present invention to specific embodiments, but rather to encompass all modifications, equivalents, and alternatives falling within the spirit and technical scope of the present invention.

While terms such as "first" and "second" may be used to describe various components, these components should not be limited by these terms. These terms are used solely to distinguish one component from another. For example, without departing from the scope of the present invention, the first component could be referred to as the "second component," and similarly, the second component could also be referred to as the "first component." The term "and/or" includes any combination of multiple related items described herein or any of multiple related items described herein, and is non-exclusive unless otherwise indicated. The listing of items in this application is merely an exemplary description to facilitate the spirit and possible implementation methods of the present invention, and therefore is not intended to limit the scope of embodiments of the present invention.

As used herein, "A or B" can mean "only A," "only B," or "both A and B." In other words, as used herein, "A or B" can be interpreted as "A and/or B." For example, as used herein, "A, B or C" can mean "only A," "only B," "only C," or "any combination of A, B and C."

As used herein, a slash (/) or a comma can mean "and/or." For example, "A/B" can mean "A and/or B." Accordingly, "A/B" can mean "only A," "only B," or "both A and B." For example, "A, B, C" can mean "A, B, or C."

In this specification, "at least one of A and B" may mean "only A", "only B" or "both A and B". Additionally, in this specification, the expressions "at least one of A or B" or "at least one of A and/or B" may be interpreted identically to "at least one of A and B".

Additionally, in this specification, “at least one of A, B and C” can mean “only A,” “only B,” “only C,” or “any combination of A, B and C.” Additionally, “at least one of A, B or C” or “at least one of A, B and/or C” can mean “at least one of A, B and C.”

When a component is referred to as being "connected" or "connected" to another component, it should be understood that it may be directly connected or connected to that other component, but that there may be other components intervening. Conversely, when a component is referred to as being "directly connected" or "connected" to another component, it should be understood that there are no other components intervening.

The terminology used in this application is only used to describe specific embodiments and is not intended to limit the present invention. The singular expression includes the plural expression unless the context clearly indicates otherwise. In this application, it should be understood that the terms "comprise" or "have" indicate the presence of a feature, number, step, operation, component, part, or combination thereof described in the specification, but do not exclude in advance the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by those of ordinary skill in the art to which this invention pertains. Terms defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning within the context of the relevant technology, and shall not be interpreted in an idealized or overly formal sense unless explicitly defined herein.

In describing the invention in this application, embodiments may be described or illustrated in terms of unit blocks that perform the described function or functions. The blocks may be expressed in this application as one or more devices, units, modules, parts, etc. The blocks may be implemented in hardware by one or more logic gates, integrated circuits, processors, controllers, memories, electronic components, or information processing hardware implementation methods, but not limited thereto. Alternatively, the blocks may be implemented in software by application software, operating system software, firmware, or information processing software implementation methods, but not limited thereto. A single block may be implemented by being separated into multiple blocks that perform the same function, or conversely, a single block may be implemented to perform the functions of multiple blocks simultaneously. The blocks may also be implemented by being physically separated or combined according to any criteria. The blocks may also be implemented to operate in an environment where their physical locations are not specified and are separated from each other by a communication network, the Internet, a cloud service, or a communication method, but not limited thereto. All of the above implementation methods are within the realm of various embodiments that can be taken by a person skilled in the field of information and communication technology to implement the same technical idea, and therefore, any detailed implementation method should be interpreted as being included within the technical idea scope of the invention of the present application.

Hereinafter, preferred embodiments of the present invention will be described in more detail with reference to the attached drawings. To facilitate a comprehensive understanding of the present invention, the same reference numerals will be used for identical components in the drawings, and redundant descriptions of identical components will be omitted. Furthermore, the multiple embodiments are not mutually exclusive, and it is assumed that some embodiments may be combined with one or more other embodiments to form new embodiments.

digital video codec

Figure 1 is a conceptual diagram of a video communication system according to one embodiment of the present invention. The video communication system (100) may be configured to include at least two terminals (110, 120) connected to each other via a network (105).

In one embodiment of the present invention, the above-described FIG. 1 may refer to a block diagram for configuring a unidirectional video communication network. A first terminal (110) among the terminals may encode video data in order to transmit (111) the video data via a network (105). A second terminal (120) among the terminals may be configured to receive (121) the encoded video data via a network and decode and display the same.

In another embodiment of the present invention, the above-described FIG. 1 may refer to a block diagram for configuring a two-way video communication network. For the two-way video communication, each terminal (110, 120) may be configured to encode video data acquired by itself for video transmission (112, 122) to each other terminal via the network. Each terminal may also be configured to receive (113, 123) video data transmitted by another terminal via the network, decode the same, and display the decoded video data.

The terminals (110, 120) shown in Fig. 1 may be exemplified as devices such as server computers, personal computers, portable computers, and smartphones, depending on the embodiment, but are not limited thereto. The present invention is applicable to all environments for establishing a one-way or two-way video communication network, and it should be understood that the network (105) may be established by any means for transporting encoded video data between the terminals (110, 120).

In one embodiment of the present invention, the network (105) may refer to a wired or wireless communication network. Depending on the embodiment, the network may be configured to communicate information using any communication standard, which may include packet-based communication. The packet communication may be understood to include packets, for example, known as TCP or UDP.

However, in another embodiment of the present invention, the network (105) may be understood to include a process of information transmission using a recording medium. In this case, the configuration of the network is not limited to a communication medium, and should be understood to include a process of temporarily storing and physically transporting information on a hard disk, a solid state disk (SSD), a flash memory, a compact disc (CD), a digital versatile disc (DVD), a Blu-ray disc, and other mechanical, electronic, or optical recording media.

Any other means of information communication or transport, regardless of the method employed, can be considered within the scope of the present invention as long as it has a structure that supports the transmission and decoding of video data in an encoded state. Therefore, in addition to the examples listed above, any means of information communication or transport, whether known in the past or newly available, can fall within the scope of application of the present invention.

FIG. 2 is a conceptual diagram illustrating the arrangement of an encoder and decoder in a real-time video streaming environment according to one embodiment of the present invention. The streaming system (200) illustrated in FIG. 2 can be applied to video data communication networks, including, for example, digital broadcasting, video telephony, and video conferencing. However, it should be noted that technical structures identical or similar to the streaming system can be equally applied even when information is transmitted via a recording medium, as described above.

According to one embodiment of the present invention, the streaming system may include a video source (210) that generates a video stream. The video source may include a digital video acquisition means (212), which may be, for example, a digital camera or other device, for acquiring uncompressed raw video. The raw video stream (215) may have a large capacity and may therefore be compressed by a video encoder (217) coupled or connected to the video source.

The above encoder (217) may be configured as a means including hardware, software, or a combination of the two, configured to implement an image encoding method and/or an implementation method thereof according to one embodiment of the present invention.

Through the encoder (217), an encoded bitstream (219) having a reduced capacity compared to the original video stream can be output. The bitstream (219) can be provided in real time via a relay device, which may be referred to as a streaming server (220), and/or can be stored in a recording medium (225) of the streaming server (220) for subsequent use.

The streaming system (200) may include at least one streaming client (230, 240) that connects to the streaming server (220) to receive the encoded bit string (229) in real time or obtain it later. The streaming client may include a video decoder (232) that obtains the encoded bit string (229) (which may also be considered as a copy of the bit string (219) received by the streaming server), decodes the bit string (229), and outputs the resulting video data as video data in a form that can be displayed by a display (235) or other visual, auditory, or other sensory display means.

As described above, the functions for encoding and decoding video data are collectively called a coder-and-decoder, or video codec.

FIG. 3 is a conceptual diagram of a functional unit of a video decoder according to an embodiment of the present invention. As shown in FIG. 3, a receiving unit (310) can receive at least one encoded video data to be decoded by a decoder (305). In an embodiment of the present invention, the encoded video data may be independent for each reception, and the decoding procedure of each independent video data may be independent from the decoding procedure of other video data. The encoded video data may be received by the receiving unit (310) through a hardware or software connection (315) to a device storing the same, and as described above, the storing device may be a type of streaming server located at the other end of a communication network, or may mean a physical recording medium, but is not limited thereto.

The above-described receiving unit (310) can receive the encoded video data together with other data accompanying it, such as encoded audio data or other auxiliary data, and each of the data can be separated from the video data and provided to an appropriate processing function unit (312) other than the video decoder.

When the video data is provided through a communication network, a buffer memory (320) may be coupled between the receiving unit (310) and the decoder (305) to minimize delay and disconnection according to the network environment. The buffer memory (320) may refer to a computer-readable recording medium that temporarily stores the received video data and stably supplies it to a parser (330) corresponding to the input terminal of the decoder (305). However, if the bandwidth of the communication network is sufficient, if video data is read from a recording medium in a local location that is not physically separated, or if the possibility of communication delay is not predicted in other environments, the buffer memory may be unnecessary.

The video decoder (305) may include the parser (330) as its input terminal to interpret the encoded video data. The parser may perform a function of separating (parsing) a plurality of pieces of information stored in the form of a bit string in the encoded video data according to a predetermined rule, and, if necessary, performing an entropy decoding (335) of entropy-coded video data, thereby performing a function of reconstructing symbols (338), which are paragraphs of video encoding information. The symbols (338) may include all information for controlling the operation of the decoder (305), and/or may further include information for controlling a device that is attached to and operable with the decoder (305), such as a display device. Control information for controlling the above display device may include information in a format called supplementary enhancement information (SEI) or video usability information (VUI).

As described above, the parser (330) may be configured to perform entropy decoding (335) of the encoded video data. The entropy encoding method of the encoded video data may vary depending on the encoding standard, and decoding may be performed accordingly. Representative examples of the entropy encoding standard may include variable length coding, Huffman coding, and arithmetic coding, and each of the encoding methods may be a context-adaptive or context-sensitive method depending on the standard, and may also be based on principles widely known to those skilled in the art.

The parser (330) may be configured to extract at least one picture from the encoded video data. The definition of the picture may vary depending on the encoding standard, and depending on the standard, one or more of the examples listed below may correspond simultaneously and overlappingly. The picture may be grouped, defined, and/or divided into encoding/decoding units such as, for example, groups of pictures (GOPs), pictures/frames, tiles, slices, macroblocks, blocks, subblocks, transform units (TUs), and prediction units (PUs).

The parser (330) may be configured to extract encoding information, such as transform coefficients, quantization parameters (QPs), and/or motion vectors, from the encoded video data. The parser (330) may be configured to perform entropy decoding (335) and parsing operations on the video data received from the buffer memory, and to selectively decode symbols (338) representing the encoding information. In addition, the parser (330) may be configured to selectively supply a specific symbol (338) to a specific decoding function unit within the decoder (305), such as an inverse quantization and inverse transform unit (340), an intra prediction unit (350), an inter prediction unit (355), or a loop filter unit (360). Control of such information supply can be determined by the information sequence contained in the encoded video, and may vary depending on the encoding standard, and is not limited within the scope of the embodiments of the present invention, and is not described in detail in this conceptual diagram.

The decoder (305) may be comprised of a number of conceptual functional units that receive and process the encoded information provided by the parser (330). It should be readily apparent that these conceptual functional units may be combined or further subdivided, depending on implementation needs. For example, they may be further separated for ease of implementation, or integrated into one for operational efficiency. In any case, each functional unit may be configured to perform close interaction with each other. However, despite the possibility of such integration or separation, the following description will be given as a combination of conceptual functional units to illustrate the video data decoding procedure applied as an embodiment of the present invention.

The decoder may include an inverse quantization and inverse transformation unit (340). The inverse quantization and inverse transformation unit (340) may be configured to receive encoding information including a method to be used for numerical transformation (transform), a block size, quantization coefficients for recovering quantized information, and distinction information of a quantization matrix that simplifies and represents the quantized coefficients from the parser (330), and may be configured to output block values (341) that can be input to an aggregator (370) as a result of processing the encoding information.

In one embodiment of the present invention, the output values of the inverse quantization and inverse transformation unit (340) may include intra-prediction encoded block values. The intra-predicted block values may refer to values that can be decoded using prediction information within a picture currently being decoded, for example, the current frame, but not using prediction information from a previously decoded picture, for example, the previous frame.

The prediction information within the current picture may be provided by the intra prediction unit (350). According to an embodiment of the present invention, the intra prediction unit (350) generates a block value of the same form as the block being decoded as the prediction information by using picture information of a spatially adjacent area derived from a picture currently being decoded and of which decoding has been partially completed. The picture information may be provided (381) from a buffer for the current picture, a so-called line buffer (380). The merging unit (370), according to an embodiment, may be configured to merge the prediction information (351) generated by the intra prediction unit (350) with the block values (341) provided by the inverse quantization and inverse transformation unit (340).

In another embodiment, the output values of the inverse quantization and inverse transformation unit (340) may include block values subjected to motion compensation as inter-prediction encoded block values, and in some cases, block values subjected to motion compensation. In this case, the inter-prediction unit (355) may extract and use sample information (386) used for motion-based prediction from a reference picture buffer (385). The information (356) derived by performing motion compensation on the sample information based on the symbols (338) included in the block values as the output values may be configured to be merged with the block values (341) provided by the inverse quantization and inverse transformation unit (340) by the merger unit (370). In this case, the block values (341) may be referred to as so-called differential or residual values.

The position information within the memory used by the inter prediction unit (355) to extract the sample information from the reference picture may be determined by a motion vector provided to the inter prediction unit (355) and composed of a combination of symbols (338) for indicating, for example, X, Y, and other specific points of the reference picture. The inter prediction unit (355) may also include a function for interpolating and using the sample values when a so-called 'subsampling'-capable motion vector is provided, and may further include a function for predicting and reinforcing the value of the motion vector.

The output values (371) of the above merging unit (370) may be provided to the loop filter unit (360) and processed by various loop filtering methods. The loop filter unit (360) may be configured to receive not only the block unit output (371) of the merging unit (370) but also the symbol (338) provided from the parser (330) to control its operation. The output of the loop filter unit (360) may be output to an external display means such as the display device through an output connection (390), but may be stored (361) in a line buffer (380) for use in prediction for interpreting a subsequent intra or inter coded block value, and may also be stored in a reference picture buffer (385) through this.

Certain pictures, such as frames, after their decoding is completed, can be utilized as reference pictures for performing predictive decoding in a subsequent decoding process. A picture (or frame) can be gradually accumulated in a line buffer (380) and decoded. When the decoding of a frame is completed, the contents of the line buffer (380) are transferred (383) to the reference picture buffer (385), and a new line buffer (380) can be allocated for decoding the new frame.

The above video decoder (305) may be configured to perform a decoding operation according to a predetermined video compression technique that may be documented by various international standards or commercial standards. The standards may include, for example, international standard recommendations such as H.264, H.265, and H.266 defined by the International Telecommunication Union Standardization Sub-Division (ITU-T). Those skilled in the art will understand that each of the above recommendations is equivalent to an international standard jointly defined by the International Organization for Standardization (ISO) and the International Electrotechnical Commission (IEC). The encoded video data may comply with a specific bitstream syntax defined by the relevant standards, as defined and required by the video compression standard documents and standard documents, and specifically by the profiles and levels specified within such documents. In addition, the complexity of the encoded video data may be limited to a certain level to comply with the profiles and levels. For example, a profile or level may be configured to limit a maximum picture size, a maximum decoding speed, and a maximum reference picture size. These limitations may, in some embodiments, also be further limited by metadata signals for a hypothetical reference decoder (HRD) and HRD buffer management included in the encoded video data.

According to one embodiment of the present invention, the receiver (310) may receive additional redundant data along with the encoded video. The additional data may be considered as part of the encoded video data. The additional data may include information that may be used by the decoder (305) to properly decode the data or to more accurately reconstruct an image that approximates the original image. The additional data may be provided in the form of, for example, layers for temporal, spatial, or signal-to-noise ratio (SNR) enhancement, redundant slices, redundant pictures, and forward error correction codes.

Figure 4 is a functional unit conceptual diagram of a video encoder according to one embodiment of the present invention. The encoder (405) may be configured to receive original video information (402) from a video source (401) and perform encoding.

The original video information (402) may have any suitable bit depth, for example, 8 bits, 10 bits, 12 bits, etc. In addition, the original video information (402) may have any suitable color space, for example, R/G/B, Y/U/V, Y/Cb/Cr, etc. In addition, the original video information (402) may have any suitable sampling structure corresponding to the color space, for example, Y/Cb/Cr 4:2:0, Y/Cb/Cr 4:4:4, etc. The original video information (402) having such a predetermined format may be provided to the encoder in the form of a digital video stream.

In a one-way video communication network, the original video information (402) can be obtained from a recording medium storing a previously prepared video source. In a two-way video communication network, the original video information (402) can be obtained from an image acquisition device, such as a camera, that generates at least one video transmission stream included in the two-way video communication.

The video data including the above original video information (402) may be configured as a plurality of pictures configured to simulate motion by playing them in chronological order. In addition to pictures, the pictures may also be expressed in concepts such as frames. The pictures may include one or more samples depending on the type of sampling structure, color space, etc. being used. Those skilled in the art will understand that the samples are closely related terms to pixels in digital images. The operation of the encoder will be described below with reference to such samples.

According to one embodiment of the present invention, the encoder (405) may be configured to encode and compress pictures (and/or grouped or segmented information) constituting the original video information (402) in real time (or according to other temporal requirements required according to the implementation method) into the form of encoded video information.

In the encoder (405), the control unit (450) may be a functional unit configured to control an appropriate encoding speed. The control unit (450) may be configured to control other functional units and be functionally coupled to the following functional units as described below. The parameters set by the control unit (450) may include parameters related to bitrate control, such as picture skip, quantizer, and variable values for applying image quality optimization techniques, and may also include values such as picture size, group of pictures (GOP) structure, and maximum search range of motion vectors. A person skilled in the art will be able to understand various other functions that the control unit (450) may have, and such other functions may be added or removed according to the design of a video encoder optimized for individual system design.

According to an embodiment of the present invention, the encoder (405) may be configured to operate in a structure such as a "coding loop" well known to those skilled in the art. To simplify the description by way of example, the coding loop may be configured with an internal encoder (so-called "source coder") (410) which is responsible for receiving a picture to be encoded and generating symbols based on at least one reference picture that has been encoded in the past, and a local decoder (420) configured to be connected to the internal encoder. The local decoder (420) may be configured to perform an operation to reproduce sample data to be generated by a decoder (490) located at an actual remote location that will receive encoded video information from the encoder (405) by receiving an output of the internal encoder (410).

Video data composed of sample data reconstructed by the internal decoder (420) may be configured to be input to the reference picture buffer of the encoder (405). As described above, the internal decoder (420) is implemented to reproduce the result output by the encoder (405) and to be decoded by a remote decoder, so the video data recorded in the reference picture buffer may also be identical in bit unit to the information of the reference picture buffer of the remote decoder. That is, the prediction function unit that may be included in the encoder (405) may read the same values as the sample values of the previous frame that the decoder will later refer to in the decoding process from the reference picture buffer of the encoder (405).

As described above, the principle of achieving matching of the reference picture buffers between the encoder (405) and the decoder (490) by means of the internal decoder (420) on the encoder (405) side is well known to those skilled in the art, and a method of responding to an environment in which such an environment is not guaranteed (e.g., information loss due to communication failure, etc.) can also follow what is known to those skilled in the art.

An embodiment of the operation method of the internal decoder (420) has been described in detail above with reference to FIG. 3. The decoder of FIG. 3 may be regarded as the aforementioned "remote" decoder (490). The internal decoder (420) may be implemented excluding lossless encoding and decoding sections such as the parser (330) or entropy decoding (335). This is because the internal encoder (405) is implemented to simply reproduce the operation of a decoder located at a remote location, and thus may directly decode symbols without requiring a process of compressing and then decompressing symbols. Accordingly, the functional units preceding the parser and entropy decoder as shown in FIG. 3 may not be provided or may be implemented at least partially.

As described above, according to a preferred embodiment of the present invention, any decoder function (excluding a parser and an entropy decoder) present in the decoder can naturally exist as a substantially identical function in the corresponding encoder (405).

The operation of the encoding function unit that may be included in the above encoder (405) can be considered as the reverse operation of the decoder function unit. Therefore, the embodiment can be explained by performing the operation of the decoder function unit in reverse. For example, a quantization and transform function unit corresponding to the inverse quantization and inverse transform unit may be provided, and an inter prediction encoding unit corresponding to the inter prediction unit may be provided. In addition, some additional explanations will be added.

The internal encoder (410) may be configured to perform encoding on input picture information, for example, an input frame, by a prediction encoding method executed by a prediction encoding unit (440) that operates by referencing at least one reference picture information, for example, at least one temporally previous encoded picture (or frame) from a reference picture buffer (430) from video data designated as a reference frame. In this case, the encoder (405) may be configured to encode a differential between blocks of samples constituting the input picture and blocks of samples constituting the reference picture.

The internal decoder (420) can decode video data that can be designated as the reference picture from symbols generated by the internal encoder (410). As described above, since the video data is identical to the decoding operation performed by the remote decoder, the video data used as the reference picture may be provided to the encoder (405) in a form that has undergone lossy compression and has been partially damaged, and this operation may be intended to ensure operational consistency with the decoder.

The prediction encoding unit (440) may be configured to perform a prediction search operation within the encoder (405). The prediction search operation may refer to an operation corresponding to the inter prediction or intra prediction described in the description of the decoder. For picture information that is input and scheduled to be newly encoded, the prediction unit may access the reference picture buffer (430) to retrieve information such as a motion vector, a block shape, and metadata that may include the same, which are information indicating a point of a reference picture that can function as prediction reference information suitable for the new picture information, and a sample block to be actually referenced. The above prediction encoding unit (440) may operate on the basis of the so-called "sample block by pixel block" criteria in order to obtain appropriate prediction reference information. According to one embodiment of the present invention, at least one prediction reference information designating at least one reference picture information stored in the reference picture buffer (430) may be designated for the input picture, as determined based on the search results obtained by the prediction encoding unit (440).

In one embodiment of the present invention, the control unit (450) may be configured to manage the overall encoding operation of the internal encoder (410), including setting parameters used to encode video data.

All outputs of the above-described functional units may be subjected to entropy encoding (460) in order to be finally output. The entropy encoding (460) may include various entropy coding techniques, such as variable length coding, Huffman coding, and arithmetic coding, for the symbols generated by the various functional units as described above, and each encoding method may be a context-adaptive or context-sensitive method according to the standard, or may be based on principles widely known to those skilled in the art. Such entropy encoding (460) can typically achieve lossless compression, and thus can be configured to convert at least one symbol generated by the functional units into encoded video data.

The above control unit (450) may, when controlling the operation of the encoder (405), apply to each picture (or frame) the type of encoding in which a specific picture is encoded during the encoding period. Depending on the type, the method by which the picture is encoded may be affected. Depending on the embodiment, the type may include what is categorized as the following "frame type."

Fig. 5 is a conceptual diagram of a frame type according to one embodiment of the present invention. The following description will be made with reference to Fig. 5.

An intra (“I”) picture (510) may refer to a picture that can be encoded and decoded using only its own information without referring to other picture information in video data through predictive encoding. The “I” picture may be designated by names such as a key frame, an independent/instantaneous decoder referh (IDR) frame, and a clean random-access (CRA) frame, depending on the video encoding standard, and the “I” pictures designated by the various names as described above may have various modifications and application methods as permitted by each standard and may be partially different from each other. In addition to those listed above, various application methods for implementing the “I” picture may be by various methods that are already known to those skilled in the art or may be newly provided.

A prediction ("P") picture (520) may refer to a picture that can be encoded and decoded through intra or inter prediction based on at least one prediction information and/or a motion vector that designates at least one reference picture to predict sample values of a block constituting the picture. The "P" picture may be configured to refer to only one reference frame, or may be configured to refer to one or more reference frames, according to a video encoding standard. When referring to more than one reference frame, sample information and/or associated metadata derived from multiple reference pictures may be used to reconstruct a single block. However, in common cases, a picture designated as a "P" picture may be understood as a picture that performs reference only to a temporally preceding picture.

A bidirectional prediction ("B") picture (530) may refer to a picture that can be encoded and decoded through intra or inter prediction based on at least one piece of prediction information and/or a motion vector that designates at least two reference pictures in order to predict sample values of blocks constituting the picture. In a common case, a picture designated as the "B" picture is distinguished from a picture designated as the "P" picture, and may be understood as a picture that performs a reference without being limited to a temporally preceding picture.

Video data may be spatially divided into a plurality of sample blocks during the encoding and decoding process, and encoding may be performed in units of the blocks. The block units may include, but are not limited to, sizes such as 4x4, 8x8, 4x8, or 16x16 in units of horizontal/vertical pixels, as is widely known. The block may be encoded using a predictive encoding method with reference to any other (already encoded) blocks, as permitted and/or restricted by the type specified for each picture in which the block is included. For example, the blocks of the "I" picture (510) may be encoded without using a predictive encoding method, or with reference to blocks that have already been encoded within the same partial picture. That is, only the so-called intra prediction method may be used. In contrast, the "P" picture (520) may further reference a reference picture encoded in at least one previous time unit, and thus, inter prediction may also be used for encoding along with intra prediction. In the case of a "B" picture (530), reference can be made not only to a previously encoded picture in the encoding order but also to a later reference picture in terms of time unit. However, it is widely known that there may be blocks within a "P" picture or a "B" picture that are encoded without relying on predictive encoding.

The above video encoder (405) may be configured to perform encoding operations according to a predetermined video compression technique that may be documented by various international standards or commercial standards. Examples of the above standards may include all of those described in the above decoder.

According to one embodiment of the present invention, the transmitter (470) may buffer the encoded video data generated by the entropy encoding in order to provide/transmit the video data (ultimately to a remote decoder (490)) to a device storing the encoded video data via a hardware or software connection (495). According to an embodiment, when providing/transmitting the encoded video data from the video encoder (405), the transmitter (470) may receive and merge other data accompanying the encoded video data, for example, encoded audio data or other auxiliary data, from a separate source (480).

According to one embodiment of the present invention, the transmitter (470) may be configured to transmit additional data along with the encoded video. The additional data may be considered part of the encoded video data. The additional data may include information that can be used by a decoder to properly decode the data or to more accurately reconstruct an image that approximates the original image. Examples of the additional data may include all of the examples previously presented with respect to the receiver (310) of the decoder.

The present invention can be implemented by a digital video compression standard that is widely used and understood by those skilled in the art as described above. The digital video compression standard may include at least one of compression standards known by the standard name such as MPEG-2, MPEG-4 Video, H.263, H.264/AVC, H.265/HEVC, H.266/VVC, VC-1, AV1, QuickTime, VP-9, VP-10, and Motion JPEG.

Fig. 6 is a conceptual diagram illustrating the structure of a video encoder according to another embodiment of the present invention. What is depicted in Fig. 6 may be a rough structure of a video encoder widely known as a standard code such as ITU-T H.266 and ISO/IEC 23090-3, and also known as MPEG-I Part 3 or versatile video coding (VVC).

According to FIG. 6, a video encoder (605) may be configured to receive uncompressed and unencoded original video data (601) as input and output an encoded bit stream (602). The video data (601) may be supplied directly to a luma mapping unit (610a) when intra-encoded, or may be supplied to a luma mapping unit (610b) via an inter-prediction unit (620) including motion vector extraction. In the intra-encoded case, the mapped luma signal may be supplied to an output merger (606) by selecting (608) at least one of an intra-prediction encoded signal via an intra-prediction unit (625) or an inter-prediction encoded signal output from the luma mapping unit (610b) via the inter-prediction unit (620). The result of the above output merger can be applied to a chroma scaling unit (615). (The operation of the luminance signal mapping unit (610) and the operation of the chroma scaling unit (615) are collectively referred to as a luma mapping/chroma scalaing (LMCS) process.) The reduced chroma signal can be provided to a transform unit (630), and the transform unit (630) can perform an adaptive color transform, particularly on the chroma signal. The coefficients derived as a result of the transform are applied to a quantization unit (640) and quantized. As a result, lossy compression is achieved, and the result of the lossy compression can be output as a bit string (602) through a multi-hypothesis CABAC (650), which is a lossless compression method.

Meanwhile, the result of the lossy compression may actually enter the decoding process by going through the processes of inverse quantization (645), inverse transform (635), and luminance signal expansion (617) to generate a coding loop. The result of the luminance signal expansion may be supplied to the internal merger (607) together with the result of selecting (608) at least one of the previously generated intra prediction encoding signal or inter prediction encoding signal. The result of the internal merger may go through inverse luma mapping (617), and then may go through processing such as a deblocking filter (660), sample adaptive offset (SAO) (670), and an adaptive loop filter (ALF) to reproduce the image quality improvement process in the decoder. The result of reproducing the operation in the decoder as described above is applied to the reference picture buffer (690) and can be reused for prediction encoding by the inter prediction unit (620).

The present invention can also be utilized by or incorporated into an enhanced compression model (ECM), which is an implementation of a next-generation video codec currently being developed by the Joint Video Experts Team (JVET), an international standardization expert organization. The enhanced compression model can include an enhanced intra prediction coding method, an enhanced inter prediction coding method, an enhanced transform and transform coefficient coding method, an enhanced adaptive loop filtering method, a bilateral filtering method, a new sample adaptive offset (SAO) method for improving image quality, an extended entropy coding method, and an improved gradual decoding refresh (GDR) technique.

Composition of the present invention

Fig. 7 is a conceptual diagram of IBC (intra block copy) according to one embodiment of the present invention. Referring to Fig. 7, the IBC technology can be configured to obtain prediction information of a block to be encoded/decoded (current block) (710) from a reference block (720) belonging to an area (790) in which decoding has already been completed within the same screen (picture). Prediction vector information indicating the position of the reference block (720) within the same picture can include a block vector (BV) (730).

However, the prediction vector shown in the present invention is not limited to the block vector (BV) for intra-screen prediction as described above, and the present invention can be applied to application technologies based on motion vectors (MV) for inter-screen prediction in the same or similar manner, and, as described below, it should be understood that it can also be utilized in cases where the BV and MV are used simultaneously. In addition, the present invention does not exclude the possibility that a method based on an embodiment of the present invention may be applied in the same or similar manner in cases where the MV and/or the BV are not explicitly defined in the bitstream syntax but are implicitly derived by the decoder, when a chained vector that is determined by sequentially referencing the MV and/or the BV is used, or when any type of prediction vector is used by any other method.

Although the IBC compression method is an intra prediction method, the method of transmitting the motion information by deriving the motion information of the block to be decoded can be configured similarly to the inter prediction method. Therefore, according to one embodiment of the present invention, the method of executing the IBC compression can be specifically divided into a skip/merge mode (IBC skip/merge mode) and an AMVP mode (IBC amvp mode). The skip/merge mode and the AMVP mode can be understood as a mode for encoding information related to the IBC and transmitting the related information in a similar manner to the skip, merge, and AMVP modes when encoding a motion vector (MV) in inter prediction.

However, it should be understood that the prediction mode indicated in the present invention includes all definitions of various modes that can be used to implement the prediction encoding/decoding method indicated in the present invention, including the distinction according to the above-described example. In addition, the information expressed as the prediction mode for convenience in the present invention may be a single piece of information or may be composed of multiple pieces of information, and may be included in a bit string as a single phrase or multiple phrases. For example, the IBC skip mode, the IBC merge mode, and the IBC AMVP mode may each be defined as prediction modes based on a single bit string phrase and/or a single information value, but as another example, the bit string phrase and/or information value indicating whether or not IBC is used may be combined with the bit string phrase and/or information value indicating whether or not one of the skip/merge/AMVP modes is used, thereby deriving the prediction mode. It is self-evident that other bit string syntax and/or information values indicating the prediction mode, such as whether intra-screen/inter-screen prediction is performed, whether template matching is performed, and whether combination with other intra-screen/inter-screen prediction modes is performed, can be used to represent the prediction mode, either merged or independently.

According to one embodiment of the present invention, a bitstream syntax structure that can be provided for the IBC encoding method may be as exemplified below or similar thereto. Table 1 below shows a part of the structure of a sequence parameter set (SPS) in a bitstream syntax structure for the VVC standard or an encoding/decoding standard derived therefrom. In the SPS syntax structure, when the flag information indicated by "sps_ibc_enabled_flag" is activated, "sps_six_minus_max_num_ibc_merge_cand", which is syntax information indicating the maximum size of an IBC skip/merge candidate list, may be configured to be transmitted.

seq_parameter_set_rbsp(　) { sps_ibc_enabled_flag if(sps_ibc_enabled_flag) sps_six_minus_max_num_ibc_merge_cand …

Table 2 below shows part of the structure of a coding block (CU, coding unit) in the bitstream syntax structure for the VVC standard or an encoding/decoding specification derived therefrom. A person skilled in the art will understand that the CU corresponds to one of the block structures corresponding to the coding unit previously indicated as a "current block", etc. In the CU syntax structure, "pred_mode_ibc_flag", which is syntax information indicating whether the corresponding block is encoded in IBC mode, and "general_merge_flag", which is syntax information indicating whether the encoding mode is merge mode, may be configured to be transmitted. In this case, when the IBC skip/merge mode is applied, "merge_idx" syntax information provided to indicate one of the IBC merge candidate lists may be included. In this regard, reference may be made to the syntax information structure of "merge_data" as shown in Table 3 below. The syntax information may be configured to define a prediction vector, for example, a BV, of the corresponding block. Meanwhile, when the IBC AMVP mode is applied, syntax information including "mvd", "mvp", and "avmr" may be included, and the "merge_idx" and/or the "mvd", "mvp", and "avmr" syntax information may be applied as values defining a prediction vector, for example, a BV, of the corresponding block, respectively. In expressing a BV among the prediction vectors, syntaxes such as the "mvd", "mvp", and "amvr" may be identified or included as syntax information based on the BV, such as the "bvd (BV difference)", "bvp (bv predictor)" index and/or flag, and the "abvr" (adaptive BV resolution)" index, respectively. That is, any description appearing in this specification including "MV" may be interpreted as being replaced with "BV", and it should be understood that the referred objects may be used interchangeably as long as there is no obvious technical conflict and it conforms to the technical spirit of the present invention.

Both the IBC skip/merge mode and the IBC AMVP mode described above can be forms of uni-prediction that rely on a single prediction vector.

coding_unit(x0,　y0,　cbWidth,　cbHeight,　cqtDepth,　treeType,　modeType) { if(((sh_slice_type==I && cu_skip_flag[x0][y0]==0) ||
(sh_slice_type!=I && (CuPredMode[chType][x0][y0]!=MODE_INTRA ||
(((cbWidth==4 && cbHeight==4) || modeType==MODE_TYPE_INTRA)
&& cu_skip_flag[x0][y0]==0)))) &&
cbWidth<=64 && cbHeight <= 64 && modeType != MODE_TYPE_INTER &&
sps_ibc_enabled_flag && treeType != DUAL_TREE_CHROMA ) pred_mode_ibc_flag … } else if(treeType!=DUAL_TREE_CHROMA) { /* MODE_INTER or MODE_IBC */ if(cu_skip_flag[x0][y0]==0 ) general_merge_flag [x0][y0] if(general_merge_flag[x0][y0]) merge_data (x0, y0, cbWidth, cbHeight, chType) else if(CuPredMode[chType][x0][y0]==MODE_IBC) { mvd_coding (x0, y0, 0, 0) if(MaxNumIbcMergeCand>1) mvp_l0_flag [x0][y0] if(sps_amvr_enabled_flag &&
(MvdL0[x0][y0][0]!=0 || MvdL0[x0][y0][1]!=0)) amvr_precision_idx [x0][y0] } else { …

merge_data(x0,　y0,　cbWidth,　cbHeight,　chType) { if(CuPredMode[chType][x0][y0]==MODE_IBC) { if(MaxNumIbcMergeCand>1) merge_idx [x0][y0] } else { …

According to one embodiment of the present invention, at least one of two values may be selected to select a precision for indicating a value of a prediction vector (e.g., BV) of an arbitrary decoding target block, and the corresponding information may be configured to be transmitted by the "amvr_precision_idx" syntax information shown in Table 2 above. In addition, according to one embodiment of the present invention, the AMVR (adaptive motion vector resolution) value may correspond to a value indicating an adaptive resolution for a BVD (block vector difference), and may be expressed using "AmvrShift" syntax information. Table 4 below shows in detail the expression of the "AmvrShift" value and its meaning.

amvr_
flag amvr_
precision_
idx AmvrShift inter_affine_flag==1 CuPredMode[chType][x0][y0]==MODE_IBC) inter_affine_flag==0 && CuPredMode[chType][x0][y0]!=MODE_IBC 0 - 2 (1/4 luma sample) - 2 (1/4 luma sample) 1 0 0 (1/16 luma sample) 4 (1 luma sample) 3 (1/2 luma sample) 1 1 4 (1 luma sample) 6 (4 luma samples) 4 (1 luma sample) 1 2 - - 6 (4 luma samples)

Referring to Table 4 above, when the block to be decoded is encoded in IBC mode, the "AmvrShift" value can have a 1-pixel or 4-pixel value. That is, it means that the expression unit of the prediction vector representing the motion of the block, for example, BV, can be at least one of a 1-pixel unit or a 4-pixel unit. Depending on the embodiment, the above expression method may be because, compared to existing real-life images, there is no need to express the motion information of the prediction vector with sub-pixel precision in the case of computer graphic processed images. Of course, according to various embodiments of the present invention, a more precise information expression resolution of the prediction vector may be adopted, and the present invention does not limit the possibility of such modified implementation.

According to one embodiment of the present invention, the present invention can be configured to provide a method for encoding and decoding useful for screen content. When a block to be decoded is encoded in IBC mode, the present invention provides a method for calculating and/or transmitting information about a prediction vector, e.g., a block vector (BV), and also provides a method for redefining the corresponding block vector during the decoding process.

Conventional video encoding/decoding technologies have calculated, transmitted, and restored BV values, which are intra-screen prediction information, less precisely than motion vectors (MVs), which are inter-screen prediction information. For example, in the case of general inter-screen prediction, MV values are expressed with 1/4 pixel precision, and in the case of the "affine" method, MV values are expressed with 1/16 pixel precision, while BV values, which are intra-screen prediction information, are expressed with 1 pixel or 4 pixel precision. The present invention can be utilized to provide a method for more precisely expressing BV of IBC, which is a method of encoding screen contents, through various embodiments. Depending on the embodiment, the present invention may be configured to use bi-predictive BV information to increase the accuracy of the BV.

According to one embodiment of the present invention, the IBC encoding method can be subdivided into a number of modes including an IBC skip/merge mode, an IBC MBVD (merge with block vector difference) mode, and an IBC BVP (block vector prediction) mode. In the skip/merge mode, the BV value can be transmitted using a candidate index value for the BV, and in the MBVD mode, a "bvd" (BV difference) value can be transmitted in addition to the candidate index value, and in the BVP mode, the BV value can be transmitted based on individual values including a "bvd" (BV difference), a "bvp" (BV predictor) index, and an "abvr" (adaptive BV resolution).

First embodiment

According to one embodiment of the present invention, when applying the IBC BVP mode, information related to the precision of the prediction vector (e.g., BV) of the target block can be transmitted in a method indicating a sub-pixel unit. In the specification of the present invention, the sub-pixel can be understood as a term indicating a precision/resolution of less than 1/2 pixel, 1/4 picture, 1/8 pixel, 1/16 pixel, 1/32 pixel, and less than 1 pixel that is not limited to the above-described examples, which are smaller than an integer pixel. In addition, in the specification of the present invention, the sub-pixel should be considered to be compatible with all terms that indicate substantially the same concept by other names, and can be considered equivalent to terms such as sub-pel, fractional pixel, and fractional-pel, for example.

According to the method according to the above-described embodiment, a method can be provided that can express the precision of a prediction vector more precisely than the existing integer pixel precision (e.g., 1 pixel or 4 pixel unit), and can transmit the value of a prediction vector (i.e., BV) at the level of, for example, 1/2 pixel and/or 1/4 pixel.

According to one embodiment of the present invention, an "abvr_flag" bit string syntax flag value indicating an adaptive block vector resolution (ABVR) may be provided, and if the value of the "abvr_flag" is determined to be false (which may be expressed in a manner including "No", "N", "False", "0", etc.), the BV has a precision of 1 pixel unit, and if the value is determined to be true (which may be expressed in a manner including "Yes", "Y", "True", "1", etc.), the BV can be expressed by a plurality of resolutions, and in this case, information related to the resolution may be configured to be expressed as an "abvr_precision_idx" value. Depending on the embodiment, even if the "abvr_flag" flag value is determined to be false, it may be configured to have a precision of a sub-pixel (e.g., 1/4 pixel) unit as a basic precision value.

Tables 5 to 17 below illustrate various embodiments of a method for configuring a prediction vector precision, i.e., an "AbvrShift" value, expressed by a combination of the "abvr_flag" value and the "abvr_precision_idx" value. The embodiments exemplified below are examples of various methods that can be implemented by combining the above values, and the implementation method of the present invention is not limited to the embodiments below and may include various combinations that can be inspired or derived from the embodiments below. That is, each of the embodiments below can be generalized to a process of reading the "abvr_precision_idx" value when the value of the "abvr_flag" falls within a specific range and determining the precision of the prediction vector accordingly.

abvr_flag abvr_precision_idx AbvrShift 0 - 4 (1 luma sample) 1 0 4 (1 luma sample) 1 1 6 (4 luma samples) 1 2 2 (1/4 luma sample)

abvr_flag abvr_precision_idx AbvrShift 0 - 4 (1 luma sample) 1 0 4 (1 luma sample) 1 1 6 (4 luma samples) 1 2 3 (1/2 luma sample)

abvr_flag abvr_precision_idx AbvrShift 0 - 4 (1 luma sample) 1 0 4 (1 luma sample) 1 1 6 (4 luma samples) 1 2 2 (1/4 luma sample) 1 3 3 (1/2 luma sample)

abvr_flag abvr_precision_idx AbvrShift 0 - 4 (1 luma sample) 1 0 6 (4 luma samples) 1 1 2 (1/4 luma sample)

abvr_flag abvr_precision_idx AbvrShift 0 - 4 (1 luma sample) 1 0 2 (1/4 luma samples) 1 1 6 (4 luma samples)

abvr_flag abvr_precision_idx AbvrShift 0 - 2 (1/4 luma sample) 1 0 4 (1 luma samples) 1 1 6 (4 luma samples)

abvr_flag abvr_precision_idx AbvrShift 0 - 4 (1 luma sample) 1 0 6 (4 luma samples) 1 1 3 (1/2 luma sample)

abvr_flag abvr_precision_idx AbvrShift 0 - 4 (1 luma sample) 1 0 3 (1/2 luma samples) 1 1 6 (4 luma samples)

abvr_flag abvr_precision_idx AbvrShift 0 - 3 (1/2 luma sample) 1 0 4 (1 luma samples) 1 1 6 (4 luma samples)

abvr_flag abvr_precision_idx AbvrShift 0 - 4 (1 luma sample) 1 0 6 (4 luma samples) 1 1 2 (1/4 luma sample) 1 2 3 (1/2 luma sample)

abvr_flag abvr_precision_idx AbvrShift 0 - 4 (1 luma sample) 1 0 2 (1/4 luma sample) 1 1 6 (4 luma samples) 1 2 3 (1/2 luma sample)

abvr_flag abvr_precision_idx AbvrShift 0 - 2 (1/4 luma sample) 1 0 4 (1 luma sample) 1 1 6 (4 luma samples) 1 2 3 (1/2 luma sample)

abvr_flag abvr_precision_idx AbvrShift 0 0 2 (1/4 luma sample) 0 1 3 (1/2 luma sample) 1 0 4 (1 luma sample) 1 1 6 (4 luma samples)

As described above, the detailed values and order of the "abvr_flag", "abvr_precision_idx", and "AbvrShift" values presented in the embodiments disclosed through this specification are presented as examples for the convenience of explaining the present invention, and it will be easily understood that according to the present invention, a method of calculating and transmitting the resolution of BV in sub-pixel units by various methods can be applied without being limited to the form shown in the above-described table.

Second embodiment

According to one embodiment of the present invention, a method of redefining information of a prediction vector, for example, BV, during a decoding process can be applied. If the precision value of the prediction vector is calculated and transmitted in sub-pixel units as in the above-described embodiment, a disadvantage may occur in that the amount of bit information expressing the corresponding information increases. To overcome this, according to one embodiment of the present invention, a method can be applied in which the vector value is processed in integer-pixel units when calculating and transmitting the prediction vector (BV), and the encoder/decoder refines it in sub-pixel units during the decoding process.

The embodiment of the present invention is not limited to calculating and/or transmitting as integer pixels and redefining them in sub-pixel units as described above, and should be understood to include any method of compensating during the decoding process with a precision unit more precise than the prediction vector precision at the time of calculation/transmission by other precision units or methods. For example, a method of transmitting a BV value in units of 1/2 pixels and compensating for it in units of 1/4 pixels should also be considered to be included in an embodiment of the present invention. Table 18 below shows a method of transmitting a prediction vector by single precision according to an embodiment of the present invention. As shown in Table 18 below, a prediction vector (e.g., BV) can be transmitted based on a precision value of either 1 pixel or 4 pixels.

abvr_precision_flag AbvrShift 0 4 (1 luma sample) 1 6 (4 luma samples)

According to one embodiment of the present invention, when redefining the value of a prediction vector, for example, BV, a template matching (TM) method may be used. The TM may refer to a process of setting an adjacent left and/or upper certain area of a block to be encoded/decoded (i.e., a current block) as a current template, setting an adjacent left and/or upper certain area of a reference block as a corresponding reference template, and then measuring an error value between the template of the current block and the template of the reference block to select a reference block (or a prediction block) with the smallest error value. Here, the search range of the reference block may be limited to a reconstructed area of a predefined size within the current picture, and the error value of a pixel belonging to the template may be configured to be measured using a calculation method such as SAD (Sum of Absolute Differences) or MSE (Mean Squared Error).

FIG. 8 is a conceptual diagram for redefining a prediction vector based on template matching according to an embodiment of the present invention. For example, referring to FIG. 8, a predetermined area around a current block (810) may be defined as a current template (815) area. Next, a plurality of reference blocks (820) may be searched for among reconstructed areas (890) within the same picture, and a block whose surrounding reference template area (825) most similarly matches the current template (815) may be searched for, and this may be defined as a final reference block (820). As described above, the position of the found reference block (820) may be used to determine a prediction vector, i.e., BV (830), based on the current block (810). In this case, depending on the embodiment, the vector value redefined as the prediction vector (830) value of the current block may be configured to be stored in memory and utilized for blocks to be encoded/decoded thereafter. The above-mentioned stored prediction vector can be utilized in the process of determining the prediction vector of subsequent blocks, and can be utilized in a manner similar to, for example, the HMVP (History-based Motion Vector Prediction) method.

FIG. 9 is a flowchart illustrating a process of redefining a prediction vector according to an embodiment of the present invention. Referring to FIG. 9, first, the "abvr_precision_flag" value, which is bit string syntax information indicating the resolution of a prediction vector (such as bv) for a block to be encoded/decoded, may be parsed (S910). Next, the "BV_refinement_enabled_flag" value, which is information indicating whether the function of redefining the prediction vector is allowed, may be checked (S920). The "abvr_precision_flag" may be transmitted in the form of an index when there are three or more defined resolution/precision values. The present invention does not limit the number of resolution/precision values specified.

Next, if the "BV_refinement_enabled_flag" value is determined to be true (which may be expressed in a manner including "Yes", "Y", "True", "1", etc.) (S920), the "BV_refinement_flag" value, which is bit string syntax information indicating whether to perform prediction vector redefinition of the target block, may be parsed (S930). At this time, the "bv_refinement_enabled_flag" may be obtained from at least one location among a header area including a sequence parameter set (SPS), a picture parameter set (PPS), a picture header (PH), and a slice header (SH), as a high-level syntax (HLS).

If the above "BV_refinement_flag" value is determined to be true (which may be expressed in a manner including "Yes", "Y", "True", "1", etc.) (S940), the TM process may be executed based on a precision in a unit more refined than the precision of the prediction vector that can be determined according to the "abvr_precision_flag" transmitted as the "BV precision" value of the target block (S950), and the value of the prediction vector, i.e., BV, may be redefined based on the result (S960). For example, when the transmitted "bv precision" is in 1-pixel units, the value of the prediction vector may be redefined in 1/4-pixel units through the TM process.

According to another embodiment of the present invention, the process of determining the "BV_refinement_enabled_flag" (S920) may be omitted, and parsing of the "BV_refinement_flag" value indicating whether to perform prediction vector redefinition for the target block (S930) may be performed.

According to another embodiment of the present invention, a series of processes from the process of determining the "BV_refinement_enabled_flag" (S920) to the process of determining the "BV_refinement_flag" (S940) can be omitted, and the process can be configured to proceed directly to the TM process (S950) and redefine the prediction vector value (S960) without asking whether to redefine the prediction vector for the target block.

According to one embodiment of the present invention, the value of the transmitted prediction vector (BV) is set to the position value of the initial reference block, a TM process is performed around the initial reference block, and a final updated vector value is obtained through this, and the vector value obtained in this way can be configured to be used as the final prediction vector (BV) value of the target block. During the TM process, a compensation process for the prediction vector (BV) can be performed down to the sub-pixel unit, and the prediction vector value determined as a result can have precision in the sub-pixel unit. Through this vector refinement process, the precision of the prediction vector (BV) can be increased. In addition, since the initial position is determined by the vector value transmitted in the process, the search range for confirming the reference block in the target block is limited, thereby providing an effect of reducing the complexity of the decoder.

It will be readily understood that the improved method according to the above-described embodiment is applicable to IBC skip/merge mode, IBC MBVD mode, and IBC BVP mode.

Third embodiment

According to one embodiment of the present invention, a method of redefining information of a prediction vector, for example, BV, during a decoding process may be applied. According to the embodiment, when a prediction vector (BV) value of a target block is transmitted, the decoder may be configured to divide the target block into sub-blocks of a predefined size, and then refine and/or compensate for the vector value for each sub-block. For example, if the target block is 32x16, it may be divided into eight 8x8 unit sub-blocks, and then the vector value may be redefined for each 8x8 sub-block. However, the specific method for dividing the sub-blocks is not limited by the present invention. For example, the size of the sub-block is not limited to 8x8. In the present invention, all predetermined pixel units that can divide the encoding target block into a size smaller than the corresponding block may be regarded as corresponding to the sub-block. Similar to what has been explained through the description of other embodiments above, a method using template matching (TM) can be used when redefining the BV value for each sub-block.

Fig. 10 is a conceptual diagram for redefining a sub-block-based prediction vector according to an embodiment of the present invention. Referring to Fig. 10, a target block (1010) having a size of 32x16 pixels is illustrated as being divided into eight 8x8 sub-blocks. A method for compensating a prediction vector (BV) value for each sub-block unit is exemplarily described. Based on an initial BV value (1030) of the target block (1010), the BV value can be redefined for each of the 8x8 sub-blocks belonging to a virtual reference block (1020) indicated by the initial BV value.

As illustrated in FIG. 10, the template most similar to the first sub-template (1016) of the first 8x8 sub-block (1011) can be searched for through the TM process. If the corresponding first reference template (1026) is found through the search, the first sub-reference block (1021) can be found according to the result of compensating and correcting the BV value of the first 8x8 sub-block (1011) (1031). Similarly, the template most similar to the second sub-template (1017) of the second 8x8 sub-block (1012) can be searched for through the TM process. If the corresponding second reference template (1027) is found through the search, the second sub-reference block (1022) can be found according to the result of compensating and correcting the BV value of the second 8x8 sub-block (1012) (1032).

At this time, among the areas included in the second sub-template (1021), the left side may use the right area information derived from the predicted value of the first 8x8 sub-block (1011) (which may be based on the first sub-reference block (1021) and/or the first reference template (1026). That is, according to an embodiment of the present invention, in the process of sequentially redefining the BV value of the divided sub-block, it may be configured not to use the restored value for the preceding sub-block belonging to the same block (for example, the first sub-block (1011) preceding the second sub-block (1012) in the target block (1010). As described above, the same process may be repeated for eight 8x8 sub-blocks belonging to the target block (1010), so that the BV value for each sub-block may be compensated and corrected. According to an embodiment, the initial BV value (1030) of the target block (1010) may be transmitted independently via a bit string syntax, or may be a value derived from other information.

According to one embodiment of the present invention described above, when redefining a prediction vector (BV), redefinition is performed for each sub-block rather than for each target block, thereby increasing the precision of the prediction vector (BV), and further improving the precision of the prediction vector through compensation correction with a precision more detailed than the precision of the prediction vector of the target block when redefining using TM. According to one embodiment, after transmitting the prediction vector (BV) value of the target block with a precision of 1 pixel unit, compensation correction can be performed in units of 1/4 pixel when redefining the BV value for each sub-block. According to another embodiment, it is also possible to compensate-correct the prediction vector (BV) of the target block and the sub-block with the same precision. For example, after transmitting the vector value of the target block with a precision of 1 pixel unit, compensation correction can be performed again with a precision of 1 pixel unit for the vector value of the sub-block.

The embodiment of the present invention is not limited to calculating and/or transmitting the vector value of the target block as an integer pixel as described above and redefining it in sub-pixel units for each sub-block, and it should be understood that it includes any method for compensating the vector for each sub-block with a precision unit that is equal to or more precise than the prediction vector precision at the time of calculating/transmitting for the target block by other precision units or methods. For example, a method of transmitting the BV value of the target block in units of 1/2 pixels and compensating for the BV value of the sub-block in units of 1/4 pixels should also be considered to be included in one embodiment of the present invention.

In implementing the above embodiment, the flowchart shown in Fig. 9 may be applied identically or similarly. A detailed description of the processing steps in the flowchart may be described in accordance with the above description. However, each step related to whether or not to redefine and execute a vector may be interpreted as being related to whether or not to redefine and execute a sub-block. As in the above-described embodiment, according to another embodiment of the present invention, the process of determining the "BV_refinement_enabled_flag" (S920) may be omitted, and parsing of the "BV_refinement_flag" value indicating whether to perform prediction vector redefinition for the sub-block (S930) may be performed. According to another embodiment of the present invention, a series of processes from the process of determining the "BV_refinement_enabled_flag" (S920) to the process of determining the "BV_refinement_flag" (S940) may be omitted, and the process may be configured to proceed directly to the TM process (S950) without asking whether to redefine the prediction vector for the sub-block, and redefine the prediction vector value for each sub-block (S960).

It will be readily understood that the improved method according to the above-described embodiment is applicable to IBC skip/merge mode, IBC MBVD mode, and IBC BVP mode.

Fourth embodiment

According to one embodiment of the present invention, a prediction method using a prediction vector, e.g., a BV, defined by uni-predictive and/or bi-prediction is proposed. The bi-prediction may refer to a method of deriving two different reference blocks (or prediction blocks) within a current picture, and performing prediction fusion on the two reference blocks using a method including a weighted sum or a weighted average to ultimately generate a single prediction block.

FIG. 11 is a conceptual diagram illustrating a prediction method using bidirectional prediction according to an embodiment of the present invention. Referring to FIG. 11, after two reference blocks (1121, 1122) are derived by using two prediction vectors (1131, 1132) for a target block (1110), a final reference block can be obtained through a weight-based operation (e.g., weighted sum or weighted average) on the predicted block values derived from the two reference blocks (1121, 1122). According to various embodiments, the weights may be defined as 5:5 to be mutually equal, or may be transmitted through separate bitstream syntax information. For example, a weight distribution value such as 4:6 or 3:7 may be transmitted through an expression method such as a flag value or index value included in the syntax information. In addition, the weights may be derived from other information derived from the bitstream during the decoding process, instead of being transmitted independently by the bitstream. The above derivation may be made directly in the form of the above weights, or in a form corresponding to the above-described flag values or index values.

As described above, the process of deriving two reference blocks (1121, 1122) can be accomplished by obtaining a prediction vector for each reference block, for example, a BV value (1131, 1132), by one of various modes including a skip/merge mode, an MBVD mode, and a BVP mode. For example, the BV (1131) for the first reference block (1121) can be selected and transmitted by the skip/merge mode, and the BV (1132) for the second reference block (1122) can be selected and transmitted by the MBVD mode. For another example, the BV (1131) for the first reference block (1121) can be selected and transmitted by the merge mode, and the BV (1132) for the second reference block (1122) can be selected and transmitted by the BVP mode. As another example, weights for the two reference blocks (1121, 1122) may be derived according to each of the above modes. For example, a higher weight (e.g., 7) may be assigned to a reference block using the BVP mode, and a lower weight (e.g., 3) may be assigned to a reference block using the merge mode. With reference to the above-described embodiments, it should be understood that various other applications are possible, not limited to the examples described above, regarding modes and weights.

According to one embodiment of the present invention, the two reference blocks (1121, 1122) can select and transmit prediction vectors, e.g., BV values, in the same mode. For example, the BV (1131) for the first reference block (1121) can be selected and transmitted by merge mode, and the BV (1131) for the second reference block (1121) can also be selected and transmitted by merge mode. According to an embodiment, the weights required for the process of merging the two reference blocks (1121, 1122) can be derived according to the transmission order. For example, a higher weight (e.g., 7) can be assigned to a vector value transmitted first, and a lower weight (e.g., 3) can be assigned to a vector value transmitted later. Conversely, a lower weight (e.g., 3) can be assigned to a vector value transmitted first, and a higher weight (e.g., 7) can be assigned to a vector value transmitted later. With reference to the above-described embodiments, it can be seen that various other applications are possible, not limited to the examples described above, regarding weights determined dependent on the transmission order. For example, a predetermined arbitrary weight can be assigned dependent on the transmission order.

Hereinafter, additional embodiments for determining weights for two reference blocks when performing bidirectional prediction using two reference blocks in IBC mode are described.

According to one embodiment of the present invention, bit string syntax information, for example, "two_ref_block_flag", indicating whether to use two reference blocks for a target block may be parsed. At this time, if the "two_ref_block_flag" value is determined to be true (which may be represented in a manner including "Yes", "Y", "True", "1", etc.), the bit string syntax for two prediction vector (BV) values may be parsed, and if the "two_ref_block_flag value is determined to be false (which may be represented in a manner including "No", "N", "False", "0", etc.), the bit string syntax for one prediction vector (BV) value may be parsed. The mode for the two reference blocks may be transmitted by adopting one of various methods including the skip/merge mode, the MBVD mode, and the BPV mode described above in the present invention.

According to one embodiment of the present invention, bit string syntax information indicating whether to use two reference blocks (e.g., the "two_ref_block_flag"), for example, a "two_ref_block_enabled_flag" value, may be set so that if the "two_ref_block_enabled_flag" value is determined to be true (which may be expressed in a manner including "Yes", "Y", "True", "1", etc.), the "two_ref_block_flag" may be parsed. At this time, the "two_ref_block_enabled_flag" may be obtained from at least one location among a header area including a sequence parameter set (SPS), a picture parameter set (PPS), a picture header (PH), and a slice header (SH) as a high-level syntax (HLS).

According to one embodiment of the present invention, direction flag information for a target block may be transmitted. Fig. 12 is a flowchart for processing direction flag information according to one embodiment of the present invention. Referring to Fig. 12, direction flag information (e.g., "direction_flag") may be parsed (S1210). Next, it may be determined (S1220) whether the direction flag information indicates application of either uni-predictive or bi-predictive. If the direction flag information indicates uni-predictive, one BV information may be transmitted (S1230), and if it indicates bi-predictive, two BV information may be transmitted (S1235).

According to one embodiment of the present invention, in skip/merge mode and MBVD mode, directionality flag information of a target block is determined according to a final selected candidate index value, and in BVP mode, it may be determined according to bit string syntax information such as "direction_flag". In the skip/merge mode and MBVD mode, the prediction vector (BV) information indicated by the final selected candidate index value may indicate uni-predictive or bi-predictive. Meanwhile, if the "direction_flag" syntax information value is determined to be bi-predictive in the BVP mode, information on two prediction vector (BV) values may be parsed. The prediction vector (BV) information may include syntax information of at least one of "bvd", "bvp_idx" (or "bvp_flag"), "abvr_idx", and "ref_idx".

Even in this case, when bidirectional prediction is applied, after deriving two reference blocks (1121, 1122), the final reference block can be obtained through a weight-based operation (e.g., through a weighted sum or a weighted average) on the predicted block values derived from the two reference blocks (1121, 1122). According to various embodiments, the weights may be defined as 5:5 to be mutually equal, or may be transmitted through separate bitstream syntax information. For example, a weight distribution value such as 4:6 or 3:7 may be transmitted through an expression method such as a flag value or an index value included in the syntax information. In addition, the weights may be derived from other information derived from the bitstream during the decoding process instead of being independently transmitted by the bitstream. The derivation may be performed directly in the form of the weights, or in a form corresponding to the flag value or index value described above.

The above prediction fusion can be achieved in various ways that can be applied from the method based on the two reference blocks described above. For example, three or more reference blocks can be used with appropriate weights, and at least one of the reference blocks included in the prediction fusion can be derived by a method that is not based on a prediction vector, such as a method that includes DC prediction or planar prediction.

Fifth embodiment

According to one embodiment of the present invention, a method using an extended bidirectional prediction vector can be used. Using the extended bidirectional prediction vector, intra-screen prediction and inter-screen prediction can be performed simultaneously. For example, a reference block pointed to by a BV, which is an intra-screen prediction vector, within a current picture can be derived, and a reference block pointed to by a MV, which is an inter-screen prediction vector, within a reference picture can be derived. As described above, after deriving two different reference blocks, the two reference blocks are predicted and fused using a method including a weighted sum or a weighted average to ultimately generate a single prediction block.

Fig. 13 is a conceptual diagram illustrating a prediction method using extended bidirectional prediction according to one embodiment of the present invention. Referring to Fig. 13, a first reference block (1320) is derived from an intra-screen prediction from a decoded area (1390) within a current picture using a block vector (BV) (1330) for a target block (1310), and a second reference block (1340) is derived from an inter-screen prediction within a reference picture using a motion vector (MV) (1350). Finally, a final reference block can be obtained through a weight-based operation (e.g., through a weighted sum or a weighted average) on the predicted block values derived from the two reference blocks (1320, 1340). According to various embodiments, the weights may be defined as 5:5 to be mutually equal, or may be transmitted through separate bitstream syntax information. For example, a weight distribution value such as 4:6 or 3:7 can be transmitted through a representation method such as a flag value or index value included in the syntax information. In addition, the weight may be derived from other information derived from the bit string during the decoding process instead of being transmitted independently by the bit string. The derivation may be performed directly in the form of the weight value, or in a form corresponding to the flag value or index value described above.

As described above, the first reference block (1320) by intra-screen prediction can be derived based on the BV (1330), and the second reference block (1340) by inter-screen prediction can be derived based on the MV (1350). At this time, the BV information can be obtained by one of various intra-screen prediction modes including the IBC skip/merge mode, the IBC MBVD mode, and the IBC BVP mode. In addition, the MV information can be obtained by one of various inter-screen prediction modes including the skip/merge mode, the MMVD mode, and the AMVP mode. According to various embodiments, the weights can be defined as 5:5 to be mutually equal, or can be defined differently according to the prediction mode. For example, a higher weight (e.g., 7) can be assigned to a reference block based on inter-screen prediction, and a lower weight (e.g., 3) can be assigned to a reference block based on intra-screen prediction. Conversely, a lower weight (e.g., 3) can be assigned to a reference block based on inter-screen prediction, and a higher weight (e.g., 7) can be assigned to a reference block based on intra-screen prediction. With reference to the above-described embodiments, it can be seen that various other applications are possible, not limited to the examples described above, with respect to weights determined depending on the prediction mode. That is, any predetermined weight can be assigned depending on the prediction mode.

According to one embodiment of the present invention, a method of performing intra-screen prediction (BV) and inter-screen prediction (MV) together may be distinguished as separate prediction modes, and a dedicated flag/index value indicating the prediction mode may be defined as bit string syntax information (e.g., "combined_bv_mv_pred_flag") and used.

For example, if the "combined_bv_mv_pred_flag" value is determined to be true (which may be represented by a method including "Yes", "Y", "True", "1", etc.), the syntax information for the BV value and the syntax information for the MV value may be configured to be parsed according to each mode. At this time, the mode expressing the BV value may be configured to use one mode implicitly predetermined among various intra-screen prediction modes including IBC skip/merge mode, IBC MBVD mode, and IBC BVP mode, or may be configured to explicitly designate one mode through separate syntax information. The mode expressing the MV value may be configured to use one mode implicitly predetermined among various inter-screen prediction modes including skip/merge mode, MMVD mode, and AMVP mode, or may be configured to explicitly designate one mode through separate syntax information. In any case, as the mode for the BV and the mode for the MV are determined, the syntax information of each prediction vector can be parsed according to the corresponding mode.

According to one embodiment of the present invention, bit string syntax information (e.g., the "combined_bv_mv_pred_flag") indicating whether to perform intra-screen prediction (BV-based) and inter-screen prediction (MV-based) together, for example, a "combined_bv_mv_pred_enabled_flag" value, may be set to indicate whether to use the bit string syntax information (e.g., the "combined_bv_mv_pred_enabled_flag"), and if the "combined_bv_mv_pred_enabled_flag" value is determined to be true (which may be expressed in a manner including "Yes", "Y", "True", "1", etc.), the "combined_bv_mv_pred_flag" may be configured to be parsed. At this time, the "combined_bv_mv_pred_enabled_flag" is a high-level syntax (HLS) and can be obtained from at least one location among the header areas including the sequence parameter set (SPS), the picture parameter set (PPS), the picture header (PH), and the slice header (SH).

The above prediction fusion can be achieved in various ways that can be applied from the method based on the two reference blocks described above. For example, three or more reference blocks can be used with appropriate weights, and at least one of the reference blocks included in the prediction fusion can be derived by a method that is not based on a prediction vector, such as a method that includes DC prediction or planar prediction.

Encoder and decoder

It is obvious that the encoding method according to the present invention can be applied equally to an encoder and a decoder. The encoding method according to the present invention can be used as one of the methods for generating sample values of a prediction block for performing intra- and/or inter-picture prediction in an encoder. When residual signal information is encoded by a difference value with respect to such a prediction block, the decoder can obtain decoded samples by generating sample values of the same prediction block using a corresponding and/or symmetrical method and combining the difference value with such a prediction block. Additionally, as exemplified through the internal decoder (420) and the coding loop including the same in FIG. 4, this decoding process can be implemented in the same manner within the encoder to predict the state of the decoder.

The encoding method according to the present invention described above can be implemented through an encoder as a device. The encoder as a device can be implemented in a form that maintains the conventional encoder structure exemplified through FIGS. 1 to 6 above or applies a predetermined change thereto. However, the form of implementation is not necessarily limited to that exemplified, and any form of encoder structure that can function as a video encoder should be considered an encoder established by the present invention as long as it implements the technical idea of the present invention.

In addition, the method for decoding the encoding result according to the present invention described above can be implemented through a decoder as a device. The decoder as a device can be implemented in a form that maintains the conventional decoder structure exemplified through FIGS. 1 to 6 above or applies a predetermined change thereto. However, the form of implementation is not necessarily limited to that exemplified, and any form of decoder structure that can function as a video decoder should be considered a decoder established according to the present invention as long as it implements the technical idea of the present invention.

A person skilled in the art will readily understand that a bit string encoded by the above-described method and device can be decoded by applying a method symmetrical and/or reverse to the encoding method. In one embodiment, when reading information for decoding from the encoded bit string, at least one variable length coded phrase included in the encoded bit string can be interpreted, and furthermore, in one embodiment, the variable length coding can be performed by an entropy coding method. The technical details and application method of implementing such a decoding procedure can be readily understood from the above-described encoding procedure.

The encoder and/or decoder described herein may correspond to a device including a processor and a memory, each of which may be implemented as a computing device. The processor that may be included in the encoder and/or decoder described herein may mean one or more general-purpose computers or special-purpose computers, such as a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable array (FPA), a programmable logic unit (PLU), a microprocessor, or any other device capable of executing instructions and responding.

Even if the processor is expressed singularly for ease of understanding, those skilled in the art will appreciate that the processor may include multiple processing elements and/or multiple types of processing elements. For example, a device according to one embodiment of the present invention may include multiple processors or one processor and one controller as the processor. Furthermore, the processor may be implemented using various processing configurations, such as a parallel processor or a multi-core processor.

The processor may be configured to execute an operating system (OS) and one or more software programs running on the operating system. Furthermore, the processor may access, store, manipulate, process, and generate data in response to the execution of the software.

The software may include a computer program, code, instructions, or a combination of one or more of these, and may be configured to control the processor to perform a desired operation and to issue commands to the processor, either independently or collectively. The software may be permanently or temporarily embodied in any type of machine, component, physical device, virtual equipment, computer storage medium or device, or transmitted signal wave, for interpretation by the processor or for providing commands or data to the processor. The software may also be distributed over networked computer systems, and stored or executed in a distributed manner.

The software may also be implemented in the form of program commands that can be executed through various computer means and recorded or stored in the memory. The memory may be a computer-readable recording medium, and program commands, data files, data structures, etc. may be recorded singly or in combination in the computer-readable recording medium. The program commands stored in the memory may be based on a command system specifically designed and configured for the embodiment of the present invention, or may follow a command system known and available to those skilled in the art of computer software, such as a command system exemplified by the assembly language, C, C++, Java, Python, etc. It should be understood that the command system and the program commands therefrom include not only machine language codes such as those generated by a compiler, but also high-level language codes that can be executed by the device and/or the processor according to an embodiment of the present invention using an interpreter or the like.

The computer-readable recording medium constituting the device according to one embodiment of the present invention, including the memory described in this specification, may include a temporary or volatile recording medium that is maintained only while the processor is operating, such as a processor cache, a RAM, a flash memory, or a relatively non-volatile or long-term recording medium, such as a magnetic media such as a hard disk, a floppy disk, and a magnetic tape, an optical media such as a CD-ROM, a DVD, a magneto-optical media such as a floptical disk, or a solid state memory, or may include a read-only recording medium, such as a ROM arranged on hardware, and further, the hardware itself configured to perform an operation equivalent to a series of program commands by a hard-wired structure by circuit wiring, and each step for performing the operation for implementing the embodiment of the present invention can be viewed as being recorded by the connection and arrangement of the hardware components, and thus the connection and arrangement method is the memory and It is obvious to a person skilled in the art that they can be considered equivalent.

The embodiments described above with respect to the processor and the memory are not mutually exclusive and may be selected or combined and implemented as needed. For example, a single hardware device may be configured to operate as a module composed of one or more of the software to perform the operations of an embodiment of the present invention, and vice versa. As another example, in the present specification, all or part of the operations assigned to a certain functional unit may be implemented by one or more of the software stored in a device according to an embodiment of the present invention (preferably, in any one of the recording media belonging to the category of the memory) and configured to be executed by the processor. In such a case, such a functional unit may be referred to as a functional unit "included" in the processor.

Although the present invention has been described with reference to drawings and embodiments, as already mentioned above, it does not mean that the scope of protection of the present invention is limited to the drawings or embodiments presented above, and it will be understood that a person skilled in the relevant technical field can modify and change the present invention in various ways within a scope that does not depart from the spirit and scope of the present invention described in the claims of the present invention patent.

Claims (20)

In a method for decoding an encoded video bit stream,
A step of reading prediction mode information for prediction decoding of a current block of a picture currently being decoded from the above bit string;
A step of determining whether the above prediction mode information indicates a mode based on intra block copy (IBC);
A step of reading, from the bit string, at least one prediction vector specifying the location of at least one reference block from at least one picture including the picture currently being decoded, based on the prediction mode information;
A step of dividing the current block into two or more sub-blocks;
A step of determining whether the above prediction mode information indicates a mode that uses refinement of the prediction vector;
In the case of a mode using the above redefinition, a step of individually compensating and redefining the prediction vector for each sub-block based on the prediction vector;
A step of obtaining two or more sub-reference blocks individually for each of the sub-blocks based on the prediction vector, and combining the respective sub-reference blocks to obtain at least one reference block;
A step of generating a prediction block based on at least one reference block; and
A decoding method, comprising: a step of performing predictive decoding for the current block based on the predicted block.
In the first paragraph,
The step of reading the above prediction vector is:
A step of obtaining an adaptive block vector resolution (ABVR) flag value from the bit string; and
A step of reading precision representation information of a block vector when the ABVR flag value falls within the first range;
A step of determining the precision of the prediction vector based on the ABVR flag information and the precision expression information; and
A decoding method, comprising: a step of reading the prediction vector based on the above precision;
In the second paragraph,
The accuracy of the above prediction vector is
A decoding method characterized in that it is determined as one of at least one precision unit including a sub-pixel unit.
In the third paragraph,
The accuracy of the above prediction vector is
If the above ABVR flag value does not fall within the first range, in 1/4 pixel units,
A decoding method characterized in that, when the ABVR flag value corresponds to the first range, the value of the precision representation information is determined in units of 1 pixel when the value of the precision representation information is the first value, and in units of 4 pixels when the value of the precision representation information is the second value.
In any one of paragraphs 2 to 4,
The above first range is characterized in that it means a case where the ABVR flag value is “1”,
A decryption method, characterized in that the above precision expression information is an index value having at least two cases.
delete delete In the first paragraph,
The step of redefining the above prediction vector is:
A decoding method characterized in that it operates based on a template matching method having a search range based on a point pointed to by the above prediction vector.
In the first paragraph,
The step of redefining the above prediction vector is:
A decoding method characterized by performing compensation correction with a precision higher than the precision of the above prediction vector.
In paragraph 9,
The step of redefining the above prediction vector is:
A decoding method characterized by compensating and correcting a prediction vector read in units of integer pixels in units of subpixels.
In the first paragraph,
further comprising a step of determining whether the above prediction mode information indicates a mode that allows two or more prediction vectors;
The step of reading the above prediction vector includes a step of reading a first prediction vector; and a step of reading a second prediction vector;
The step of obtaining the above reference block includes: a step of obtaining a first reference block based on the first prediction vector; and a step of obtaining a second reference block based on the second prediction vector;
A decoding method, characterized in that the step of generating the prediction block generates the prediction block through prediction fusion based on two or more reference blocks including the first reference block and the second reference block.
In paragraph 11,
The above first prediction vector and the above second prediction vector are block vectors for intra-screen prediction,
A decoding method, characterized in that the first prediction vector and the second prediction vector are configured to indicate the positions of different reference blocks within an already decoded area of the current picture.
In paragraph 11,
The above first prediction vector is a block vector for prediction within the screen and is configured to indicate the location of the first reference block within an already decoded area of the current picture,
A decoding method, characterized in that the second prediction vector is a motion vector for inter-screen prediction and is configured to indicate the position of the second reference block within a previously decoded picture.
In paragraph 11,
The above prediction fusion is performed by a weighted sum or weighted average performed by a combination of weights,
A decryption method, characterized in that information about the above weight is obtained from the bit string.
In a video encoding method for generating an encoded bit stream,
A step of dividing a current block of a picture currently being encoded into two or more sub-blocks;
For at least one sub-block, determining at least one prediction vector that specifies the location of at least one reference block within at least one picture including the picture currently being encoded;
A step of individually compensating and refining the prediction vector for each sub-block based on the prediction vector;
A step of obtaining two or more sub-reference blocks individually obtained for each sub-block based on the prediction vector, and combining the respective sub-reference blocks to obtain at least one reference block;
A step of generating a prediction block based on at least one reference block;
A step of performing prediction encoding for the current block based on the prediction block;
A step of determining a prediction mode for the current block based on the result of the above prediction encoding as a mode based on intra block copy (IBC);
A step of determining whether the prediction mode for the current block is a mode that uses redefinition of the prediction vector based on the result of the above prediction encoding;
A step of generating a bit string syntax representing the prediction mode and the at least one prediction vector based on the prediction encoding mode; and
An encoding method, comprising: a step of recording the bit string syntax into the bit string.
In paragraph 15,
The above prediction vector is determined by at least one precision including a sub-pixel unit,
An encoding method, wherein the step of generating the bit string syntax includes: a step of generating a bit string syntax representing a unit of precision based on an adaptive block vector resolution (ABVR) flag value and precision representation information of a block vector; and a step of generating a bit string syntax representing the prediction vector based on the unit of precision.
In paragraph 15,
Further comprising a step of compensating the above prediction vector,
The step of obtaining the above reference block is characterized in that it operates based on the above compensation-corrected prediction vector,
The step of determining the above prediction mode is characterized in that the prediction mode is determined as a mode that uses refinement of the prediction vector,
An encoding method, characterized in that the step of generating the bit string syntax generates a bit string syntax representing a prediction vector before the compensation correction.
In paragraph 15,
The step of determining the above prediction vector includes the step of determining a first prediction vector; and the step of determining a second prediction vector;
The step of obtaining the above reference block includes: a step of obtaining a first reference block based on the first prediction vector; and a step of obtaining a second reference block based on the second prediction vector;
The step of generating the prediction block is characterized in that the prediction block is generated through prediction fusion based on two or more reference blocks including the first reference block and the second reference block,
An encoding method, characterized in that the step of determining the prediction mode determines the prediction mode as a mode that allows two or more prediction vectors.
In paragraph 18,
The above first prediction vector is a block vector for prediction within the screen and is configured to indicate the location of the first reference block within an already encoded area of the current picture,
An encoding method, characterized in that the second prediction vector is a motion vector for inter-screen prediction and is configured to indicate the position of the second reference block within a picture in which encoding has been completed previously.
In a decoding device configured to decode a video bit stream encoded by a computer device,
processor;
memory;
An input section into which the above bit string is input;
An output section that outputs the decrypted video;
A reference buffer that stores information about at least one decoded picture, including the picture currently being decoded;
A bitstream parsing unit comprising a function of reading, from the bitstream, prediction mode information for prediction decoding of a current block of the currently decoded picture; determining whether the prediction mode information indicates a mode based on intra block copy (IBC); and reading, from the bitstream, at least one prediction vector designating a location of at least one reference block from at least one of the pictures based on the prediction mode information;
A prediction decoding unit comprising a function of dividing the current block into two or more sub-blocks; determining whether the prediction mode information indicates a mode using refinement of a prediction vector; and, if the mode uses refinement, individually compensating and refining the prediction vector for each sub-block based on the prediction vector; obtaining two or more sub-reference blocks individually for each sub-block from at least one picture stored in the reference buffer based on the prediction vector, combining each of the sub-reference blocks to obtain at least one reference block; generating a prediction block based on the at least one reference block; and performing prediction decoding on the current block based on the prediction block; and
A decoder device comprising a video decoding unit configured to decode the bit string based on the predicted decoding result.