WO2025071286A1 - Method and apparatus for encoding/decoding video on basis of improved template determination for local illuminance compensation - Google Patents

Abstract

The present invention relates to video compression technology and, more specifically, to the improvement of local illuminance compensation (LIC) technology, which is included in prediction coding technology that contributes to the enhancement of compression performance in video encoders and decoders. A method for decoding an encoded video bitstream, according to an embodiment of the present invention, comprises the steps of: obtaining at least one prediction vector for a current block, which is currently being decoded from the bitstream; determining at least one reference block on the basis of the prediction vector; defining at least one of a first template which considers the current block to be a base block, and a second template which considers the reference block to be the base block; determining, for at least one of the templates, a template configuration mode that includes information on a method for using the template; on the basis of the at least one template configuration mode, obtaining LIC information by referring to information on at least one pixel derived from at least one of the templates; generating a prediction sample for the current block by correcting the reference block by means of an LIC method; and reconstructing the current block by using the prediction sample.

Description

Method and device for video encoding/decoding based on improved local illumination compensation template determination

The present invention relates to video compression technology, and more particularly, to improvement of local illuminance compensation (LIC) technology included in prediction coding technology that contributes to improvement of compression performance in a video encoder and decoder.

The present invention may be in the same technical field as at least one of digital video compression technology standards known by the standard names such as MPEG-2, MPEG-4 Video, H.263, H.264/AVC, H.265/HEVC, H.266/VVC, VC-1, AV1, QuickTime, VP-9, VP-10, Motion JPEG, or in the technical field for improving the inherent efficiency of the standard, or in the technical field for improving or replacing the standard.

Digital video encoding and decoding are widely used in various digital video applications. For example, digital television broadcasting, transmission of video through communication networks, video calls/video conversations/video chats, recording and providing video content using optical media including VCD (video compact disc)/DVD (digital versatile disc)/Blu-Ray, all procedures for producing, editing, collecting, and distributing video content, and devices such as video shooting devices and camcorders for shooting and recording video for various reasons including personal, commercial, industrial, and security purposes, all depend on video encoding and decoding technologies.

Accordingly, implementations that may be referred to as digital video encoders and decoders may form part of a wide range of devices, including digital televisions, digital broadcasting systems, wireless broadcasting systems, computers in the form of notebooks/desktops/tablets, e-book readers, digital cameras, digital recording devices, digital multimedia playback devices, video game devices/terminals/console, mobile phones (including smartphones) with multimedia playback capabilities, devices for video conferencing, and other devices related to the generation, recording, and provision of digital video.

The above digital video encoders and decoders can be implemented by a digital video compression standard that is widely used and understood by those skilled in the art. The digital video compression standard can include at least one of compression standards known by standard names such as MPEG-2, MPEG-4 Video, H.263, H.264/AVC, H.265/HEVC, H.266/VVC, VC-1, AV1, QuickTime, VP-9, VP-10, and Motion JPEG.

Video encoders and decoders can be implemented to encode or decode digital video information more efficiently while complying with the above standards, or by improving or modifying the above standards. Attempts to modify the above standards can also lead to the derivation of new standards. Among well-known examples is the so-called enhanced compression model (ECM), which is an attempt to improve and replace the existing H.266/VVC standard, which is being developed by the joint video experts team (JVET), a joint international standardization group of ISO, IEC, and ITU-T.

Among the various detailed technologies applied to conventional video encoders and decoders, there exists a technology collectively called local illuminance compensation (LIC). The LIC technology refers to a technology that calculates a prediction residual by correcting (i.e., compensating) illuminance for a value of a prediction target block selected for predictive encoding in inter-picture predictive encoding (so-called inter encoding) and/or intra-picture predictive encoding (so-called intra encoding), and a decoding process corresponding to each of the above encoding methods, and in particular, it can mean that the compensation is performed locally, for example, on a block-by-block basis.

Conventional LIC technology has limited application methods, and it is expected that encoding efficiency can be improved through various improvements. The present invention aims to solve these technical problems and provide improved encoding efficiency.

The present invention proposes an improved LIC implementation method to solve the above-described technical problems.

According to an embodiment of the present invention for solving the above-described technical problem, a method for decoding an encoded video bitstream comprises the steps of: obtaining at least one prediction vector for a current block currently being decoded from the bitstream; determining at least one reference block based on the prediction vector; defining at least one of a first template including at least one pixel adjacent to the current block as a reference block and a second template including at least one pixel adjacent to the reference block as a reference block; determining a template configuration mode including information on a method of using the template with respect to at least one of the first template and the second template; extracting information related to Local Illumination Compensation (LIC) with reference to at least one pixel information derived from at least one of the first template and the second template based on the at least one template configuration mode, and obtaining LIC information from the extracted information; generating a prediction sample for the current block by correcting the reference block by a LIC method operating based on the LIC information; and A step of restoring the current block using a sample may be included.

The above template configuration mode may be characterized by including at least one mode among a mode that uses the upper pixel of the reference block, a mode that uses the left pixel of the reference block, and a mode that uses the upper and left pixels of the reference block based on weights.

A mode used based on the above weights may be characterized by dividing the upper and left areas of the reference block into at least one zone and applying different zone-specific weights to each zone.

The above zone-specific weights are selected from a predefined table, and the step of determining the template configuration mode may include a step of obtaining information representing an index selected from the table from the bit string.

Information indicating the above index may be characterized as being implicitly determined on the decoder side based on previously decrypted video information.

The above template configuration mode may be characterized by including a mode for selecting and using only some pixels included in either the first template or the second template.

Some of the above pixels may be characterized in that they are selected through subsampling.

The above subsampling may be characterized in that, when the template includes an upper pixel area of the reference block, the area is divided into a first number of horizontal divisions, and a second number of pixels are selected from the divided area.

The above subsampling may be characterized in that, when the template includes a left pixel area of the reference block, the area is vertically divided into a first number of areas, and a second number of pixels are selected from the divided areas.

The above subsampling may be characterized by dividing the template into a first number of regions in at least one of horizontal and vertical directions, and selecting a second number of pixels from the divided regions.

The above subsampling may be characterized by dividing the upper pixel area in a horizontal direction when the template includes an upper pixel area of the reference block, and dividing the left pixel area in a vertical direction when the template includes a left pixel area of the reference block.

It may be characterized in that at least one of the first number and the second number is determined based on a power of 2.

The step of determining the template configuration mode may include the step of obtaining a mode signal instructing to use at least one template configuration mode from the bit string, and the step of determining the template configuration mode based on the mode signal.

The step of determining the template configuration mode may include a step of implicitly determining the template configuration mode on the decoder side based on previously decrypted video information.

According to an embodiment of the present invention for solving the above-described technical problem, a method for encoding a bit stream generated from a video comprises the steps of: determining at least one prediction vector for a current block currently being encoded; determining at least one reference block based on the prediction vector; defining at least one of a first template including at least one pixel adjacent to the current block as a reference block and a second template including at least one pixel adjacent to the reference block as a reference block; determining a template configuration mode including information on a method of using the template with respect to at least one of the first template and the second template; extracting information related to Local Illumination Compensation (LIC) with reference to at least one pixel information derived from at least one of the first template and the second template based on the at least one template configuration mode, and acquiring LIC information from the extracted information; generating a prediction sample for the current block by correcting the reference block by a LIC method operating based on the LIC information; It may include a step of encoding, and a step of recording the encoding result in the bit string.

The above template configuration mode may be characterized by including at least one mode among a mode that uses the upper pixel of the reference block, a mode that uses the left pixel of the reference block, and a mode that uses the upper and left pixels of the reference block based on weights.

A mode used based on the above weights may be characterized by dividing the upper and left areas of the reference block into at least one zone and applying different zone-specific weights to each zone.

The above template configuration mode may be characterized by including a mode for selecting and using only some pixels from at least one of the above templates.

Some of the above pixels may be characterized in that they are selected through subsampling.

The step of determining the prediction vector may include a step of referencing the first template according to a prediction template configuration mode, and the step of obtaining the LIC information may include a step of referencing the first template according to a first LIC template configuration mode, a step of referencing the second template according to a second LIC template configuration mode, and a step of determining a LIC linear model through an operation based on a result of the referencing, wherein the LIC linear model may be a function defined by coefficients including a slope (α) and a bias (β).

The above prediction template configuration mode may include a mode for selecting and using only some pixels from the first template, and the step of determining the prediction vector may be characterized in that it is configured to determine the prediction vector through a template-based search based on the selected some pixels.

At least one of the first LIC template configuration mode and the second LIC template configuration mode may include a mode for selecting and using only some pixels from at least one of the templates, and the step of obtaining the LIC information may be characterized in that it is configured to determine at least one of the coefficients of the LIC linear model through an operation based on the selected some pixels.

According to an embodiment of the present invention for solving the above-described technical problem, a decoder device configured to decode a video bitstream encoded by a computing device comprises a receiving unit for receiving a bitstream, an output unit for outputting a decoded video, a reference buffer for storing information on at least one decoded picture, a processor, and a memory for storing instructions executable by the processor, wherein the instructions obtain at least one prediction vector for a current block currently being decoded from the bitstream, determine at least one reference block based on the prediction vector, define at least one of a first template including at least one pixel adjacent to the current block as a reference block, and a second template including at least one pixel adjacent to the reference block as a reference block, determine a template configuration mode including information on a method of using the template for at least one of the first template and the second template, and, based on the at least one template configuration mode, with reference to at least one pixel information derived from at least one of the first template and the second template, The method may be configured to include commands for extracting information related to local illumination compensation (LIC), obtaining LIC information from the extracted information, generating a prediction sample for the current block by compensating the reference block by a LIC method operating based on the LIC information, and restoring the current block using the prediction sample.

According to the present invention, there is a beneficial effect of improving prediction performance by LIC technology and simultaneously minimizing overhead in the information capacity of a bitstream to be encoded.

According to the present invention, at least one of the following effects can be derived from video encoding and decoding: improvement of encoding efficiency, improvement of decoding efficiency, improvement of video quality, reduction of computational amount, reduction of software size, reduction of hardware size, and improvement of other performances related to encoding and decoding.

Figure 1 is a conceptual diagram of a video communication system according to one embodiment of the present invention.

FIG. 2 is a conceptual diagram of the arrangement of an encoder and decoder in a real-time video streaming environment according to one embodiment of the present invention.

Figure 3 is a conceptual diagram of a functional unit of a video decoder according to one embodiment of the present invention.

Figure 5 is a conceptual diagram of a frame type according to one embodiment of the present invention.

Figure 6 is a conceptual diagram showing the structure of a video encoder according to the H.266/VVC standard.

Figure 7 is a conceptual diagram showing the general structure of a LIC application method according to one embodiment of the present invention.

FIG. 8 is an exemplary diagram showing an example of a template application mode according to one embodiment of the present invention, and

Figure 9 is a conceptual diagram showing the shape and coefficient positions of a cross filter according to one embodiment of the present invention.

The present invention can have various modifications and various embodiments, and specific embodiments are illustrated in the drawings and described in detail. However, this is not intended to limit the present invention to specific embodiments, but should be understood to include all modifications, equivalents, or substitutes included in the spirit and technical scope of the present invention.

Although the terms first, second, etc. may be used to describe various components, the components should not be limited by the terms. The terms are only used to distinguish one component from another. For example, the first component could be referred to as the second component, and similarly, the second component could also be referred to as the first component, without departing from the scope of the present invention. The term "and/or" includes any combination of a plurality of related listed items or any of a plurality of related listed items, and is non-exclusive unless otherwise indicated. The listing of items in this application is merely an exemplary description to easily explain the spirit and possible implementation methods of the present invention, and therefore, is not intended to limit the scope of embodiments of the present invention.

As used herein, “A or B” can mean “only A”, “only B”, or “both A and B”. In other words, as used herein, “A or B” can be interpreted as “A and/or B”. For example, as used herein, “A, B or C” can mean “only A”, “only B”, “only C”, or “any combination of A, B and C”.

As used herein, a slash (/) or a comma can mean "and/or". For example, "A/B" can mean "A and/or B". Accordingly, "A/B" can mean "only A", "only B", or "both A and B". For example, "A, B, C" can mean "A, B, or C".

As used herein, “at least one of A and B” can mean “only A”, “only B” or “both A and B”. Additionally, as used herein, the expressions “at least one of A or B” or “at least one of A and/or B” can be interpreted identically to “at least one of A and B”.

Additionally, in this specification, "at least one of A, B and C" can mean "only A", "only B", "only C", or "any combination of A, B and C". Additionally, "at least one of A, B or C" or "at least one of A, B and/or C" can mean "at least one of A, B and C".

When it is said that a component is "connected" or "connected" to another component, it should be understood that it may be directly connected or connected to that other component, but that there may be other components in between. On the other hand, when it is said that a component is "directly connected" or "directly connected" to another component, it should be understood that there are no other components in between.

The terminology used in this application is only used to describe specific embodiments and is not intended to limit the present invention. The singular expression includes the plural expression unless the context clearly indicates otherwise. In this application, it should be understood that the terms "comprises" or "has" and the like are intended to specify the presence of a feature, number, step, operation, component, part or combination thereof described in the specification, but do not exclude in advance the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts or combinations thereof.

Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. Terms defined in commonly used dictionaries, such as those defined in common dictionaries, should be interpreted as having a meaning consistent with the meaning they have in the context of the relevant art, and shall not be interpreted in an idealized or overly formal sense, unless expressly defined in this application.

In describing the invention in this application, the embodiments may be described or illustrated in terms of unit blocks that perform the described function or functions. The blocks may be expressed as one or more devices, units, modules, parts, etc. in this application. The blocks may be implemented in hardware by one or more logic gates, integrated circuits, processors, controllers, memories, electronic components, or information processing hardware implementation methods that are not limited thereto. Alternatively, the blocks may be implemented in software by application software, operating system software, firmware, or information processing software implementation methods that are not limited thereto. One block may be implemented by being separated into multiple blocks that perform the same function, or conversely, one block may be implemented to perform the functions of multiple blocks simultaneously. The blocks may also be implemented by being physically separated or combined by any criterion. The blocks may be implemented to operate in an environment where their physical locations are not specified and are separated from each other by a communication network, the Internet, a cloud service, or a communication method that is not limited thereto. All of the above implementation methods are within the scope of various embodiments that can be taken by a person skilled in the field of information and communication technology to implement the same technical idea, and therefore, any detailed implementation method should be interpreted as being included within the scope of the technical idea of the invention of the present application.

Hereinafter, with reference to the attached drawings, a preferred embodiment of the present invention will be described in more detail. In order to facilitate an overall understanding in describing the present invention, the same reference numerals are used for the same components in the drawings, and duplicate descriptions of the same components are omitted. In addition, it is assumed that the multiple embodiments are not mutually exclusive, and that some embodiments can be combined with one or more other embodiments to form new embodiments.

Digital Video Codec

Figure 1 is a conceptual diagram of a video communication system according to one embodiment of the present invention. The video communication system (100) may be configured to include at least two terminals (110, 120) that are connected to each other through a network (105).

In one embodiment of the present invention, the above-described FIG. 1 may mean a block diagram for configuring a one-way video communication network. A first terminal (110) among the terminals may encode the video data in order to transmit (111) the video data through a network (105). A second terminal (120) among the terminals may be configured to receive (121) the encoded video data through a network and decode and display it.

In another embodiment of the present invention, the above-described FIG. 1 may mean a block diagram for configuring a two-way video communication network. For the two-way video communication, each terminal (110, 120) may be configured to encode video data acquired by itself for video transmission (112, 122) to each other terminal via the network. Each terminal may also be configured to receive (113, 123) and decode video data transmitted by another terminal via the network, and display the decoded video data.

The terminals (110, 120) shown in Fig. 1 may be exemplified as devices such as server computers, personal computers, portable computers, and smartphones, depending on the embodiment, but are not limited thereto, and may be any computing device generally used. For example, according to an embodiment of the present invention, each terminal (110, 120) may mean a desktop computer, a laptop computer, a tablet PC, a mobile phone, a smart phone, a PDA (personal digital assistant), a workstation, an electronic calculator, a server computer, a cloud computer, a virtual computer, a quantum computer, or any other electronic, electrical, or quantum computing device implemented in a movable or non-movable form, and in particular, it may be interpreted as any device designed to operate as a terminal device according to an embodiment of the present invention among such devices, capable of operating as a terminal device according to an embodiment of the present invention, or capable of installing and/or executing a computer program that enables operating as a terminal device according to an embodiment of the present invention or performing a method corresponding to such an operation.

Each of the above terminals (110, 120) may be implemented by a plurality of functional units that are interconnected in various forms, such as a bus, a circuit, or a relationship between a routine and a subroutine, and configured to exchange information within each of the above terminals (110, 120). In addition, through the interconnection, for the purpose of executing or supporting the operation of a functional unit that mainly requires calculation among the above functional units, the above terminals may be configured to include a processor (130) having a calculation function and a memory (140) connected to the processor.

The processor (130) described in this specification may mean one or more general-purpose computers or special-purpose computers, such as a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable array (FPA), a programmable logic unit (PLU), a microprocessor, or any other device capable of executing instructions and responding.

Even if the processor (130) is expressed singularly for the convenience of understanding, a person having ordinary skill in the art will recognize that the processor (130) may include a plurality of processing elements and/or a plurality of types of processing elements. For example, a device according to an embodiment of the present invention may include a plurality of processors or one processor and one controller as the processor (130). In addition, the processor (130) may be implemented by various processing configurations, such as a parallel processor or a multi-core processor.

The processor (130) may be configured to execute an operating system (OS) and one or more software executed on the operating system. In addition, the processor may access, store, manipulate, process, and generate data in response to the execution of the software. The software may include a computer program, a code, an instruction, or a combination of one or more of these, and may configure a processing device to operate as desired or independently or collectively command a processing device. The software may be permanently or temporarily embodied in any type of machine, component, physical device, virtual equipment, computer storage medium or device, or transmitted signal wave to be interpreted by the processor (130) or to provide instructions or data to the processor. The software may be distributed among a plurality of computer systems connected to the network (105), such as the terminals (110, 120), and stored or executed in a distributed manner.

The software may also be implemented in the form of program commands that can be executed through various computer means and recorded or stored in the memory (140). The memory (140) may be a computer-readable recording medium, and program commands, data files, data structures, etc. may be recorded singly or in combination in the computer-readable recording medium. The program commands stored in the memory (140) may be based on a command system specifically designed and configured for the embodiment of the present invention, or may follow a command system known and available to those skilled in the art of computer software, for example, a command system exemplified by the assembly language, C, C++, Java, Python, etc. It should be understood that the command system and the program commands thereby include not only machine language codes created by a compiler, but also high-level language codes that can be executed by the device and/or the processor (130) according to one embodiment of the present invention using an interpreter, etc.

The computer-readable recording medium constituting the device according to one embodiment of the present invention, including the memory (140) described in this specification, may include a temporary or volatile recording medium that is maintained only while the processor is operating, such as a processor cache, a RAM, a flash memory, or a relatively nonvolatile or long-term recording medium, such as a magnetic media such as a hard disk, a floppy disk, and a magnetic tape, an optical media such as a CD-ROM, a DVD, a magneto-optical media such as a floptical disk, or a solid state memory, or a read-only recording medium such as a ROM arranged on hardware, and further, the hardware itself configured to perform an operation equivalent to a series of program commands by a hard-wired structure by circuit wiring, and since each step for performing the operation implementing the embodiment of the present invention can be viewed as being recorded by the connection and arrangement of the hardware components, the connection and arrangement method It is obvious to those skilled in the art that this can soon be seen as equivalent to the above memory (140).

The embodiments described above with respect to the processor (130) and the memory (140) are not mutually exclusive, and may be selected or combined and implemented as needed. For example, one hardware device may be configured to operate as a module composed of one or more of the software to perform the operations of the embodiments of the present invention, and vice versa. As another example, in the present specification, all or part of the operations assigned to a certain functional unit may be implemented by one or more of the software stored in the device according to one embodiment of the present invention (preferably, in any one of the recording media belonging to the category of the memory) and configured to be executed by the processor, and in this case, such a functional unit may be referred to as a functional unit "included" in the processor.

The present invention is applicable to all environments for forming a one-way or two-way video communication network, and it should be understood that the network (105) can be formed by any means for transporting encoded video data between the terminals (110, 120).

In one embodiment of the present invention, the network (105) may mean a wired or wireless communication network. At this time, depending on the embodiment, the network may be configured to communicate information using any communication standard, and the communication standard may include packet-based communication. The packet communication may be understood to mean including, for example, a packet known as TCP or UDP. The wired communication method of the network (105) may be a method of connecting to an external communication network by a telephone line of the RJ-11 standard, an Ethernet cable belonging to various categories of the RJ-45 standard, other coaxial cables, metal cables, optical cables, and other various wired media. The wireless communication method of the above network (105) may include, depending on the embodiment, a short-range wireless communication method including Bluetooth, Wi-Fi, Zigbee, and NFC (near field communication), or may include a long-range wireless communication method that may be referred to as a name of a wireless communication technology collectively called a generation name of communication standards such as Wibro, WiMax, Global Systems for Mobile Communications (GSM), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Long-Term Evolution (LTE), New Radio (NR), and other 2G, 3G, 4G, 5G, and 6G, and a name of an international standard wireless communication standard such as IMT-2000, IMT-Adcanced, IMT-2020, and IMT-2030. Of course, even if any conventional or newly developed wired or wireless communication means, method, standard, and protocol are applied to the implementation of the network (105), there is no problem in achieving the purpose of the present invention as long as it is a means configured to perform transmission and reception in an information and communication device such as the terminal (110, 120). In addition, it is obvious that one network (105) can be configured in a mixed manner by one or more wired and/or wireless standards.

However, in another embodiment of the present invention, the network (105) may be understood to include a process of information transmission using a computer-readable recording medium. In this case, the configuration of the network is not limited to a communication medium, and should be understood to include a process of temporarily storing and physically transporting in a computer-readable memory and/or recording medium. The computer-readable recording medium used for the information transmission may be understood to mean, in particular, a relatively non-volatile or long-term recording medium such as a magnetic media such as a hard disk, a floppy disk, and a magnetic tape, an optical media such as a CD-ROM, a DVD, a magneto-optical media such as a floptical disk, or a solid state memory, which is mainly used as a means of transporting data between computing devices.

Any other means of information communication or transportation applied can be considered to fall within the scope of embodiments of the present invention as long as it has a structure that supports transmitting video data in an encoded state and decoding it. Accordingly, in addition to some of the examples listed above, all means of information communication or transportation known in the past or newly provided can fall within the scope of application of the present invention.

FIG. 2 is a conceptual diagram of the arrangement of an encoder and decoder in a real-time video streaming environment according to one embodiment of the present invention. The streaming system (200) illustrated in FIG. 2 can be considered to be applied to, for example, a video data communication network including digital broadcasting, video telephony, and video conferencing. However, it should be noted that the technical structure identical to or similar to the streaming system can be equally applied to a case where information is transmitted via a recording medium as described above.

According to one embodiment of the present invention, the streaming system may include a video source (210) that generates a video stream. The video source may include a digital video acquisition means (212), which may be, for example, a digital camera or other device, for acquiring uncompressed raw video. The raw video stream (215) may have a huge capacity and may therefore be compressed by a video encoder (217) coupled or connected to the video source.

The above encoder (217) may be configured as a means including hardware, software, or a combination of the two, configured to implement an image encoding method and/or an implementation method thereof according to one embodiment of the present invention.

Through the encoder (217), an encoded bit stream (219) with a reduced capacity compared to the original video stream can be output. The bit stream (219) can be provided in real time for communication by a relay device, which may be referred to as a streaming server (220), for example, and/or can be stored in a recording medium (225) of the streaming server (220) for subsequent use.

The streaming system (200) may include at least one streaming client (230, 240) that connects to the streaming server (220) to receive the encoded bit string (229) in real time or obtain it later. The streaming client may include a video decoder (232) that obtains the encoded bit string (229) (which may also be considered as a copy of the bit string (219) received by the streaming server), decodes the bit string (229), and outputs the resulting video data as video data in a form that can be displayed by a display (235) or other visual, auditory, or other sensory display means.

As described above, the functions for encoding and decoding video data are collectively called a coder-and-decoder, or video codec.

FIG. 3 is a conceptual diagram of a functional unit of a video decoder according to an embodiment of the present invention. As shown in FIG. 3, a receiving unit (310) can receive at least one encoded video data to be decoded by a decoder (305). In an embodiment of the present invention, the encoded video data can be independent for each reception, and a decoding procedure of each independent video data can be independent from a decoding procedure of other video data. The encoded video data can be received by the receiving unit (310) through a hardware or software connection (315) to a device storing the same, and as described above, the storing device may be a kind of streaming server located at the other end of a communication network, or may mean a physical recording medium, but is not limited thereto.

The above receiving unit (310) can receive the encoded video data together with other data accompanying it, such as encoded audio data or other auxiliary data, and each of the data can be separated from the video data and provided to an appropriate processing function unit (312) other than the video decoder.

When the video data is provided through a communication network, a buffer memory (320) may be coupled between the receiving unit (310) and the decoder (305) in order to minimize delay and disconnection according to the network environment. The buffer memory (320) may mean a computer-readable recording medium that temporarily stores the received video data and stably supplies it to a parser (330) corresponding to the input terminal of the decoder (305). However, if the bandwidth of the communication network is sufficient, if the video data is read from a recording medium in a local location that is not physically separated, or if the possibility of communication delay is not predicted in other environments, the buffer memory may be unnecessary.

The video decoder (305) may include the parser (330) as its input terminal to interpret the encoded video data. The parser may perform a function of separating (parsing) a plurality of pieces of information stored in the form of a bit string in the encoded video data according to a predetermined rule, and, if necessary, performing an entropy decoding (335) of entropy-coded video data, thereby performing a function of reconstructing symbols (338), which are paragraphs of video encoding information. The symbols (338) may include all information for controlling the operation of the decoder (305), and/or may further include information for controlling a device that is attached to the decoder (305) and can operate, such as a display device, such as a display. Control information for controlling the above display device may include information in a format called supplementary enhancement information (SEI) or video usability information (VUI).

As described above, the parser (330) may be configured to perform entropy decoding (335) of the encoded video data. The entropy encoding method of the encoded video data may vary depending on the encoding standard, and decoding may be performed correspondingly. Representative examples of the entropy encoding standard may include variable length coding, Huffman coding, and arithmetic coding, and each of the encoding methods may be a context-adaptive or context-sensitive method depending on the standard, and may be based on principles widely known to those skilled in the art.

The parser (330) may be configured to extract at least one partial image from the encoded video data. The definition of the partial image may vary depending on the encoding standard, and depending on the standard, one or more of the examples listed below may correspond simultaneously and overlappingly. The partial image may be defined as a unit such as a group of pictures (GOPs), pictures/frames, tiles, slices, macroblocks, blocks, subblocks, transform units (TUs), or prediction units (PUs).

The parser (330) may be configured to extract encoding information, such as transform coefficients, quantization parameters (QPs), and/or motion vectors, from the encoded video data. The parser (330) may be configured to perform entropy decoding (335) and parsing operations on the video data received from the buffer memory, and to selectively decode symbols (338) representing the encoding information. In addition, the parser (330) may be configured to selectively supply a specific symbol (338) to a specific decoding function unit within the decoder (305), such as an inverse quantization and inverse transform unit (340), an intra prediction unit (350), an inter prediction unit (355), or a loop filter unit (360). Control of such information supply can be determined by the information sequence contained in the encoded video, and may vary depending on the encoding standard, and is not limited within the scope of the embodiments of the present invention, and is not described in detail in this conceptual diagram.

The above decoder (305) may be composed of a plurality of conceptual functional units that receive and process the encoding information from the parser (330). It is obvious that these conceptual functional units may be combined with each other or further subdivided according to implementation needs. For example, they may be further separated for ease of implementation and integrated into one for efficiency of operation. In any case, each functional unit may be configured to perform close interaction with each other. However, despite the possibility of such integration or separation, the following will be described as a combination of conceptual functional units in order to illustrate a decoding procedure of video data applied as an embodiment of the present invention.

The above decoder may include an inverse quantization and inverse transformation unit (340). The inverse quantization and inverse transformation unit (340) may be configured to receive encoding information including a method to be used for numerical transformation from the parser (330), a size of a block, quantization coefficients for recovering quantized information, and distinction information of a quantization matrix that simplifies and represents the quantized coefficients, and may be configured to output block values (341) that may be input to an aggregator (370) as a result of processing the encoding information.

In one embodiment of the present invention, the output values of the inverse quantization and inverse transformation unit (340) may include intra-prediction encoded block values. The intra-predicted block values may mean values that can be decoded without using prediction information from a previously decoded partial image, for example, a previous frame, but using prediction information within a partial image currently being decoded, for example, a current frame.

The prediction information within the current partial image may be provided by the intra prediction unit (350). According to an embodiment of the present invention, the intra prediction unit (350) generates a block value having the same form as the block being decoded as the prediction information by using the image information of a spatially adjacent area derived from a partial image currently being decoded and of which decoding has been partially completed. The partial image information may be provided (381) from a current image buffer, a so-called line buffer (380). The merging unit (370) may be configured to merge the prediction information (351) generated by the intra prediction unit (350) with the block values (341) provided by the inverse quantization and inverse transformation unit (340), according to an embodiment.

In another embodiment, the output values of the inverse quantization and inverse transformation unit (340) may include block values subjected to motion compensation as inter-prediction encoded block values, and in some cases, block values subjected to motion compensation. In this case, the inter-prediction unit (355) may extract and use sample information (386) used for motion-based prediction from a reference picture buffer (385). The information (356) derived by performing motion compensation on the sample information based on the symbols (338) included in the block values as the output values may be configured to be merged with the block values (341) provided by the inverse quantization and inverse transformation unit (340) by the merging unit (370). In this case, the block values (341) may be referred to as so-called differential or residual values.

The position information in the memory used by the inter prediction unit (355) to extract the sample information from the reference image may be determined by a motion vector provided to the inter prediction unit (355) which is composed of a combination of symbols (338) for representing, for example, X, Y, and other specific points of the reference image. The inter prediction unit (355) may also include a function capable of interpolating and using the sample values when a so-called 'subsampling' capable motion vector is provided, and may further include a function for predicting and reinforcing the value of the motion vector.

The output values (371) of the above merging unit (370) may be provided to the loop filter unit (360) and processed by various loop filtering methods. The loop filter unit (360) may be configured to receive not only the block unit output (371) of the merging unit (370) but also the symbol (338) provided from the parser (330) and control its operation. The output of the loop filter unit (360) may be output to an external display means such as the display device through an output connection (390), but may be stored (361) in a line buffer (380) for use in prediction for interpreting a subsequent intra or inter encoded block value, and may also be stored in a reference image buffer (385) through this.

Specific partial images, such as frames, after their decoding is completed, can be utilized as reference images for performing predictive decoding in a subsequent decoding process. One partial image, such as a frame, can be gradually accumulated in a line buffer (380) and decoded, and when one frame is decoded, the contents of the line buffer (380) can be transferred (383) to the reference image buffer (385), and a new line buffer (380) can be allocated for decoding a new frame.

The above video decoder (305) may be configured to perform a decoding operation according to a predetermined video compression technique that may be documented by various international standard specifications or commercial standards. The standards may include, for example, international standard recommendations such as H.264, H.265, and H.266 defined by the International Telecommunication Union Standardization Sub-Tier (ITU-T). A person skilled in the art will understand that each of the above recommendations is equivalent to an international standard jointly defined by the International Organization for Standardization (ISO) and the International Electrotechnical Conference (IEC). The encoded video data may comply with a specific bitstream syntax defined by the corresponding standard as defined in the video compression standard document and standard document, and specifically, by the profile and level specified within such document. In addition, the complexity of the encoded video data may be limited to a certain level in order to comply with the profile and level. For example, a profile or level may be configured to limit a maximum picture size, a maximum decoding speed, and a maximum reference picture size. The limitations may also, in some embodiments, be further limited via metadata signals for a hypothetical reference decoder (HRD) and HRD buffer management included in the encoded video data.

According to one embodiment of the present invention, the receiver (310) may receive additional redundant data together with the encoded video. The additional data may be considered as a part of the encoded video data. The additional data may include information that may be used by the decoder (305) to properly decode the data, or to more accurately reconstruct an image that approximates the original image before encoding. The additional data may be provided in the form of, for example, layers for temporal, spatial, or signal-to-noise ratio (SNR) enhancement, redundant slices, redundant pictures, and forward error correction codes.

FIG. 4 is a functional unit conceptual diagram of a video encoder according to one embodiment of the present invention. The encoder (405) may be configured to receive original video information (402) from a video source (401) and perform encoding.

The above original video information (402) may have any suitable bit depth, for example, 8 bits, 10 bits, 12 bits, etc. In addition, the original video information (402) may have any suitable color space, for example, R/G/B, Y/U/V, Y/Cb/Cr, etc. In addition, the original video source may have any suitable sampling structure corresponding to the color space, for example, may have a format such as Y/Cb/Cr 4:2:0, Y/Cb/Cr 4:4:4. The original video source having such a predetermined format may be provided to the encoder in the form of a digital video stream.

In a one-way video communication network, the original video information (402) can be obtained from a recording medium storing a previously prepared video original. In a two-way video communication network, the original video information (402) can be obtained from an image acquisition device, such as a camera, that generates at least one video transmission stream included in the two-way video communication.

The video data including the above original video information (402) may be configured as a plurality of partial images configured to simulate motion by being played back in time order. The partial images may be expressed as concepts such as pictures or frames, for example. The partial images may include one or more samples depending on the type of sampling structure, color space, etc. being used. A person skilled in the art will understand that the sample and the pixel in a digital image are closely related terms. Hereinafter, the operation of the encoder will be described based on such samples.

According to one embodiment of the present invention, the encoder (405) may be configured to encode and compress (partial) images constituting the original video information (402) in real time (or according to other temporal requirements required according to the implementation method) into the form of encoded video information.

In the above encoder (405), the control unit (450) may be a functional unit configured to control an appropriate encoding speed. The control unit (450) may be configured to control other functional units and be functionally coupled to the following functional units as described below. The parameters set by the control unit (450) may include parameters related to bitrate control, such as skip of the picture, quantizer, variable values for applying a picture quality optimization technique, and may also include values such as the size of the picture, the structure of a group of pictures (GOP), and the maximum search range of a motion vector. A person skilled in the art will be able to understand various other functions that the control unit (450) may have, and such other functions may be added or removed according to the design of a video encoder optimized for an individual system design.

According to an embodiment of the present invention, the encoder (405) may be configured to operate in a structure such as a "coding loop" well known to those skilled in the art. To briefly explain by way of example, the coding loop may be configured with an internal encoder (so-called "source coder") (410) which is responsible for receiving an image to be encoded and generating symbols based on at least one reference image that has been encoded in the past, and a local decoder (420) which is configured to be connected to the internal encoder. The local decoder (420) may be configured to perform an operation to reproduce sample data to be generated by a decoder (490) located at an actual remote location that will receive encoded video information from the encoder (405) by receiving an output of the internal encoder (410).

The video data composed of sample data reconstructed by the internal decoder (420) may be configured to be input to the reference picture buffer of the encoder (405). As described above, the internal decoder (420) is implemented to reproduce the result output by the encoder (405) and to be decoded by a remote decoder, so the video data recorded in the reference picture buffer may also be identical in bit units to the information of the reference picture buffer of the remote decoder. That is, the prediction function unit that may be included in the encoder (405) may read the same values as the sample values of the previous frame that the decoder will later refer to in the decoding process from the reference picture buffer of the encoder (405).

As described above, the principle of achieving matching of reference image buffers between the encoder (405) and the decoder (490) by means of the internal decoder (420) on the encoder (405) side is well known to those skilled in the art, and a method of responding to an environment in which such an environment is not guaranteed (e.g., information loss due to communication failure, etc.) can also follow what is well known to those skilled in the art.

One embodiment of the operation method of the internal decoder (420) has been described in detail with reference to FIG. 3 above. The decoder of FIG. 3 may be considered as the "remote" decoder (490) described above. The internal decoder (420) may be implemented excluding lossless encoding and decoding sections such as the parser (330) or the entropy decoding (335). This is because the internal encoder (405) is implemented to simply reproduce the operation of the decoder located at a remote location, and thus the symbols may be decoded directly without requiring a process of compressing and then decompressing the symbols. Accordingly, the functional sections preceding the parser and the entropy decoder as shown in FIG. 3 may not be provided or may be implemented at least partially.

As described above, according to a preferred embodiment of the present invention, any decoder function (excluding a parser and an entropy decoder) present in the decoder can naturally exist as a substantially identical function in the corresponding encoder (405).

The operation of the encoding function unit that may be included in the above encoder (405) may be considered as the reverse of the decoder function unit. Therefore, the embodiment may be explained by generally performing the operation of the decoder function unit in reverse. For example, a quantization and transform function unit corresponding to the inverse quantization and inverse transform unit may be provided, and an inter prediction encoding unit corresponding to the inter prediction unit may be provided. In addition, some additional explanations will be added.

The above internal encoder (410) may be configured to perform encoding on input image information, for example, an input frame, by a predictive encoding method executed by a predictive encoding unit (440) that operates by referencing at least one temporally previous encoded partial image, for example, frames, from a reference image buffer (430) from at least one reference image information, for example, video data designated as a reference frame. In this case, the encoder (405) may be configured to encode a differential between blocks of samples constituting the input image and blocks of samples constituting the reference image.

The internal decoder (420) can decode video data that can be designated as the reference picture from symbols generated by the internal encoder (410). As described above, since the video data is identical to the decoding operation performed by the remote decoder, the video data used as the reference picture can be provided to the encoder (405) in a form in which some damage has occurred due to lossy compression, and this operation can be intended for operation consistency with the decoder.

The prediction encoding unit (440) may be configured to perform a prediction search operation within the encoder (405). The prediction search operation may mean an operation corresponding to the inter prediction or intra prediction described in the description of the decoder. For image information that is input and is scheduled to be newly encoded, the prediction unit may access the reference image buffer (430) to retrieve information, such as a motion vector, a block shape, and metadata that may include the motion vector, which are information indicating points of a reference image that can function as prediction reference information suitable for the new image information, and a sample block to be actually referenced. The prediction encoding unit (440) may operate on the basis of the so-called "sample block by pixel block" in order to obtain appropriate prediction reference information. According to one embodiment of the present invention, at least one prediction reference information designating at least one reference image information stored in the reference image buffer (430) may be specified for the input image, as determined based on the search results obtained by the prediction encoding unit (440).

In one embodiment of the present invention, the control unit (450) may be configured to manage the overall encoding operation of the internal encoder (410), including setting parameters used to encode video data.

All outputs of the above-described functional units may be subjected to entropy encoding (460) in order to be finally output. The entropy encoding (460) may include various entropy coding techniques, such as variable length coding, Huffman coding, and arithmetic coding, for the symbols generated by the various functional units as described above, and each encoding method may be a context-adaptive or context-sensitive method according to the standard, and may also be based on principles widely known to those skilled in the art. The entropy encoding (460) can typically achieve lossless compression, and thus may be configured to convert at least one symbol generated by the functional units into encoded video data.

The above control unit (450) can apply the type of encoding of a specific partial image during an encoding section to each partial image, such as a picture or a frame, when controlling the operation of the encoder (405). Depending on the type, the method by which the partial image is encoded can be affected. Depending on the embodiment, the type can include what is classified as a “frame type” as follows.

Fig. 5 is a conceptual diagram of a frame type according to one embodiment of the present invention. The following description will be made with reference to Fig. 5.

An intra ("I") picture (510) may mean a picture that can be encoded and decoded only with its own information without referring to other picture information in the video data by predictive encoding. The "I" picture may be designated by names such as a key frame, an independent/instantaneous decoder referh (IDR) frame, and a clean random-access (CRA) frame according to a video encoding standard, and the "I" pictures designated by various names as described above may have various modifications and application methods as permitted by each standard and may be partially different from each other. In addition to those listed above, various application methods for implementing the "I" picture may be by various methods that are already known to those skilled in the art or may be newly provided.

A prediction ("P") picture (520) may mean a picture that can be encoded and decoded through intra or inter prediction based on at least one prediction information and/or a motion vector that designates at least one reference picture to predict sample values of a block constituting the picture. The "P" picture may be configured to refer to only one reference frame, or may be configured to refer to one or more reference frames, according to a video encoding standard. When referring to one or more reference frames, sample information and/or associated metadata derived from a plurality of reference pictures may be used for reconstructing a single block. However, in common cases, a picture designated as a "P" picture may be understood as a picture that performs reference only to a temporally preceding picture.

A bidirectional prediction ("B") picture (530) may mean a picture that can be encoded and decoded through intra or inter prediction based on at least one prediction information and/or a motion vector that designates at least two reference pictures to predict sample values of blocks constituting the picture. In a common case, a picture designated as the "B" picture is distinguished from a picture designated as the "P" picture, and may be understood as a picture that performs a reference without being limited to a temporally preceding picture.

Video data may be spatially divided into a plurality of sample blocks in the process of encoding and decoding, and encoding may be performed in block units. The block units may include, but are not limited to, sizes such as 4x4, 8x8, 4x8, or 16x16 in units of horizontal/vertical pixels, as is widely known. The block may be encoded by a predictive encoding method with reference to any other (already encoded) blocks, as permitted and/or restricted by the type specified for each partial picture including the block. For example, the blocks of the "I" picture (510) may be encoded without using a predictive encoding method, or with reference to blocks already encoded in the same partial picture. That is, only the so-called intra prediction method may be used. In contrast, the "P" picture (520) may further refer to a reference picture encoded in at least one previous time unit, and thus inter prediction may also be used for encoding along with intra prediction. In the case of a "B" picture (530), reference can be made not only to a previously encoded picture in the encoding order but also to a later reference picture in terms of time unit. However, it is widely known that there may exist blocks within a "P" picture or a "B" picture that are encoded without relying on predictive encoding.

The above video encoder (405) may be configured to perform an encoding operation according to a predetermined video compression technique that may be documented by various international standard specifications or commercial standards. Examples of the above standards may include all of those described in the above decoder.

According to one embodiment of the present invention, the transmitter (470) may buffer the encoded video data generated by the entropy encoding in order to provide/transmit the video data (ultimately to a remote decoder (490)) to a device storing the encoded video data via a hardware or software connection (495). According to an embodiment, when providing/transmitting the encoded video data from the video encoder (405), the transmitter (470) may receive and merge other data accompanying the encoded video data, for example, encoded audio data or other auxiliary data, from a separate source (480).

According to one embodiment of the present invention, the transmitter (470) may be configured to further transmit additional data together with the encoded video. The additional data may be considered as part of the encoded video data. The additional data may include information that may be used by the decoder to properly decode the data, or to more accurately reconstruct an image that approximates the original image before encoding. Examples of the additional data may include all of the examples presented above with respect to the receiver (310) of the decoder.

The present invention can be implemented by a digital video compression standard widely used and understood by those skilled in the art as described above. The digital video compression standard may include at least one of compression standards known by standard names such as MPEG-2, MPEG-4 Video, H.263, H.264/AVC, H.265/HEVC, H.266/VVC, VC-1, AV1, QuickTime, VP-9, VP-10, and Motion JPEG.

Fig. 6 is a conceptual diagram showing the structure of a video encoder according to the H.266/VVC standard. What is shown in Fig. 6 corresponds to the rough structure of a video encoder widely known as a standard code such as ITU-T H.266 and ISO/IEC 23090-3, and also under the name of MPEG-I Part 3 or the general name of versatile video coding (VVC).

According to FIG. 6, the video encoder (605) may be configured to receive uncompressed and unencoded original video data (601) as input and output an encoded bit stream (602). The video data (601) may be supplied directly to a luma mapping unit (610a) in the case of intra encoding, or may be supplied to a luma mapping unit (610b) via an inter prediction unit (620) including motion vector extraction. In the case of intra encoding, the mapped luma signal may be supplied to an output merger (606) alone, or by selecting (608) at least one of an intra prediction encoded signal via the intra prediction unit (625) or an inter prediction encoded signal output from the luma mapping unit (610b) via the inter prediction unit (620). The result of the above output merger can be applied to a chroma scaling unit (615). (The operation of the luminance signal mapping unit (610) and the operation of the chroma scaling unit (615) are collectively referred to as a luma mapping/chroma scalaing (LMCS) process.) The reduced chroma signal can be provided to a transform unit (630), and the transform unit (630) can perform an adaptive color transform, particularly on the chroma signal. The coefficient derived as a result of the transform is applied to a quantization unit (640) and quantized. As a result, lossy compression is achieved, and the result of the lossy compression can be output as a bit string (602) through a multi-hypothesis CABAC (650), which is a lossless compression method.

Meanwhile, the result of the lossy compression may enter a decoding procedure by going through inverse quantization (645), inverse transform (635), and luma signal expansion (617) processes to generate a coding loop. The result of the luma signal expansion may be supplied to an internal merger (607) together with a result of selecting (608) at least one of the previously generated intra prediction encoding signal or inter prediction encoding signal. The result of the internal merger may go through inverse luma mapping (617), and then may go through processing such as a deblocking filter (660), a sample adaptive offset (SAO) (670), and an adaptive loop filter (ALF) to reproduce an image quality improvement process in a decoder. The result of reproducing the operation in the decoder as described above is applied to the reference image buffer (690) and can be reused for prediction encoding by the inter prediction unit (620).

The present invention can also be utilized by or in combination with the enhanced compression model (ECM), which is an implementation of a next-generation video codec being developed by the joint video experts team (JVET), an international standardization expert group, to improve H.266/VVC. According to the standardization progress document of the above JVET, document number ISO/IEC JTC 1/SC 29/WG 5 N 190 (also document number JVET AC2025-v1), the improved compression model may include an improved intra prediction coding method, an improved inter prediction coding method, an improved transform and transform coefficient coding method, an improved adaptive loop filtering method, a bilateral filtering method, a new sample adaptive offset (SAO) method for image quality improvement, an extended entropy coding method, and an improved gradual decoding refresh (GDR) technique, and such techniques may be included in the present invention as background techniques for implementing the present invention.

General LIC application method

The present invention provides a method for applying an improved local illuminance compensation (LIC) technique that can be used in the video encoding and decoding field including the above-described embodiments, and a device to which such a method is applied.

FIG. 7 is a conceptual diagram showing the general structure of a LIC application method according to an embodiment of the present invention. In order to solve the problem of reduced compression efficiency due to the difference in illumination between frames, Weighted Prediction (WP) is applied in traditional video coding standards. The WP is applied on a frame-by-frame basis, and is a technology that compensates for brightness changes in a reference block in order to accurately obtain a prediction value when encoding the current frame. The WP is applied to both H.264/AVC and H.265/HEVC, and can be used by dividing it into an explicit mode and an implicit mode depending on whether or not a brightness compensation parameter is transmitted for each slice.

The above WP-based technology can compensate for the predicted value of the reference frame for the entire image by responding well to the brightness change between frames, but it cannot properly perform brightness compensation for local brightness changes due to technical limitations applied on a frame-by-frame basis. To overcome this, the LIC technology is a technology that induces appropriate brightness compensation for local predicted brightness changes.

The above LIC is a technology for modeling (760) local brightness changes between a final prediction block (770) for a current block (710) and a final prediction block (770) by a function (750) that approximates between a value (730) based on a current block (710) and a value (740) based on a reference sample (720) between a template (715) adjacent to a current block (710) and a template (725) adjacent to a reference block (720). The parameters of the function can be represented by a scale α and an offset β that form a linear equation, which can be expressed in the form of α*p[x]+β as a linear equation for compensating for brightness changes, where p[x] can mean a reference sample pointed to by a motion vector (MV) (717) at a location x on a reference picture in an inter prediction method. In one implementation of the above LIC, when wrap around motion compensation is enabled, the MV can be clipped considering an offset that takes into account the circular pixel space structure. Since α and β can be derived based on the current block template and the reference block template, no signaling overhead is required for them, except when a LIC usage indication flag is signaled in the bitstream to indicate the usage of the LIC (preferably for the Advanced Motion Vector Prediction (AMVP) mode).

However, the LIC technology disclosed in the present invention can be equally utilized for intra prediction, and for example, in the case where a BV (block vector) for intra prediction is used instead of the MV (717) as a prediction vector for the current block, in the case where the MV and/or the BV is derived from a decoder, in the case where a chained vector is used that is determined by sequentially referencing the MV and/or the BV, or in the case where any type of prediction vector is used by any other method, the present invention does not exclude the possibility that a method based on the LIC technology according to an embodiment of the present invention may be equally applied.

Among the applied implementation methods of the present invention, an implementation method citing the method proposed in JVET Technical Contribution JVET-O0066 can be modified as follows and used for a block that is inter-coded between screens, for example, an inter-coded unit (inter-coded CU).

According to one embodiment of the present invention, neighboring samples in the intra aspect can be used to derive the LIC parameter. This allows the accuracy of LIC to be improved by utilizing information of intra-encoded neighboring blocks.

According to one embodiment of the present invention, LIC can be disabled for blocks having less than 32 luminance samples. This reduces unnecessary operations and improves coding efficiency for small-sized blocks.

According to one embodiment of the present invention, in non-sub-block and affine mode, LIC parameter derivation can be performed based on template block samples corresponding to the current CU instead of partial template block samples corresponding to the first 16x16 unit from the upper left. This allows more information to be utilized to improve the accuracy of LIC.

According to one embodiment of the present invention, samples of a reference block template can be generated using MC (motion compensation) together with the MV of the corresponding block without rounding to integer pixel precision. This enables more precise prediction, thereby improving coding efficiency.

According to one embodiment of the present invention, in case of a bi-directional inter prediction CU, two sets of LIC parameters may be derived separately for L0 and L1 prediction samples. The L0 and L1 prediction samples may mean respective samples obtained in two prediction directions used in the bi-directional prediction. For example, according to one embodiment, the L0 prediction sample may mean a prediction sample derived from a reference picture that is generally temporally earlier than the current picture. That is, it may represent prediction information obtained from a past frame. Furthermore, according to one embodiment, the L1 prediction sample may mean a prediction sample derived from a reference picture that is generally temporally later than the current picture. That is, it may represent prediction information obtained from a future frame.

According to one embodiment of the present invention, the L0 and L1 prediction samples may be obtained from different reference pictures, and they may be at different temporal positions. As described above, by separately deriving the LIC parameters by utilizing the L0 and L1 prediction samples, more accurate brightness compensation considering temporal directionality can be provided.

According to one embodiment of the present invention, the same method can be repeatedly applied to derive the L0 and L1 LIC parameters. Specifically, the L0 LIC parameter is first derived by minimizing the difference between the L0 template prediction T0 and the template T, and the samples of T can be updated by subtracting the corresponding samples of T0. Next, the L1 parameter that minimizes the difference between the L1 template prediction T1 and the updated template can be calculated. Finally, the L0 parameter can also be refined again in the same manner.

According to another embodiment of the present invention, in the process of separately deriving LIC parameters for the L0 and L1 prediction samples, weights considering temporal characteristics of each direction can be applied. For example, if the L0 prediction sample is temporally closer to the current picture, a higher weight can be given to the L0 LIC parameter. Conversely, if the L1 prediction sample is closer, a higher weight can be given to the L1 LIC parameter. This allows for more effective utilization of temporal correlation. The present invention proposes a method and device for enhancing compression performance by appropriately performing brightness compensation of LIC.

How to decide on a template

The above-described LIC-based method can be configured to find a reference block by utilizing a set template, and generate a change in brightness between a current block (e.g., CU) and the reference block as a linear model by utilizing an L-shaped template. The linear model can be configured in the form of a linear function of α*p[x]+β. The p[x] can be defined as a reference value as a prediction pixel, and the α and β can represent a slope and a bias of the derived linear model. According to research in the standardization activity of JVET in the past, two or more linear models can be used at the same time. According to one embodiment of the present invention, the method of deriving the linear model parameters is not limited to a first-order linear model, but can be set as a second-order or higher-order line fitting model. This can help model more complex brightness changes more accurately.

According to one embodiment of the present invention, the LIC template setting method can be improved to provide various modes. That is, the L-shaped template may not be used uniformly. For example, a mode that uses only the top or left template may be used, or a mode that uses the top and left templates but uses each template with the same weight or with different weights may be used. Fig. 8 is an exemplary diagram showing an example of a template application mode according to one embodiment of the present invention. In summary, the following modes may be included and used in the LIC template setting process, but are not limited to these examples.

(a) Mode that uses only the top pixel (815) of the current block (810) as a template

(b) Mode that uses only the left pixel (827) of the current block (820) as a template

(c) Mode that uses the top (835) and left (837) pixels of the current block (830) as templates with the same weight (1:1).

(d) Mode that uses the top (845) and left (847) pixels of the current block (840) as templates with different weights (N:M).

According to one embodiment of the present invention, the mode of using only the upper pixel of the current block as a template may use at least one pixel selected based on pixels located within a distance of at least one pixel in the upper direction of the current block as a template. In addition, according to one embodiment of the present invention, the width of the template region corresponding to the pixel located at the upper end may be extended to the left or right by the number of unused left pixels. For example, when the size of the current block is 16x16 and the left pixel is not used for the template, the width of the upper template region may be extended from 16 pixels to 32 pixels.

According to one embodiment of the present invention, the mode of using only the left pixel of the current block as a template may use pixels located within a distance of at least 1 pixel in the left direction of the current block as templates. In addition, according to one embodiment of the present invention, the height of the template region corresponding to the pixels located on the left may be extended upward or downward by the number of unused upper pixels. For example, when the size of the current block is 16x16 and the upper pixels are not used for the template, the height of the left template region may be extended from 16 pixels to 32 pixels.

According to one embodiment of the present invention, when setting a LIC template, at least one of one or more modes including the above-described modes may be selected and used. In this case, as a method of selecting one of the above-described one or more modes, at least one of the methods described below may be used.

According to one embodiment of the present invention, as a method for selecting the mode, a method may be used in which an encoder selects a mode exhibiting an optimal bit rate-to-distortion (RD) performance by using Rate-Distortion Optimization (RDO) as an evaluation criterion, and informs a decoder of the corresponding mode information. According to one embodiment, assuming that the four types of template modes exemplified through the above FIG. 8 are supported, RDO evaluation may be performed on each of the above (a) to (d) modes, and a mode exhibiting the lowest encoding cost and/or good RD performance may be selected as the LIC usage mode of the current block, and the corresponding information may be included in a bit stream and transmitted to the decoder. The decoder may parse the bit stream and then configure a template according to the selected LIC usage mode. Other LIC-based prediction block-related encoding and decoding processes may be performed according to the same or similar procedures as in the past.

According to one embodiment of the present invention, a method of utilizing a template may be used as a method of selecting the mode. According to one embodiment, the mode may be selected by an operation based on each L-shaped template (i.e., an upper pixel and a left pixel) composed of surrounding pixels of a current block and a reference block. According to one embodiment, an optimal template mode may be selected by utilizing an upper restoration value based on an L-shaped template of a current block and an upper value of a reference block (first mode), utilizing a left restoration value based on an L-shaped template of a current block and a left value of a reference block (second mode), utilizing an upper and left restoration value based on an L-shaped template of a current block and an upper and left value of a reference block with the same weight (third mode, equal weight), or utilizing an upper and left restoration value based on an L-shaped template of a current block and an upper and left value of a reference block with different weights (fourth mode, different weight). When selecting the above optimal template mode, the SAD (Sum of Absolute Differences), SATD (Sum of Absolute Transformed Differences), or MR-SAD (Mean-removal SAD) values and/or their average values, which are calculated by utilizing values based on the templates (and/or templates of the templates) of the current block and the reference block, may be utilized as an evaluation criterion. According to a preferred embodiment of the present invention, the above-described evaluation method may be configured to operate identically in the encoder and the decoder based on a pixel area that has been previously encoded/decoded in the encoding/decoding process. Accordingly, when applying this method, since the template mode can be determined implicitly, there is no need to record separate information in the bit string regarding which specific mode is used, and the decoder may be configured to determine the mode by itself.

According to one embodiment of the present invention, as a method of selecting the mode, the LIC mode can be explicitly determined by utilizing information of surrounding blocks. According to one embodiment, when there are available blocks on the upper and left sides, the LIC mode of the current block can be explicitly determined by utilizing prediction information of available surrounding blocks. According to one embodiment of the present invention, the explicit determination method can include a method of configuring a candidate set and selecting at least one candidate belonging to the candidate set as the LIC mode. The method of configuring the candidate set can be provided in various ways. For example, when there is a block using LIC around the current LIC block, the corresponding LIC mode can be included in the candidate set. In addition, usage history information on the LIC mode of a block previously used as the LIC mode can be stored and included in the candidate set. In addition, even if LIC is not used in the surrounding block, if at least one surrounding block is encoded by the intra-angular prediction mode, the appropriate LIC mode can be included in the candidate group by considering the angle of the directional prediction mode. For example, if the directional prediction mode has a horizontal angle, a mode that mainly refers to horizontal pixels as shown in (b) of FIG. 8 as a more appropriate LIC mode for the direction can be included in the candidate group. In addition, the candidate group can also be configured by considering information related to the inter-screen prediction method. When configuring the candidate group, according to one embodiment of the present invention, if a specific number of candidate group values cannot be selected when configuring the candidate group, the modes of the candidate group can be arbitrarily added according to a predetermined order or information. According to one embodiment of the present invention, when the candidate group is configured, the template (or template of the template) (for example, one upper line, one left line, one upper and left line, etc.) can be used to perform re-ordering of the order of the candidate group. The above reordering can be performed based on the encoding information requirement cost calculated according to the template matching result, and according to one embodiment, the reordering can be performed in order of low cost. According to one embodiment, the information requirement cost can be calculated by SAD, SATD, or MR-SAD, and the reordering can also be performed based on this cost.

Finally, once the above candidate group is formed, the optimal LIC mode can be selected through RDO, etc., and an index thereof can be included in a bit string and transmitted to a decoder. The decoder can be configured to form the candidate group in the same manner as the encoder, and to reconstruct the LIC mode used in the encoder by utilizing the information transmitted through the bit string.

According to one embodiment of the present invention, as a method of selecting the mode, the LIC mode can be implicitly determined by utilizing information of surrounding blocks. According to one embodiment, when there are available blocks on the top and left, the LIC mode of the current block can be implicitly determined by utilizing available surrounding prediction information. For example, when the current block is encoded by LIC, if there is a block using LIC in the surrounding, the LIC mode of the current block can be determined based on the frequency of the LIC mode used in the surrounding block. In addition, even if LIC is not used in the surrounding block, if at least one surrounding block is encoded by the intra angular prediction mode, the appropriate LIC mode can be determined by considering the angle of the directional prediction mode. For example, when the directional prediction mode has a horizontal angle, as a LIC mode more appropriate for the corresponding direction, a mode that mainly refers to pixels in the horizontal direction, as shown in (b) of FIG. 8, can be determined as the LIC mode of the current block. According to a preferred embodiment of the present invention, the above-described LIC mode selection method can be configured to operate identically in an encoder and a decoder based on a pixel area that has been previously encoded/decoded in an encoding/decoding process. Therefore, when applying this method, there is no need to record separate information in a bit string regarding which specific mode has been used, and the decoder can be configured to check the LIC mode by itself.

According to one embodiment of the present invention, in the method for deriving the mode on the decoder side using the implicit decision method in the above-described embodiment, it is obvious that various decoder-side mode derivation methods known in the art or newly provided can be commonly used or applied.

According to one embodiment of the present invention, in order to set up a LIC template, a mode in which the top and the left are both used with the same weight, or the top and the left are used with different weights, can be defined and used. Various methods can be used to define the weights. According to one embodiment, the same weights used in a conventional existing LIC can be applied and utilized when generating α and/or β of the linear model. According to another embodiment, the top and left regions can be divided, and the weights according to the regions can be applied separately when generating α and/or β. According to one embodiment of the present invention, the weights for each of the divided regions can use values determined in advance through experiments, etc., and the determined weight values can be used by forming a table, and an index designating a specific value of the table can be determined explicitly or implicitly and provided to a decoder.

According to one embodiment of the present invention, the table may be preferably specified in a form in which the sum of one weight set is a power of 2. This may have the purpose of easily replacing the division operation in the weighted sum operation based on the weight with a shift operation. For example, if the weight set is {3, 5}, since the sum becomes 8 (2 ³ ), the division operation required for the weight set can be replaced with a 3-bit right binary bit shift operation.

According to one embodiment of the present invention, the weight value may be applied in a form that is determined on-the-fly by utilizing information of surrounding blocks. It will be easily understood that this method corresponds to a kind of implicit signaling method. For example, when there are available blocks on the top and left, the weight value may be determined by utilizing information including inter-prediction information, intra-prediction information, and block size of the available surrounding blocks. In this case as well, similarly to the above, the weight value may be determined as a value that is a power of 2 for the purpose of easily replacing a division operation with a shift operation.

According to one embodiment of the present invention, in order to reduce the complexity of a method of selecting one of the modes described above when setting a LIC template, a method of reducing the number of pixels used in the template by limiting the pixels to some pixels rather than all pixels within a template area may be applied. According to one embodiment, when generating α and β after finding a reference block, only some pixels rather than all may be utilized. According to one embodiment, when finding a reference block, all pixels of the template may be additionally utilized, and when generating α and β, only some pixels rather than all may be utilized. According to one embodiment, when finding a reference block, pixels of a template reduced to include only some pixels rather than all may be additionally utilized, and when generating α and β, all pixels may be utilized.

As described above, for the purpose of reducing complexity, methods for reducing the use of only some, rather than all, pixels used in the template may include, but are not limited to, the following methods.

According to one embodiment of the present invention, when using the top, left, or top and left full templates, the number of pixels is reduced by utilizing a subsampling method corresponding to a multiple of 2, such as 2:1, 4:1, etc., and the reduced pixels can be used to find a reference block as described above or to generate α and β.

According to one embodiment of the present invention, when only the upper template is used, the number of pixels is reduced by horizontally dividing the area of the entire upper template into N and selecting M pixels, and the reduced pixels can be used to find a reference block as described above or to generate α and β. According to an embodiment, the N and the M can be varied depending on the block size and the level of desired complexity. According to a preferred embodiment of the present invention, at least one of the N or M can be defined based on a power of 2 for ease of operation.

For example, the region of the entire top template may be horizontally divided into four parts, and two pixels corresponding to regions 1/4 and 3/4 may be selected, and the selected pixels may be used to find a reference block or to generate α and β. For another example, the region of the entire top template may be horizontally divided into eight parts, and three pixels corresponding to regions 2/8, 4/8, and 6/8 may be selected, and the selected pixels may be used to find a reference block or to generate α and β. For another example, the region of the entire top template may be horizontally divided into eight parts, and four pixels corresponding to regions 1/8, 3/8, 5/8, and 7/8 may be selected, and the selected pixels may be used to find a reference block or to generate α and β. As another example, when only the upper template is used, the area of the entire upper template is divided into eight horizontal sections, and five pixels corresponding to areas 0/8, 2/8, 4/8, 6/8, and 8/8 are selected, and the selected pixels can be used to find a reference block or to generate α and β.

According to one embodiment of the present invention, when only the left template is used, the number of pixels is reduced by vertically dividing the area of the entire left template by N and selecting M pixels, and the reduced pixels can be used to find a reference block as described above or to generate α and β. According to an embodiment, the N and the M can be varied depending on the block size and the level of desired complexity. According to a preferred embodiment of the present invention, at least one of the N or M can be defined based on a power of 2 for ease of operation.

For example, the entire left template may be vertically divided into four sections, and two pixels corresponding to sections 1/4 and 3/4 may be selected, and the selected pixels may be used to find a reference block or to generate α and β. For another example, the entire left template may be vertically divided into eight sections, and three pixels corresponding to sections 2/8, 4/8, and 6/8 may be selected, and the selected pixels may be used to find a reference block or to generate α and β. For another example, the entire left template may be vertically divided into eight sections, and four pixels corresponding to sections 1/8, 3/8, 5/8, and 7/8 may be selected, and the selected pixels may be used to find a reference block or to generate α and β. As another example, the area of the entire left template can be divided vertically into eight parts, and five pixels corresponding to areas 0/8, 2/8, 4/8, 6/8, and 8/8 can be selected, and the selected pixels can be used to find a reference block or to generate α and β.

In the description of the various methods described above, the entire top template may mean a top template that extends to the left or right from the width of the current block to the height of the current block. In addition, the entire left template may mean a left template that extends to the top or bottom from the height of the current block to the width of the current block.

3x3 cross-shaped filter

According to one embodiment of the present invention, when applying the LIC method, after setting an L-shaped template from restored pixel values on the top and left of a current block currently being encoded/decoded, coefficients that can be used for a cross-shaped filter having a size of 3x3 can be generated and used from the template area. FIG. 9 is a conceptual diagram showing the shape and coefficient positions of a cross filter according to one embodiment of the present invention. Referring to FIG. 9, a cross-shaped filter (900) having a size of 3x3 as described above can be expressed by coefficient positions of A (left), B (right), C (upper), D (lower), and E (center).

According to various implementation methods of the present invention, a value filtered through the 3x3 cross-shaped filter can be expressed by one of the following mathematical formulas.

The above mathematical expression 1 represents basic linear filtering, which can be calculated by multiplying the pixel value at each location by a pre-specified coefficient (C ₁ to C ₅ ) and adding them up.

The above mathematical expression 2 represents the addition of a nonlinear term F to the basic linear filtering, and the F may mean that the square value of the central pixel E is adjusted according to the bit depth value (bitDepth) per pixel (at this time, "bitDepth_mid" may mean a median value among the range values for expressing the pixel value according to the bit depth per pixel). The ">>" operator symbol appearing in the above mathematical expression 2 may be understood as a right binary bit shift operation. Of course, depending on the embodiment, the elements constituting the nonlinear term F may change. For example, the nonlinear term F may be based on another pixel value instead of E. For another example, the nonlinear term F may be defined by an equation of the third or higher degree. For another example, constant values such as "bitDepth" and "bitDepth_mid" included and calculated in the nonlinear term may be replaced by other suitable constant values.

The above mathematical expression 3 represents the addition of a constant term G to the above mathematical expression 2, where G may be "bitDepth_mid" and may mean a median value among the expression range values of the pixel value. Of course, depending on the embodiment, the elements constituting the constant term F may change. For example, the value of the constant term F may be replaced by another suitable constant value.

According to one embodiment of the present invention, the coefficients designated for the coefficients C ₁ to C ₇ shown in the mathematical expressions 1 to 3 may be derived from an L-shaped template which may be derived from restored pixel values on the top and/or the left side of the current block. Of course, the various application methods described above may be applied in common, and in the use of the template, only the entire designated template area or a part of the template area may be used for deriving the coefficients. In addition, according to one embodiment of the present invention, the coefficient derivation method of the Wiener filter which utilizes the restored pixel values on the top and/or the left side of the current block may be utilized in the same manner or in an applied manner for deriving the coefficients. The Wiener filter may have a characteristic of deriving coefficients in a manner that minimizes the minimum mean squared error (MSE), and the coefficients may be obtained based on this characteristic.

It is obvious that the present invention is not limited to the above-described embodiments, and various modifications and applications that can be applied in relation to LIC and predictive encoding within the scope of the present invention can be applied together. For example, the size of the filter can be adjusted, and a type of filter other than a cross filter can be used. For example, a 5x5 cross filter or a 3x3 Sobel filter may be used. In addition, it is obvious that any filter configured to receive as input a pixel group having at least one shape of a square, a diamond, and a circle with a width of the first pixel and a height of the second pixel can be applied. In addition, extensions such as dynamically adjusting the shape or size of the template, or introducing a polygonal function or a nonlinear function into the linear model to model more complex illumination changes can be made. Alternatively, various methods for determining the predictive mode in an explicit or implicit manner can be utilized to implement the examples described above for the present invention, or the present invention can be implemented to operate in conjunction with such other methods.

Encoder and decoder

It is obvious that the encoding method according to the present invention can be applied equally to an encoder and a decoder. The encoding method according to the present invention can be first used as one of the methods for generating a sample value of a prediction block for performing intra-screen and/or inter-screen prediction in an encoder. When residual signal information is encoded by a differential value with respect to such a prediction block, the decoder can generate a sample value of the same prediction block using a corresponding and/or symmetrical method, and obtain decoded samples by combining the differential value with such a prediction block. Additionally, as exemplified through the internal decoder (420) and the coding loop including the same in FIG. 4, such a decoding process can be implemented identically within the encoder to predict the state of the decoder.

The encoding method according to the present invention described above can be implemented through an encoder as a device. The encoder as a device can be implemented in a form that maintains the conventional encoder structure exemplified through FIGS. 1 to 6 above or applies a predetermined change thereto. However, the form of implementation is not necessarily limited to what has been exemplified, and any form of encoder structure that can function as a video encoder should be considered an encoder established by the present invention as long as it implements the technical idea of the present invention.

In addition, the method for decoding the encoding result according to the present invention described above can be implemented through a decoder as a device. The decoder as a device can be implemented in a form that maintains the conventional decoder structure exemplified through FIGS. 1 to 6 above or applies a predetermined change thereto. However, the form of implementation is not necessarily limited to what has been exemplified, and any form of decoder structure that can function as a video decoder is considered to be a decoder established by the present invention as long as it implements the technical idea of the present invention.

A person skilled in the art will readily understand that a bit stream encoded by the above-described method and device can be decoded by applying a method symmetrical and/or inverse to the encoding method. In one embodiment, when reading information for decoding from the encoded bit stream, at least one variable length coded phrase included in the encoded bit stream can be interpreted, and furthermore, in one embodiment, the variable length coding can be performed by an entropy coding method. The technical details and application method of implementing such a decoding procedure can readily be understood from the above-described encoding procedure.

The encoder and/or decoder described in the present specification may correspond to a device including a processor and a memory, each of which may be implemented as a computing device. The processor, which may be included in the encoder and/or decoder described in the present specification, may mean one or more general-purpose computers or special-purpose computers, such as a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable array (FPA), a programmable logic unit (PLU), a microprocessor, or any other device capable of executing instructions and responding.

Even if the processor is expressed in a singular form for the convenience of understanding, a person skilled in the art will recognize that the processor may include a plurality of processing elements and/or a plurality of types of processing elements. For example, a device according to an embodiment of the present invention may include a plurality of processors or one processor and one controller as the processor. In addition, the processor may be implemented by various processing configurations, such as a parallel processor or a multi-core processor.

The processor may be configured to execute an operating system (OS) and one or more software programs running on the operating system. Additionally, the processor may access, store, manipulate, process, and generate data in response to execution of the software programs.

The software may include a computer program, code, instructions, or a combination of one or more of these, and may be configured to control the processor to perform a desired operation, and may be configured to issue instructions to the processor, either independently or collectively. The software may be permanently or temporarily embodied in any type of machine, component, physical device, virtual equipment, computer storage medium or device, or transmitted signal wave, for interpretation by the processor or for providing instructions or data to the processor. The software may also be distributed over network-connected computer systems, and stored or executed in a distributed manner.

The software may also be implemented in the form of program commands that can be executed through various computer means and recorded or stored in the memory. The memory may be a computer-readable recording medium, and program commands, data files, data structures, etc. may be recorded singly or in combination in the computer-readable recording medium. The program commands stored in the memory may be based on a command system specifically designed and configured for the embodiment of the present invention, or may follow a command system known and available to those skilled in the art of computer software, for example, a command system exemplified by the assembly language, C, C++, Java, Python, etc. It should be understood that the command system and the program commands thereby include not only machine language codes generated by a compiler, but also high-level language codes that can be executed by the device and/or the processor according to an embodiment of the present invention using an interpreter, etc.

The computer-readable recording medium constituting the device according to one embodiment of the present invention, including the memory described in this specification, may include a temporary or volatile recording medium that is maintained only while the processor is operating, such as a processor cache, a RAM, a flash memory, or a relatively non-volatile or long-term recording medium, such as a magnetic media such as a hard disk, a floppy disk, and a magnetic tape, an optical media such as a CD-ROM, a DVD, a magneto-optical media such as a floptical disk, or a solid state memory, or may include a read-only recording medium, such as a ROM arranged on hardware, and further, hardware itself configured to perform operations equivalent to a series of program commands by a hard-wired structure by circuit wiring, and since each step for performing the operations implementing the embodiments of the present invention can be viewed as being recorded by the connection and arrangement of the hardware components, the connection and arrangement method is the memory and It is obvious to a person skilled in the art that they can be considered equivalent.

The embodiments described above with respect to the processor and the memory are not mutually exclusive and may be selected or combined and implemented as needed. For example, a single hardware device may be configured to operate as a module composed of one or more of the software to perform the operations of the embodiments of the present invention, and vice versa. As another example, in the present specification, all or part of the operations assigned to a certain functional unit may be implemented by one or more of the software stored in the device according to an embodiment of the present invention (preferably, in any one of the recording media belonging to the category of the memory) and configured to be executed by the processor, and in this case, such a functional unit may be referred to as a functional unit "included" in the processor.

Although the present invention has been described with reference to the drawings and embodiments, as already mentioned above, it does not mean that the protection scope of the present invention is limited by the drawings or embodiments presented above, and it will be understood that a person skilled in the relevant technical field can modify and change the present invention in various ways without departing from the spirit and scope of the present invention described in the claims of the patent for the present invention.

Claims (20)

A method for decoding an encoded video bitstream, A step of obtaining at least one prediction vector for the current block currently being decoded from the above bit string, A step of determining at least one reference block based on the above prediction vector; A step of defining at least one of a first template including at least one pixel adjacent to the current block as a reference block, and a second template including at least one pixel adjacent to the reference block as a reference block; A step of determining a template configuration mode including information on how to use the template, for at least one of the first template and the second template; A step of extracting information related to local illumination compensation (LIC) by referring to at least one pixel information derived from at least one of the first template and the second template based on the at least one template configuration mode, and obtaining LIC information from the extracted information; A step of generating a prediction sample for the current block by correcting the reference block by a LIC method operating based on the LIC information; and A decryption method, comprising: a step of restoring the current block using the above prediction sample. In paragraph 1, The above template configuration mode is, A decoding method, characterized in that it includes at least one mode among a mode using the upper pixel of the reference block, a mode using the left pixel of the reference block, and a mode using the upper and left pixels of the reference block based on weights. In the second paragraph, The mode used based on the above weights is: A decoding method characterized in that the upper and left areas of the above-mentioned reference block are divided into at least one zone and different zone-specific weights are applied to each zone. In the third paragraph, The above weights for each zone are: Selected from a predefined table, The step of determining the above template configuration mode is: A decryption method, comprising: a step of obtaining information representing an index selected from the above table from the bit string. In paragraph 4, The information representing the above index is: A decryption method characterized in that it is implicitly determined on the decoder side based on previously decrypted video information. In paragraph 1, The above template configuration mode is, A decoding method characterized by including a mode for selecting and using only some pixels included in one of the first template and the second template through subsampling. In paragraph 6, The above subsampling is, A decryption method characterized by dividing the above template into a first number of regions in at least one of horizontal and vertical directions, and selecting a second number of pixels from the divided regions. In paragraph 7, The above subsampling is, If the above template includes the upper pixel area of the above reference block, the upper pixel area is divided in the horizontal direction, A decoding method characterized in that, when the template includes a left pixel area of the reference block, the left pixel area is divided in the vertical direction. In paragraph 7, A decryption method, characterized in that at least one of the first number and the second number is determined based on a power of 2. In paragraph 1, The step of determining the above template configuration mode is: A step of obtaining a mode signal from said bit string, wherein said mode signal instructs to use at least one template configuration mode; and A decoding method, comprising: a step of determining the template configuration mode based on the mode signal; In paragraph 1, The step of determining the above template configuration mode is: A decryption method, comprising: a step of implicitly determining the template configuration mode on the decoder side based on previously decrypted video information; A method for encoding a bit stream generated from a video, A step of determining at least one prediction vector for the current block currently being encoded, A step of determining at least one reference block based on the above prediction vector; A step of defining at least one of a first template including at least one pixel adjacent to the current block as a reference block, and a second template including at least one pixel adjacent to the reference block as a reference block; A step of determining a template configuration mode including information on how to use the template, for at least one of the first template and the second template; A step of extracting information related to local illumination compensation (LIC) by referring to at least one pixel information derived from at least one of the first template and the second template based on the at least one template configuration mode, and obtaining LIC information from the extracted information; A step of generating a prediction sample for the current block by correcting the reference block by a LIC method operating based on the LIC information; a step of encoding the current block using the above prediction sample; and An encoding method, comprising: a step of recording the encoding result in the bit string. In Article 12, The above template configuration mode is, An encoding method characterized by including at least one mode among a mode using the upper pixel of the reference block, a mode using the left pixel of the reference block, and a mode using the upper and left pixels of the reference block based on weights. In Article 13, The mode used based on the above weights is: An encoding method characterized in that the upper and left areas of the above-mentioned reference block are divided into at least one zone and different zone-specific weights are applied to each zone. In Article 12, The above template configuration mode is, An encoding method, characterized in that it includes a mode for selecting and using only some pixels from at least one of the above templates. In Article 15, Some of the above pixels are, A decoding method characterized by being selected through subsampling. In Article 15, The step of determining the above prediction vector is: A step of referencing the first template according to a prediction template configuration mode; comprising; The steps to obtain the above LIC information are: A step of referencing the first template according to the first LIC template configuration mode; and a step of referencing the second template according to the second LIC template configuration mode; and a step of determining the LIC linear model through an operation based on the result of the reference; An encoding method, characterized in that the above LIC linear model is a function defined by coefficients including a slope (α) and a bias (β). In Article 17, The above prediction template configuration mode includes a mode for selecting and using only some pixels from the first template, An encoding method, characterized in that the step of determining the prediction vector is configured to determine the prediction vector through a template-based search based on some of the selected pixels. In Article 17, At least one of the first LIC template configuration mode and the second LIC template configuration mode includes a mode for selecting and using only some pixels from at least one of the templates, An encoding method, characterized in that the step of obtaining the LIC information is configured to determine at least one of the coefficients of the LIC linear model through an operation based on the selected portion of pixels. A decoder device configured to decode a video bit stream encoded by a computing device, A receiver that receives a bit string; An output section that outputs the decrypted video; A reference buffer storing information about at least one decoded picture; processor; and A memory for storing instructions executable by the processor; A decoder device configured to include instructions for: obtaining at least one prediction vector for a current block currently being decoded from the bit string; determining at least one reference block based on the prediction vector; defining at least one of a first template including at least one pixel adjacent to the current block as a reference block, and a second template including at least one pixel adjacent to the reference block as a reference block; determining a template configuration mode including information on a method of using the template, for at least one of the first template and the second template; extracting information related to Local Illumination Compensation (LIC) with reference to at least one pixel information derived from at least one of the first template and the second template based on the at least one template configuration mode, and obtaining LIC information from the extracted information; generating a prediction sample for the current block by correcting the reference block by a LIC method operating based on the LIC information; and restoring the current block using the prediction sample.

Images