CN119815012B - Method and device for processing image block in immersive media coding - Google Patents

Method and device for processing image block in immersive media coding

Info

Publication number
CN119815012B
CN119815012B CN202311299927.5A CN202311299927A CN119815012B CN 119815012 B CN119815012 B CN 119815012B CN 202311299927 A CN202311299927 A CN 202311299927A CN 119815012 B CN119815012 B CN 119815012B
Authority
CN
China
Prior art keywords
image
block
blocks
effective information
sub
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202311299927.5A
Other languages
Chinese (zh)
Other versions
CN119815012A (en
Inventor
白雨箫
虞露
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Zhejiang University ZJU
Original Assignee
Zhejiang University ZJU
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Zhejiang University ZJU filed Critical Zhejiang University ZJU
Priority to CN202311299927.5A priority Critical patent/CN119815012B/en
Publication of CN119815012A publication Critical patent/CN119815012A/en
Application granted granted Critical
Publication of CN119815012B publication Critical patent/CN119815012B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Landscapes

  • Image Processing (AREA)

Abstract

本发明提供了一种沉浸式媒体编码中处理图像块的方法,包括:对于多视点图像中的至少一个图像块,对于所述图像块中至少一条贯穿且垂直于所述图像块边界的分割线,分别计算所述分割线两侧图像子块的有效信息密度,所述有效信息为所述图像子块在多视点中相对于其他视点的额外信息;当所述分割线两侧图像子块满足切割条件时,所述切割条件为一侧图像子块的有效信息密度大于阈值且另一侧图像子块的有效信息密度小于阈值:沿所述分割线进行切割,将所述图像块切割为两个图像块,对于包含有效信息密度小于阈值的图像子块的图像块,若不存在满足所述切割条件的分割线,则将所述图像块丢弃,不组装进入拼接图中;否则:不进行切割。

This invention provides a method for processing image blocks in immersive media encoding, comprising: for at least one image block in a multi-view image, for at least one segmentation line that runs through and is perpendicular to the boundary of the image block, calculating the effective information density of image sub-blocks on both sides of the segmentation line, wherein the effective information is the additional information of the image sub-block relative to other viewpoints in the multi-view image; when the image sub-blocks on both sides of the segmentation line meet a cutting condition, wherein the cutting condition is that the effective information density of one image sub-block is greater than a threshold and the effective information density of the other image sub-block is less than a threshold: cutting along the segmentation line to divide the image block into two image blocks; for an image block containing image sub-blocks with an effective information density less than a threshold, if there is no segmentation line that meets the cutting condition, then the image block is discarded and not assembled into the stitching image; otherwise: no cutting is performed.

Description

Method and device for processing image block in immersive media coding
Technical Field
The invention relates to the field of immersive media coding, in particular to a method and a device for processing multi-view image blocks.
Background
"Immersion" is a subjective assessment that refers to the substitution of viewers' perception of a virtual scene created and displayed by a multimedia system. As the capabilities of acquisition devices and display devices have increased year by year, immersive media has become a research hotspot in industry and scientific community as a visual multimedia capable of bringing a strong immersion to viewers. The multi-view image plus depth information is an effective immersive media representation consisting of texture images of multiple views and depth images corresponding to each texture image. By utilizing a viewpoint synthesis technology based on the depth image, the expression mode can render and obtain an image of the target viewpoint according to camera parameters of the target image and the position relation between the target viewpoint and the existing viewpoint.
Because larger information redundancy exists among multiple views, coding and transmitting multi-view sub-block spliced images can remarkably save pixel rate, before coding and decoding, the redundant pixels of other views are eliminated as much as possible by utilizing a main view (some images containing complete view information in multi-view images) image by analyzing the geometric texture relation among the multiple views, so that other view images except the main view only retain specific effective information. In consideration of coding efficiency, reserved effective pixels are converged into rectangular sub-blocks, and finally a plurality of image blocks are assembled into a spliced image for coding transmission. And the decoding end extracts all block images from the multi-view block spliced image obtained by decoding by utilizing the image block information obtained by decoding, and finally synthesizes the target image by taking the image blocks as units.
However, as the field of view of video content becomes larger, the increase of the number of source viewpoints exposes the disadvantage of processing capability of the encoding transmission scheme based on the spliced image when inputting a large amount of data, and in many industrial scenes, the pixel rate of the spliced image is limited, and the necessary sub-block images remained after the redundancy is removed from view points at present cannot be put into a map set with a limited size, and the information is lost when decoding due to the loss of the sub-block images, so that high-quality video cannot be synthesized. Thus, in order to make decisions to measure the importance of an image block, the concept of the effective information amount of a pixel is introduced, which measures the non-overlapping degree of the current pixel with pixels in other viewpoints. The higher the effective information amount, the lower the degree of characterization of pixels in other viewpoints, the larger the unique information should be retained in the stitched image.
More specifically, the effective information generally includes two channels, namely geometric information and texture information, wherein the value of each channel is independent from each other, and is calculated by the logarithm of the difference value of the parallax and the brightness of the synthesized pixel of the current pixel and the reference pixel from other viewpoints at the current pixel position. If the current pixel has no reference pixel, the information amount is the maximum value, and is determined by the parallax and the bit width of the brightness. The specific calculation mode is as follows, the parallax difference between the current pixel and the reference pixel is d g, the brightness difference is d t, and the effective information ρ is:
ρ=logdg+logdt
The geometric bit width b g and the luminance bit width b t of the video, the maximum value ρ max of the effective information is:
Since the retained pixels are aggregated into image sub-blocks, the effective information of an image sub-block is calculated by accumulating the effective information values of all effective pixels therein.
ρblock=∑pixel∈blockρpixel
Further put forward is the concept of effective information density μ, dividing sub-block effective information by the number of pixels effective
In the process of sub-block image splicing, the effective information density mu of the sub-block is used as a basis, the image sub-block with large effective information density is preferentially reserved under the limit of the priority pixel rate, the information utilization rate is improved, and the rendering quality of a decoding end is improved.
However, the distribution of the effective information may be uneven within one image sub-block, and selecting the image sub-block to be preferentially retained according to the effective information density μmay discard local areas of high effective information density in some sub-blocks together, or there is a case where local areas of low effective information density are retained together in the retained image sub-block, wasting the pixel rate. Therefore, an image sub-block cutting method based on effective information distribution is needed, so that the effective information distribution in the same sub-block after cutting is as uniform as possible, and the information rate can be improved when the image is spliced by tissues, and the coding efficiency is further improved. However, how to make the cutting more sufficient and how to reduce the ineffective cutting in the cutting process, and how to not make the cutting when the effective information density of the two sub-blocks is greater than the retention threshold, and preventing the continuity of the image area from being damaged so as to affect the coding efficiency are both difficult and challenging.
Disclosure of Invention
In order to solve the technical problems, the invention provides a method and a device for processing image blocks in immersive media coding.
For at least one image block in the multi-view image, for at least one dividing line which penetrates through and is perpendicular to the boundary of the image block in the image block, calculating the effective information density of the image sub-blocks at two sides of the dividing line respectively, wherein the effective information is the additional information of the image sub-blocks relative to other view points in the multi-view.
When the image sub-blocks at two sides of the dividing line meet the cutting condition, the cutting condition is that the effective information density of the image sub-block at one side is larger than a threshold value and the effective information density of the image sub-block at the other side is smaller than the threshold value, cutting is carried out along the dividing line, the image block is cut into two image blocks, if the image block containing the image sub-block with the effective information density smaller than the threshold value does not have the dividing line meeting the cutting condition, the image block is discarded and is not assembled into the splicing diagram, otherwise, the image block is not cut.
Further, for the image block which contains the image sub-block with the effective information density smaller than the threshold value after cutting, if a dividing line meeting the cutting condition exists, cutting the image block again.
Further, for the image block with a plurality of dividing lines for which the image sub-blocks at two sides meet the cutting condition, a dividing line with the largest difference of effective information density of the image sub-blocks at two sides of the dividing line is selected as an actual dividing line.
Further, when the image sub-blocks at two sides of the dividing line meet the cutting condition, updating a threshold according to the pixel rate limit of the spliced image and the effective information density of the image blocks which are not discarded.
Further, the invention provides a device for processing image blocks, which comprises the following modules:
The effective information density calculating module is used for respectively calculating the effective information densities of the image sub-blocks at two sides of the dividing line along the dividing line which penetrates through and is perpendicular to the boundary of the image block in the multi-view image block, wherein the effective information is the extra information of the image sub-blocks relative to other views in the multi-view
The threshold judging module is used for judging whether one of the effective information densities of the image sub-blocks at the two sides of the dividing line is larger than a threshold value, one is smaller than the threshold value, cutting is performed when the condition is met, and cutting is not performed when the condition is not met
The dividing line decision module is used for selecting the dividing line with the largest difference of the effective information density of the image sub-blocks at the two sides of the dividing line as the actual dividing line for the image block when the image sub-blocks at the two sides meet the dividing line of the cutting condition
Cutting along the dividing line under the condition of meeting the cutting condition
The assembly and splicing decision module is used for discarding image blocks which contain image sub-blocks with effective information density smaller than a threshold value and do not have a dividing line meeting the cutting condition, and does not assemble the image blocks into a splicing diagram
And the threshold updating module is used for updating the threshold according to the pixel rate limit of the spliced image and the effective information density of the image blocks which are not discarded when the image sub-blocks at the two sides of the dividing line meet the cutting condition.
Further, the image block cutting device cuts the image block containing the image sub-block with the effective information density smaller than the threshold value again if the dividing line meeting the cutting condition exists.
The method has the advantages that full cutting of the image blocks can be achieved through judgment of the granularity of the image sub-blocks, the image blocks can be discarded more accurately, the image blocks with higher effective information density can be organized under the limit of the limited pixel rate in the multi-view image splicing process, the information utilization rate of the spliced image is improved, the rendering quality is improved at the decoding end, meanwhile, invalid cutting is reduced under the condition that discarding is not needed through threshold value control cutting, the coding code rate is saved, the threshold value is updated according to the limit of the pixel rate and the effective information density of the image sub-blocks which are not discarded after each cutting, the threshold value is more accurate, and the overall coding efficiency is improved.
Drawings
The principles of the present invention may be explained by the following examples with reference to the drawings.
The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this specification, illustrate only the application and, together with the description, serve to explain the principles of the application.
FIG. 1 is a schematic diagram of three image sub-blocks A, B, C of image block ABC in one embodiment of the invention.
FIG. 2 is a schematic diagram of three image sub-blocks A, B, C of image block ABC in another embodiment of the invention.
FIG. 3 is a schematic diagram of three image sub-blocks A, B, C of image block ABC in another embodiment of the invention.
Fig. 4 is a schematic diagram of a cutting result according to an embodiment of the present invention, wherein the discarded image blocks are within a dashed box.
Fig. 5 is a schematic diagram of a cutting result according to another embodiment of the present invention, wherein the discarded image blocks are within the dashed box.
Fig. 6 is a schematic diagram of a cutting result according to another embodiment of the present invention, wherein the discarded image blocks are within the dashed box.
Fig. 7 is a schematic diagram of a cutting result according to another embodiment of the present invention, wherein the discarded image blocks are within the dashed box.
Fig. 8 is a schematic diagram of a cutting result according to another embodiment of the present invention, wherein the discarded image blocks are within the dashed box.
Fig. 9 is a schematic diagram of a cutting result according to another embodiment of the present invention, wherein the discarded image blocks are within the dashed box.
Detailed Description
Example 1
As shown in fig. 1, A, B, C is three image sub-blocks of image block ABC, the colors of the image sub-blocks representing the effective density of the corresponding image sub-blocks from light to deep as low as high.
In this example, the initial threshold lies between the effective information densities of image sub-block a and image sub-block B.
The common boundary of the image sub-block A and the image sub-block B is a dividing line which penetrates through and is perpendicular to the boundary of the image block, the effective information density of the image sub-block A is smaller than a threshold value, and the effective information density of the image sub-block B is larger than the threshold value, so that the cutting condition is met;
Therefore, the image block ABC is cut into the image block A and the image block BC by taking the common boundary of the image sub-block A and the image sub-block B as a dividing line, and the effective information density of the image block A is smaller than a threshold value, and no dividing line meeting the cutting condition exists, so that the image block ABC is discarded and not assembled into the spliced graph.
Removing image block ABC, adding image block BC, and recalculating a threshold value, wherein the updated threshold value is the minimum value of the effective information density of the image blocks and the image sub-blocks which are not discarded in the pixel rate limit of the mosaic image.
In this example, the updated threshold is less than the effective information density of image sub-block B.
For the image block BC generated after cutting, repeating the process, wherein the common boundary of the image sub-block B and the image sub-block C is a dividing line which penetrates through and is perpendicular to the boundary of the image block, but the effective information density of the image sub-block B and the effective information density of the image sub-block C are both larger than a threshold value, and the cutting condition is not met;
the cut is terminated, as shown in fig. 4, with discarded tiles within the dashed box, and the final tile ABC is cut into tile a and tile BC, where tile a is discarded and not entered into the mosaic.
Example 2
As shown in fig. 1, A, B, C is three image sub-blocks of image block ABC, the colors of the image sub-blocks representing the effective density of the corresponding image sub-blocks from light to deep as low as high.
In this example, the initial threshold lies between the effective information densities of image sub-block B and image sub-block C.
The common boundary of the image sub-block A and the image sub-block B is a dividing line which penetrates through and is perpendicular to the boundary of the image block, but the effective information density of the image sub-block A and the effective information density of the image sub-block B are smaller than a threshold value and do not meet the cutting condition;
The image block ABC is cut into an image block AB and an image block C by taking the common boundary of the image sub-block B and the image sub-block C as a dividing line, and the image block AB comprises an image sub-block A with the effective information density smaller than a threshold value, but the effective information density of the image sub-block A, B is smaller than the threshold value, and the dividing line meeting the cutting condition does not exist, so that the image block AB is discarded and not assembled into the spliced graph.
The cut is terminated, as shown in fig. 5, with discarded tiles within the dashed box, and the final tile ABC is cut into tiles AB and C, where tile AB is discarded and not entered into the mosaic.
Removing image block ABC, adding image block C, and recalculating a threshold value, wherein the updated threshold value is the minimum value of effective information density of the image blocks and the image sub-blocks which are not discarded in the pixel rate limit of the mosaic image.
Example 3
As shown in fig. 1, A, B, C is three image sub-blocks of image block ABC, the colors of the image sub-blocks representing the effective density of the corresponding image sub-blocks from light to deep as low as high.
In this example, the initial threshold is less than the effective information density of image sub-block A
The common boundary of the image sub-block A and the image sub-block B is a dividing line which penetrates through and is perpendicular to the boundary of the image block, but the effective information density of the image sub-block A and the effective information density of the image sub-block B are both larger than a threshold value and do not meet the cutting condition;
and therefore no cutting is performed. The cut is terminated and the threshold is not updated.
By the method, ineffective segmentation of the image block ABC can be reduced, and the coding rate is saved;
Example 4
As shown in fig. 1, A, B, C is three image sub-blocks of image block ABC, the colors of the image sub-blocks representing the effective density of the corresponding image sub-blocks from light to deep as low as high.
In this example, the initial threshold lies between the effective information densities of image sub-block a and image sub-block B.
The common boundary of the image sub-block A and the image sub-block B is a dividing line which penetrates through and is perpendicular to the boundary of the image block, the effective information density of the image sub-block A is smaller than a threshold value, the effective information density of the image sub-block B is larger than the threshold value, the cutting condition is met, the common boundary of the image sub-block B and the image sub-block C is a dividing line which penetrates through and is perpendicular to the boundary of the image block, but the effective information density of the image sub-block B and the effective information density of the image sub-block C are both larger than the threshold value, and the cutting condition is not met.
Therefore, the image block ABC is cut into the image block A and the image block BC by taking the common boundary of the image sub-block A and the image sub-block B as a dividing line, and the effective information density of the image block A is smaller than a threshold value, and no dividing line meeting the cutting condition exists, so that the image block ABC is discarded and not assembled into the spliced graph.
Removing image block ABC, adding image block BC, and recalculating a threshold value, wherein the updated threshold value is the minimum value of the effective information density of the image blocks and the image sub-blocks which are not discarded in the pixel rate limit of the mosaic image.
In this example, the updated threshold lies between the effective information densities of image sub-block B and image sub-block C.
Repeating the above process for the image block BC generated after cutting, wherein the common boundary of the image sub-block B and the image sub-block C is a dividing line which penetrates through and is perpendicular to the boundary of the image block, the effective information density of the image sub-block B is smaller than a threshold value, the effective information density of the image sub-block C is larger than the threshold value, and the cutting condition is met;
Therefore, the image block BC is cut into the image block B and the image block C by taking the common boundary of the image sub-block B and the image sub-block C as a dividing line, and the effective information density of the image block B is smaller than a threshold value and no dividing line meeting the cutting condition exists, so that the image block BC is discarded and not assembled into the spliced graph.
The cut is terminated, as shown in fig. 6, with discarded image blocks within the dashed box, and the final image block ABC is cut into image block a, image block B, and image block C, where image sub-a, image block B are discarded and not entered into the mosaic.
And removing the image block BC, adding the image block C, and recalculating a threshold value, wherein the updated threshold value is the minimum value of the effective information density of the image blocks and the image sub-blocks which are not discarded in the pixel rate limit of the mosaic image.
According to the method, after each cutting, the threshold value is updated according to the limitation of the pixel rate and the effective information density of the image sub-blocks which are not discarded, so that the threshold value is more accurate, the cutting is more sufficient, and the overall coding efficiency is increased.
Example 5
As shown in fig. 2, A, C, B is three image sub-blocks of the image block ACB, and the colors of the image sub-blocks from light to deep represent that the effective density of the corresponding image sub-blocks is lowest to high.
In this example, the initial threshold lies between the effective information densities of image sub-block B and image sub-block C.
The common boundary of the image sub-block A and the image sub-block C is a dividing line which penetrates through and is perpendicular to the boundary of the image block, the effective information density of the image sub-block A is smaller than a threshold value, and the effective information density of the image sub-block C is larger than the threshold value, so that a cutting condition is met;
The effective information density difference of the image sub-block A and the image sub-block C is larger, so that the common boundary of the image sub-block A and the image sub-block C is taken as an actual dividing line, the image block ACB is cut into the image block A and the image block CB, and the dividing line meeting the cutting condition is not existed because the effective information density of the image block A is smaller than a threshold value, and the image block A is discarded and not assembled into the spliced graph.
And removing the image block ACB, adding the image block CB, and recalculating a threshold value, wherein the updated threshold value is the minimum value of the effective information density of the image blocks and the image sub-blocks which are not discarded in the pixel rate limit of the mosaic image.
In this example, the updated threshold lies between the effective information densities of image sub-block C and image sub-block B.
Repeating the above process for the image block CB generated after cutting, wherein the common boundary of the image sub-block C and the image sub-block B is a dividing line which penetrates through and is perpendicular to the boundary of the image block, the effective information density of the image sub-block B is smaller than a threshold value, the effective information density of the image sub-block C is larger than the threshold value, and the cutting condition is met;
Therefore, the image block CB is cut into the image block C and the image block B by taking the common boundary of the image sub-block C and the image sub-block B as a dividing line, and the effective information density of the image block B is smaller than a threshold value and no dividing line meeting the cutting condition exists, so that the image block CB is discarded and not assembled into the spliced graph.
The cutting is terminated, as shown in fig. 7, the discarded image blocks are within the dashed frame, and the final image block ACB is cut into an image block a, an image block B, and an image block C, where the image block a, the image block B are discarded and not entered into the mosaic.
And removing the image block CB, adding the image sub-block C, and recalculating a threshold value, wherein the updated threshold value is the minimum value of the effective information density of the image blocks which are not discarded and the image sub-blocks within the pixel rate limit of the spliced image.
Example 6
As shown in fig. 2, A, C, B is three image sub-blocks of the image block ACB, and the colors of the image sub-blocks from light to deep represent that the effective density of the corresponding image sub-blocks is lowest to high.
In this example, the initial threshold lies between the effective information densities of image sub-block a and image sub-block B.
The common boundary of the image sub-block A and the image sub-block C is a dividing line which penetrates through and is perpendicular to the boundary of the image block, the effective information density of the image sub-block A is smaller than a threshold value, the effective information density of the image sub-block C is larger than the threshold value, the cutting condition is met, the common boundary of the image sub-block C and the image sub-block B is a dividing line which penetrates through and is perpendicular to the boundary of the image block, but the effective information density of the image sub-block C and the effective information density of the image sub-block B are both larger than the threshold value, and the cutting condition is not met.
Therefore, the image block ACB is cut into the image block A and the image block CB by taking the common boundary of the image sub-block A and the image sub-block C as a dividing line, and the effective information density of the image block A is smaller than a threshold value and no dividing line meeting the cutting condition exists, so that the image block A is discarded and not assembled into the spliced graph.
And removing the image block ACB, adding the image block CB, and recalculating a threshold value, wherein the updated threshold value is the minimum value of the effective information density of the image blocks and the image sub-blocks which are not discarded in the pixel rate limit of the mosaic image.
In this example, the updated threshold is located below the threshold of image sub-block B.
For an image block CB generated after cutting, repeating the process, wherein the common boundary of the image sub-block C and the image sub-block B is a dividing line which penetrates through and is perpendicular to the boundary of the image block, but the effective information density of the image sub-block C and the effective information density of the image sub-block B are both larger than a threshold value, and the cutting condition is not met;
The cut is terminated, as shown in fig. 8, with discarded image blocks within the dashed box, and the final image block ACB is cut into image block a and image block CB, where image block a is discarded and does not enter the mosaic.
Example 7
The embodiment of the invention provides a device for processing image blocks in immersive media coding
As shown in fig. 3, B, A, C is three image sub-blocks of the image block BAC, the colors of the image sub-blocks represent the effective density of the corresponding image sub-blocks from light to deep as low as high.
In this example, the initial threshold lies between the effective information densities of image sub-block a and image sub-block B.
And the effective information density calculating module is used for inputting the image sub-block A, the image sub-block B and the image sub-block C and outputting the effective information densities of the image sub-block A, the image sub-block B and the image sub-block C.
The threshold judging module is used for inputting effective information densities of an image sub-block A, an image sub-block B and an image sub-block C respectively, wherein the common boundary of the image sub-block B and the image sub-block A is a dividing line which penetrates through and is perpendicular to the boundary of the image block, the effective information density of the image sub-block B is larger than a threshold value, the effective information density of the image sub-block A is smaller than the threshold value, and a cutting condition is met;
And the dividing line decision module is used for inputting the effective information density difference value of the common boundary of the image sub-block B and the image sub-block A, the common boundary of the image sub-block A and the image sub-block C and the image sub-blocks at the two sides of the common boundary and the effective information density difference value, and outputting the effective information density difference value as an actual dividing line. Since the effective information density difference between the image sub-block a and the image sub-block C is larger, the common boundary between the image sub-block a and the image sub-block C is taken as the actual dividing line.
And the cutting module is used for inputting a common boundary of the image sub-block A and the image sub-block C of the parting line, cutting the decision and outputting the cut image block. The image block BAC is cut into an image block BA and an image block C;
The assembly and splicing decision module is used for inputting the image sub-block A with the effective information density smaller than the threshold value into the image block BA, and the image block BA has a dividing line meeting the cutting condition, so that the image block BA is not discarded.
And the threshold updating module inputs the effective information density and pixel rate limit of the image blocks BA and C which are not discarded and outputs the effective information density and pixel rate limit as an updated threshold. Removing image block BAC, adding image block BA and image block C, recalculating a threshold value, and updating the threshold value to be the minimum value of the effective information density of the image blocks which are not discarded in the pixel rate limit of the mosaic.
In this example, the updated threshold lies between the effective information densities of image sub-block a and image sub-block B.
Repeating the device flow for the image block BA generated after cutting, wherein the common boundary of the image sub-block B and the image sub-block A is a dividing line which penetrates through and is perpendicular to the boundary of the image block, the effective information density of the image sub-block B is larger than a threshold value, the effective information density of the image sub-block A is smaller than the threshold value, and the cutting condition is met;
And the image block BA is cut into the image block B and the image block A by taking the common boundary of the image sub-block B and the image sub-block A as a dividing line, and the effective information density of the image block A is smaller than a threshold value and no dividing line meeting the cutting condition exists, so that the image block BA is discarded and not assembled into the spliced graph.
The cut is terminated, as shown in fig. 9, with discarded image blocks within the dashed box, and the final image block BAC is cut into image block B, image block a, and image block C, where image block a is discarded and does not enter the mosaic.
And adding the image block B, and recalculating a threshold value, wherein the updated threshold value is the minimum value of the effective information density of the image blocks and the image sub-blocks which are not discarded in the pixel rate limit of the spliced image.
By the device, the segmentation of the granularity of the image sub-blocks can be realized, the image block A with low effective information density is accurately discarded, the image block with higher effective information density can be organized under the limit of the limited pixel rate in the multi-view image splicing process, the information utilization rate of a spliced image is improved, and the rendering quality is improved at a decoding end;
it should be noted that the above-mentioned embodiments are merely for illustrating the technical solution of the present invention, and not for limiting the same, and although the present invention has been described in detail with reference to the above-mentioned embodiments, it should be understood by those skilled in the art that the technical solution described in the above-mentioned embodiments may be modified or some technical features may be equivalently replaced, and these modifications or substitutions do not deviate the essence of the corresponding technical solution from the scope of the technical solution of the embodiments of the present invention.

Claims (8)

1.一种沉浸式媒体编码中处理图像块的方法,其特征在于,包括:1. A method for processing image blocks in immersive media coding, characterized in that it includes: 对于多视点图像中的至少一个图像块,对于所述图像块中至少一条贯穿且垂直于所述图像块边界的分割线,分别计算所述分割线两侧图像子块的有效信息密度,所述有效信息为所述图像子块在多视点中相对于其他视点的额外信息;For at least one image patch in a multi-view image, for at least one segmentation line in the image patch that runs through and is perpendicular to the boundary of the image patch, the effective information density of the image sub-patterns on both sides of the segmentation line is calculated respectively. The effective information is the additional information of the image sub-patterns relative to other viewpoints in the multi-view image. 当所述分割线两侧图像子块满足切割条件时,所述切割条件为一侧图像子块的有效信息密度大于阈值且另一侧图像子块的有效信息密度小于阈值:When the image sub-blocks on both sides of the segmentation line meet the cutting conditions, the cutting conditions are that the effective information density of one side of the image sub-block is greater than a threshold and the effective information density of the other side of the image sub-block is less than a threshold: 沿所述分割线进行切割,将所述图像块切割为两个图像块,The image block is divided into two image blocks by cutting along the dividing line. 对于包含有效信息密度小于阈值的图像子块的图像块,若不存在满足所述切割条件的分割线,则将所述图像块丢弃,不组装进入拼接图中;For an image block containing an image sub-block with an effective information density less than a threshold, if there is no dividing line that satisfies the cutting conditions, the image block is discarded and not assembled into the stitched image. 否则:otherwise: 不进行切割。No cutting is performed. 2.根据权利要求1所述的方法,其特征在于,对于包含有效信息密度小于阈值的图像子块的图像块,若存在满足所述切割条件的分割线,则对其进行再次切割。2. The method according to claim 1, wherein for an image block containing an image sub-block with an effective information density less than a threshold, if there is a dividing line that satisfies the cutting condition, it is cut again. 3.根据权利要求1所述的方法,其特征在于,对于有多条两侧图像子块满足所述切割条件的分割线的图像块,选择分割线两侧图像子块有效信息密度差值最大的分割线作为实际分割线。3. The method according to claim 1, characterized in that, for an image block with multiple segmentation lines whose two side image sub-blocks satisfy the cutting conditions, the segmentation line with the largest effective information density difference between the two side image sub-blocks is selected as the actual segmentation line. 4.根据权利要求1所述的方法,其特征在于,当所述分割线两侧图像子块满足所述切割条件时,根据拼接图像素率限制和未丢弃的图像块的有效信息密度,更新阈值。4. The method according to claim 1, wherein when the image sub-blocks on both sides of the dividing line satisfy the cutting conditions, the threshold is updated according to the pixel rate limit of the stitched image and the effective information density of the undiscarded image blocks. 5.一种处理图像块的装置,其特征在于,包括以下模块:5. An apparatus for processing image blocks, characterized in that it comprises the following modules: 计算有效信息密度模块:用于在多视点图像块中沿贯穿且垂直于所述图像块边界的分割线,分别计算所述分割线两侧图像子块的有效信息密度,所述有效信息为所述图像子块在多视点中相对于其他视点的额外信息;The effective information density calculation module is used to calculate the effective information density of the image sub-blocks on both sides of the segmentation line that runs through and is perpendicular to the boundary of the image block in the multi-view image block. The effective information is the additional information of the image sub-block relative to other viewpoints in the multi-view. 输入:多视点图像中图像块;Input: Image patches in a multi-view image; 输出:分割线两侧的图像子块的有效信息密度;Output: Effective information density of the image sub-blocks on both sides of the segmentation line; 阈值判断模块:用于判断所述分割线两侧的图像子块的有效信息密度是否一个大于阈值,一个小于阈值,满足条件则切割,不满足条件则不切割;Threshold determination module: used to determine whether the effective information density of the image sub-blocks on both sides of the dividing line is greater than the threshold and less than the threshold. If the condition is met, the blocks are cut; if the condition is not met, the blocks are not cut. 输入:分割线两侧的各自的有效信息密度,阈值;Input: The effective information density on each side of the dividing line, and the threshold; 输出:切割决策;Output: Cutting decision; 切割模块:在满足切割条件的情况下,沿着所述分割线进行切割;Cutting module: Cuts along the dividing line when the cutting conditions are met; 输入:分割线,切割决策;Input: dividing line, cutting decision; 输出:切割后的图像块;Output: The cut image patch; 组装拼接决策模块:用于丢弃包含有效信息密度小于阈值的图像子块、且不存在满足所述切割条件的分割线的图像块,不组装进入拼接图中;Assembly and stitching decision module: used to discard image sub-blocks containing effective information density less than the threshold and image blocks that do not have dividing lines that meet the cutting conditions, and not assemble them into the stitched image; 输入:图像块;Input: Image patch; 输出:组装拼接决策。Output: Assembly and splicing decision. 6.根据权利要求5所述的装置,其特征在于,对于包含有效信息密度小于阈值的图像子块的图像块,若存在满足所述切割条件的分割线,则对其进行再次切割。6. The apparatus according to claim 5, wherein for an image block containing image sub-blocks with an effective information density less than a threshold, if there is a dividing line that satisfies the cutting condition, it is cut again. 7.根据权利要求5所述的装置,其特征在于,还包括以下模块:7. The apparatus according to claim 5, characterized in that it further comprises the following modules: 分割线决策模块:对于有多条两侧图像子块满足所述切割条件的分割线时的图像块,选择分割线两侧图像子块有效信息密度差值最大的分割线作为实际分割线;Segmentation line decision module: For an image block with multiple segmentation lines whose two side image sub-blocks satisfy the cutting conditions, the segmentation line with the largest difference in effective information density between the two side image sub-blocks is selected as the actual segmentation line; 输入:多条满足切割条件的分割线,有效信息密度差值;Input: Multiple dividing lines that meet the cutting conditions, and the effective information density difference; 输出:实际分割线。Output: The actual dividing line. 8.根据权利要求5所述的装置,其特征在于,还包括以下模块:8. The apparatus according to claim 5, characterized in that it further comprises the following modules: 阈值更新模块:当所述分割线两侧图像子块满足所述切割条件时,根据拼接图像素率限制和未丢弃的图像块的有效信息密度,更新阈值;Threshold update module: When the image sub-blocks on both sides of the dividing line meet the cutting conditions, the threshold is updated according to the pixel rate limit of the stitched image and the effective information density of the undiscarded image blocks; 输入:未丢弃的图像块的有效信息密度,拼接图像素率限制;Input: Effective information density of the undiscarded image patches, pixel rate limit of the stitched image; 输出:更新后的阈值。Output: The updated threshold.
CN202311299927.5A 2023-10-09 2023-10-09 Method and device for processing image block in immersive media coding Active CN119815012B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202311299927.5A CN119815012B (en) 2023-10-09 2023-10-09 Method and device for processing image block in immersive media coding

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202311299927.5A CN119815012B (en) 2023-10-09 2023-10-09 Method and device for processing image block in immersive media coding

Publications (2)

Publication Number Publication Date
CN119815012A CN119815012A (en) 2025-04-11
CN119815012B true CN119815012B (en) 2025-12-09

Family

ID=95266743

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202311299927.5A Active CN119815012B (en) 2023-10-09 2023-10-09 Method and device for processing image block in immersive media coding

Country Status (1)

Country Link
CN (1) CN119815012B (en)

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103124347A (en) * 2012-10-22 2013-05-29 上海大学 Method for guiding multi-view video coding quantization process by visual perception characteristics
CN113347403A (en) * 2021-04-19 2021-09-03 浙江大学 Image processing method and device

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2013110555A (en) * 2011-11-21 2013-06-06 Sharp Corp Image encoder, image decoder, and method and program thereof
KR20140127177A (en) * 2013-04-23 2014-11-03 삼성전자주식회사 Method and apparatus for multi-view video encoding for using view synthesis prediction, method and apparatus for multi-view video decoding for using view synthesis prediction
US12238272B2 (en) * 2021-10-05 2025-02-25 Sony Group Corporation Adaptive mode selection for point cloud compression
CN116797948B (en) * 2023-06-12 2025-11-28 浙江大学嘉兴研究院 Cross-view scene matching method for unmanned aerial vehicle image and satellite image

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103124347A (en) * 2012-10-22 2013-05-29 上海大学 Method for guiding multi-view video coding quantization process by visual perception characteristics
CN113347403A (en) * 2021-04-19 2021-09-03 浙江大学 Image processing method and device

Also Published As

Publication number Publication date
CN119815012A (en) 2025-04-11

Similar Documents

Publication Publication Date Title
JP7751716B2 (en) Point Cloud Processing
US20200007872A1 (en) Video decoding method, video decoder, video encoding method and video encoder
JP7477528B2 (en) Point Cloud Processing
US8355596B2 (en) Image generation method and apparatus, program therefor, and storage medium which stores the program
WO2011060579A1 (en) Method for generating depth maps from monocular images and systems using the same
US12106526B2 (en) Processing a point cloud
CN102113015A (en) Use of inpainting techniques for image correction
CN103179405A (en) Multi-stage area-of-interest-based multi-view video coding method
Gu et al. Fast bi-partition mode selection for 3D HEVC depth intra coding
CN104105006A (en) Video image processing method and system
US20230306563A1 (en) Image filling method and apparatus, decoding method and apparatus, electronic device, and medium
JP2024170438A (en) Encoding and Decoding Point Clouds Using Patches of Intermediate Samples
CN104661014B (en) The gap filling method that space-time combines
CN119815012B (en) Method and device for processing image block in immersive media coding
CN102572471A (en) Occlusion layer extension
JP7602694B2 (en) Enforcing Minimum and Average Distance Constraints in Video Streaming
US20150245064A1 (en) Coding Method And Device Applied To HEVC-based 3DVC
KR101332386B1 (en) Apparatus and method for acquisition stereoscopic image data
JP2025060922A (en) Method and device for encoding/reconstructing 3D points - Patents.com
CN116261854A (en) Trimming process correction for cloud-based encoding of HDR video
CN115396696B (en) Video data transmission method, system, processing device and storage medium
CN115239753A (en) An image processing method, system, device and storage medium
CN118020293A (en) Apply minimum and average distance constraints to video streams
TWI739466B (en) Image processing method and associated encoder
EP3588962A1 (en) Video decoding method, video decoder, video encoding method and video encoder

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant