KR20120022533A - Method and apparatus for image processing based on multiple texture image - Google Patents

Abstract

Disclosed is a multi-texture image-based image processing technology capable of processing an image at an optimal image quality at a low bit rate. To this end, the multi-texture image-based image processing method according to the present invention comprises the steps of classifying the input image as a shot unit image, and selecting one frame of a plurality of frames of the shot unit image as a seed image; Detecting a plurality of feature points in the seed image; Calculating a spatiotemporal position transformation variable for each of the feature points by tracking the plurality of feature points in the plurality of frames of the shot unit image; And defining a plurality of texture images using feature points corresponding to the spatiotemporal position transformation variables.

Description

Method and apparatus for image processing based on multiple texture images based on multiple texture image

The present invention relates to a multi-texture image-based image processing method and apparatus. More specifically, the present invention relates to a multi-texture image-based image processing method and apparatus capable of processing an image at an optimal image quality at a low data rate.

A general image processing method is based on a method of processing a transform region using a discrete cosine transform, for example, an image inter-frame motion estimation process. However, the general image processing method has a problem in that it is difficult to express various real image characteristics due to incorrect estimation and modeling of image characteristics. As a result, the difference between the converted video signal and the original video signal increases, resulting in a large bit rate when the video signal is compressed. In order to solve this problem, video compression standards represented by MPEG1 / 2/4 and H.261 / 263/264 have been proposed. However, the image quality deterioration is still serious at low bit rate video compression such as 1/500 bit rate compared to the original size. . In addition, 1 / n pixel-based motion estimation and compensation methods, adaptive block size conversion domain image processing methods, multiple reference frame motion estimation and compensation methods, and generalized B-frame processing methods have been proposed and used. Image quality deterioration is serious.

An object of the present invention is to express a variety of image characteristics through a plurality of texture images and the spatio-temporal position transformation variable of the texture image.

In addition, an object of the present invention is to provide a compressed image that is significantly reduced in size compared to the original size by compressing the original image with only a plurality of texture images and a plurality of corresponding spatio-temporal position transformation variables.

In addition, the present invention aims to further reduce the size of the compressed image by approximating a plurality of predetermined texture images using similarity of the spatiotemporal position transformation variables.

In addition, an object of the present invention is to process an image with an optimum image quality at a low data rate.

The multi-texture image-based image processing method according to the present invention for achieving the above object comprises the steps of classifying the input image into a shot unit image, and selecting one frame of a plurality of frames of the shot unit image as a seed image; Detecting a plurality of feature points in the seed image; Calculating a spatiotemporal position change variable for each of the feature points by tracking the plurality of feature points in the plurality of frames of the shot unit image; And defining a plurality of texture images using feature points corresponding to the spatiotemporal position conversion variables.

In this case, the method may further include compressing the plurality of texture images and the spatiotemporal position transformation variables corresponding to the respective texture images.

At this time, in the compressing, the texture image is compressed by an image compression method, and the spatiotemporal position conversion variable is separately compressed by a bitstream compression method.

In this case, in the plurality of texture images, texture images having spatio-temporal position transformation variables having similar values calculated by obtaining correlation characteristics of texture image signals having values within a preset threshold as one texture image. And further approximating.

In this case, the detecting of the plurality of feature points detects, as the feature points, points having a change amount greater than or equal to a predetermined numerical value in the plurality of frames.

Decompressing the plurality of compressed texture images and the spatiotemporal position conversion variables corresponding to the respective texture images; Matching the texture image and the spatiotemporal position transformation variable corresponding to the texture image; Generating a visual texture using the texture image and the spatiotemporal position transformation variable; And combining the visual textures generated corresponding to each of the texture images.

In this case, the method may further include filtering and correcting an artifact at the combined boundary of the visual textures.

In addition, the multi-texture image-based image processing apparatus according to the present invention for achieving the above object is to classify the input image as a shot unit image, the seed for selecting one frame of a plurality of frames of the shot unit image as a seed image An image selector; A feature point detector configured to detect a plurality of feature points from the seed image; A variable calculator configured to track the plurality of feature points in the plurality of frames of the shot unit image and calculate a space-time position conversion variable for each feature point; And a texture image definition unit defining a plurality of texture images using feature points corresponding to the space-time position conversion variable.

In this case, the apparatus may further include a compression unit configured to compress the plurality of texture images and the spatiotemporal position conversion variables corresponding to the respective texture images.

In this case, the compression unit compresses the texture image by an image compression method and separately compresses the spatiotemporal position conversion variable by a bitstream compression method.

In this case, in the plurality of texture images, texture images having spatio-temporal position transformation variables having similar values calculated by obtaining correlation characteristics of texture image signals having values within a preset threshold as one texture image. It further includes an approximation unit that approximates in total.

At this time, the feature point detector detects, as the feature point, a point having a change amount equal to or greater than a predetermined numerical value in the plurality of frames.

At this time, the decompression unit for decompressing the plurality of compressed texture image and the space-time position conversion variable corresponding to each of the texture image; A matching unit matching the texture image and the space-time position conversion variable corresponding to the texture image; A visual texture generator for generating a visual texture using the texture image and the spatiotemporal position change variable; And a visual texture combiner that combines the generated visual textures corresponding to the respective texture images.

In this case, the apparatus may further include a correction unit configured to filter and correct an artifact at the combined boundary of the visual textures.

According to the present invention, various image characteristics may be expressed through a plurality of texture images and a space-time position transformation variable of the texture image.

In addition, the present invention can compress the original image using only a plurality of texture images and a plurality of spatio-temporal position change variables corresponding thereto, thereby providing a compressed image having a significantly reduced size compared to the original size.

In addition, the present invention can further reduce the size of the compressed image by approximating a plurality of predetermined texture images using similarities of the spatiotemporal position transformation variables.

In addition, the present invention can process an image with an optimum image quality at a low data rate. That is, the present invention can minimize image quality degradation at low data rates such as 1/500 bit rate.

1 is a flowchart illustrating an encoding method in a multi-texture image-based image processing method according to the present invention.
2 is a view for explaining a method of encoding in a multi-texture image-based image processing method according to the present invention.
3 is a flowchart illustrating a method of decoding in the multi-texture image-based image processing method according to the present invention.
4 is a diagram for describing a decoding method in the multi-texture image-based image processing method according to the present invention.
5 is a block diagram showing the configuration of a multi-texture image-based image processing apparatus according to the present invention.

Hereinafter, the present invention will be described in detail with reference to the accompanying drawings. Here, the repeated description, well-known functions and configurations that may unnecessarily obscure the subject matter of the present invention, and detailed description of the configuration will be omitted. Embodiments of the present invention are provided to more completely describe the present invention to those skilled in the art. Accordingly, the shape and size of elements in the drawings may be exaggerated for clarity.

Hereinafter, the encoding method in the multi-texture image-based image processing method according to the present invention will be described.

1 is a flowchart illustrating an encoding method in a multi-texture image-based image processing method according to the present invention. 2 is a view for explaining a method of encoding in a multi-texture image-based image processing method according to the present invention.

1 and 2, the encoding method of the multi-texture image-based image processing method according to the present invention first receives an image 200 composed of a plurality of frames (S10).

The input image 200 is classified into a shot unit image, and one frame among the plurality of frames of the shot unit image is selected as the seed image 210 (S11). The remaining frames other than the seed image 210 of the shot unit image are defined as the remaining frame image 220. That is, when the shot unit image is composed of k frames, one seed image is selected, and the remaining k-1 frames are defined as the remaining frame image 220. In this case, the shot unit image corresponds to an image that one camera continuously photographs.

A plurality of feature points are detected in the seed image 210 selected in step S11 (S12). In this case, in a plurality of frames of the shot unit image, a point having a change amount greater than or equal to a predetermined numerical value may be detected as a feature point. That is, when a specific point in the seed image 210 and the remaining frame image 220 shows a change over a predetermined value, the specific point may be detected as a feature point.

A plurality of feature points are tracked in a plurality of frames of the shot unit image to calculate a spatiotemporal position change variable for each feature point (S13). That is, the spatiotemporal position transformation variable defining the change of the feature point in the seed image 210 and the remaining frame image 220 is calculated. The space-time position transformation variable may be in the form of a function indicating an amount of change of the position of the feature point over time.

The space-time position transformation variables 211b, 212b, 213b, 214b, and Nb calculated in step S13 define a plurality of texture images 211a, 212a, 213a, 214a, and Na using feature points corresponding to each other (S14). ). At this time, the spatiotemporal position transformation variables 211b, 212b, 213b, 214b, and Nb may define one texture image by linking the same feature points.

In the plurality of texture images, texture images having similar spatiotemporal position transformation variables are combined and approximated as one texture image (S15). In this case, the similarity between the spatiotemporal position transformation variables may be calculated by obtaining correlation characteristics of the texture image signal. Similarity between the spatiotemporal position transformation variables may combine texture images having values within a preset threshold into one texture image. In FIG. 2, the first texture image 211a and the second texture image 212a are assumed to have high similarity to the space-time position transformation variable, and correspondingly, the first space-time position transformation variable 211b and the second space-time position are combined. By combining the transform variables 212b, a first approximated texture image 211a 'and a first approximated spatiotemporal position change variable 211b' are generated. Then, the third texture image 213a and the fourth texture image 214a are combined, and the third space-time position transformation variable 213b and the fourth space-time position transformation variable 214b are correspondingly combined to form a second image. An approximated texture image 213a 'and a second approximated spatiotemporal position change variable 213b' are generated.

Then, the plurality of texture images 211a, 212a, 213a, 214a, and Na and a plurality of spatiotemporal position conversion variables 211b, 212b, 213b, 214b, and Nb corresponding to the respective texture images are compressed (S16). In addition, in step S16, on the premise that the approximation step S15 of the texture image is performed, the plurality of approximation texture images 211a ', 213a', and Na 'and the plurality of approximation spatiotemporal positions corresponding to the respective approximation texture images Conversion variables 211b ', 213b', and Nb 'may be compressed. In this case, the plurality of texture images 211a, 212a, 213a, 214a, and Na may be compressed by an image compression method. The plurality of spatiotemporal position conversion variables 211b, 212b, 213b, 214b, and Nb may be compressed separately from the plurality of texture images 211a, 212a, 213a, 214a, and Na by a bitstream compression method. Of course, the plurality of approximated texture images 211a ', 213a', and Na 'are compressed by an image compression method, and the plurality of approximated spatiotemporal position conversion variables 211b', 213b ', and Nb' are separately separated by a bitstream compression method. It may be compressed.

Hereinafter, the encoding method in the multi-texture image-based image processing method according to the present invention will be described by using an equation.

에 대하여, 자기 상관 매트릭스(Autocorrelation matrix)

를 계산한다. 여기서,

는 {x,y}가

를 만족할 때의 포인트의 주변 윈도우 신호이다. 그리고, x와 y는 각각 x축 방향과 y축 방향의 픽셀 포인트이며,

는 통계적 기대 함수(Statistical expectation operator)로 정의된다. The feature point can be detected as follows. First, an input image consisting of k frames

With respect to the autocorrelation matrix

Calculate here,

Is { x , y }

The window signal around the point at which is satisfied. And x and y are pixel points in the x- and y- axis directions, respectively.

Is defined as a statistical expectation operator.

를 통하여 계산된 고유치들 즉,

및

로부터, 텍스쳐 포인트 매트릭스

를 다음의 수학식 1과 같이 구할 수 있다. At pixel point { x , y }

Eigenvalues computed through

And

Texture point matrix

Can be obtained as in Equation 1 below.

및

는 기 설정된 임계치에 해당한다. 상기 수학식 1에서 특정 픽셀 위치의

및

가 임계치

및

보다 큰 경우, 해당 특정 픽셀을 1로 정의한다. 그리고, 특정 픽셀 위치의

및

가 임계치

및

보다 작은 경우, 해당 특정 픽셀을 0으로 정의하여 텍스쳐 포인트 매트릭스를 구한다.
here,

And

Corresponds to a preset threshold. In Equation 1

And

Threshold

And

If it is larger, the specific pixel is defined as 1. And the specific pixel position

And

Threshold

And

If it is smaller, we define that particular pixel as 0 to get a texture point matrix.

In addition, a plurality of space-time position transformation variables defining each texture image and texture images thereof may be defined as in Equation 2 below.

는 다음의 수학식 3과 같이 정의될 수 있다.here,

May be defined as in Equation 3 below.

는 다음의 수학식 4와 같이, N개의 텍스쳐 이미지의 합으로써 정의될 수 있다. And, the input image consisting of k frames

May be defined as the sum of N texture images, as shown in Equation 4 below.

In addition, the i- th divided texture image in Equation 4 may be expressed by approximating Equation 5 below.

는 전달 함수를,

은 입력 영상의 l번째 프레임의 i번째 분할된 텍스쳐 이미지를,

는 x축 방향과 y축 방향의 위치변환 벡터를,

는

에서의 개략적인 추정 오류 신호를 나타낸다. 그리고, 수학식 5에서, 프레임 넘버 k는 k+1부터 l+M의 범위에 속한다. 수학식 5는 Taylor expansion에 의하여 다음의 수학식 6과 같이 근사화될 수 있다.here,

Is a transfer function,

Is the i- th partitioned texture image of the l- th frame of the input image,

Is the displacement vector in the x- and y-axis directions,

Is

Shows a rough estimation error signal in. In Equation 5, the frame number k is in the range of k + 1 to l + M. Equation 5 may be approximated by Equation 6 by Taylor expansion.

및

각각은

의 x축 방향과 y축 방향의 경사도(Gradient value)의 합을 나타낸다. 그리고, 추정 오류 신호의 제곱합에 대한 정리는 다음의 수학식 7과 같이 나타낼 수 있다. here,

And

Each one

The sum of the gradient values in the x-axis direction and the y-axis direction of. The theorem for the sum of squares of the estimated error signals may be expressed as in Equation 7 below.

크기의 최소화를 가정하여,

의 값을 구할 수 있다. 즉, 다음의 수학식 8 및 수학식 9를 계산하여,

의 값을 구한다.Where the sum of squares of the estimated error signals

Assuming minimization of size,

Can be found. That is, by calculating the following Equations 8 and 9,

Find the value of.

가 항등변환(Identity transform)식임을 가정하면, 수학식 7과 수학식 8을 통해, 다음의 수학식 10을 얻을 수 있다.here,

Assuming that is an identity transform equation, the following equation 10 can be obtained through equations (7) and (8).

에 대한 다음의 수학식 11을 얻을 수 있다.And, if you solve the equation (10),

The following equation (11) can be obtained.

및 수학식 9를 사용하여 다음의 수학식 12와 같은 변환함수

를 얻을 수 있다.And, obtained through the equation (11)

And a conversion function such as the following Equation 12 using Equation 9

Can be obtained.

의 변환함수

으로부터

를 얻을 수 있다. 그리고,

의 변환함수

으로부터

를 얻을 수 있다. 또한, 수학식 1 내지 수학식 12를 통해,

를 시드 이미지

및 변환함수

로써 표현할 수 있다. 그리고,

간의 유사성을 계산하여 텍스쳐 이미지의 근사화가 이루어질 수 있다. Summarizing Equation 8 and Equation 9,

Conversion function of

From

Can be obtained. And,

Conversion function of

From

Can be obtained. Further, through Equations 1 to 12,

Seed image

And conversion functions

Can be expressed as: And,

An approximation of the texture image can be made by calculating the similarity between them.

는 H.264 인코더와 같은 방법에 의하여 압축될 수 있다. 그리고,

는 CAVLC(Context Adaptive Variable Length Coder) 또는 CABAC(Context Adaptive Binary Arithmetic Coder)에 의하여 압축될 수 있다.
These seed images

Can be compressed by the same method as the H.264 encoder. And,

Can be compressed by Context Adaptive Variable Length Coder (CAVLC) or Context Adaptive Binary Arithmetic Coder (CABAC).

Hereinafter, a decoding method of the multi-texture image-based image processing method according to the present invention will be described.

3 is a flowchart illustrating a method of decoding in the multi-texture image-based image processing method according to the present invention. 4 is a diagram for describing a decoding method in the multi-texture image-based image processing method according to the present invention.

3 and 4, the decoding method of the multi-texture image-based image processing method according to the present invention first receives a compressed image signal (S30). In this case, the compressed image signal may be a signal in which a plurality of texture images and a plurality of spatiotemporal position transformation variables corresponding to each texture image are compressed. Of course, the compressed video signal may be a signal obtained by compressing a plurality of approximated texture images and a plurality of approximated spatiotemporal position transformation variables. Also, in the compressed image signal, the plurality of texture images and the plurality of spatiotemporal position transformation variables may be separately compressed.

Then, the compressed video signal is decompressed (S31). That is, the plurality of compressed texture images and the plurality of space-time position transformation variables corresponding to each texture image are decompressed.

In the plurality of decompressed texture images and the plurality of space-time position transformation variables, the space-time position transformation variables corresponding to each texture image and the corresponding texture image are matched one-to-one (S32). Of course, each of the approximated texture images and the approximated spatiotemporal position transformation variables corresponding to the corresponding approximated texture images may be matched. In FIG. 4, the first approximation texture image 211a 'and the first approximation spatiotemporal position transformation variable 211b' are matched, and the second approximation texture image 213a 'and the second approximation spatiotemporal position transformation variable 213b'. Is matched, and the Nth approximated texture image Na 'and the Nth approximated spatiotemporal position change variable Nb' are matched.

In operation S32, a visual texture is generated using the matched texture image and the space-time position transformation variable in operation S33. Specifically, a visual texture consisting of a plurality of frames for the texture image is generated by applying a spatiotemporal position change variable that defines a time-dependent movement of feature points, etc., to the texture image. Of course, a visual texture may be generated by using a matched approximation texture image and an approximation spatiotemporal position transformation variable. In FIG. 4, a first visual texture 211 composed of a plurality of frames is generated by using the first approximated texture image 211a 'and the first approximated spatiotemporal position change variable 211b'. Then, a second visual texture 213 composed of a plurality of frames is generated using the second approximated texture image 213a 'and the second approximated spatiotemporal position change variable 213b'. Also, an Nth visual texture N including a plurality of frames is generated by using the Nth approximated texture image Na 'and the Nth approximated spatiotemporal position change variable.

In operation S33, the visual textures generated in correspondence with the respective texture images are combined (S34). By combining the visual textures, a plurality of frames of the shot unit image are reconstructed as a whole. In FIG. 4, the first visual texture 211, the second visual texture 213, and the Nth visual texture N are combined.

Artifacts at the combined boundary of the plurality of visual textures combined in step S34 are filtered and corrected (S35). That is, the plurality of visual textures combined in step S34 are restored as a simple sum, and a defect may occur at the boundary between the visual textures. By performing removal filtering on these defects, a final corrected reconstructed image is generated.

In addition, the reconstructed images of the other shot units may be combined with the reconstructed images of the shot units obtained through operation S35 to generate the final image 200 ′.

Hereinafter, the configuration and operation of the multi-textured image-based image processing apparatus according to the present invention will be described.

5 is a block diagram showing the configuration of a multi-texture image-based image processing apparatus according to the present invention.

Referring to FIG. 5, the multi-texture image-based image processing apparatus 500 according to the present invention may include an encoder 510 and a decoder 520.

The encoder 510 includes a seed image selector 511, a feature point detector 512, a variable calculator 513, and a texture image definer 514. Also, the encoding unit 510 may further include an approximation unit 515 and a compression unit 516.

The seed image selector 511 classifies the received image into a shot unit image, and selects one frame among a plurality of frames of the shot unit image as a seed image. The seed image selector 511 defines the remaining frames as the remaining frame image except for the seed image of the shot unit image. That is, the seed image selector 511 selects one seed image when the shot unit image is composed of k frames, and defines the remaining k-1 frames as the remaining frame image 220. In this case, the shot unit image corresponds to an image that one camera continuously photographs.

The feature point detector 512 detects a plurality of feature points from the seed image selected by the seed image selector 511. In this case, the feature point detector 512 may detect, as a feature point, a point having a change amount equal to or greater than a predetermined value in a plurality of frames of the shot unit image. That is, the feature point detector 512 may detect the specific point as the feature point if the specific point in the seed image and the remaining frame image shows a change more than a predetermined value.

The variable calculator 513 tracks a plurality of feature points in a plurality of frames of the shot unit image and calculates a space-time position conversion variable for each feature point. That is, the variable calculator 513 calculates a spatiotemporal position transformation variable that defines a change of feature points in the seed image and the remaining frame image. The space-time position transformation variable may be in the form of a function indicating an amount of change of the position of the feature point over time.

The texture image defining unit 514 defines a plurality of texture images using feature points to which the spatiotemporal position conversion variables calculated by the variable calculating unit 513 correspond to each other. In this case, the texture image defining unit 514 may define one texture image by linking feature points having the same space-time position transformation variables.

The approximation unit 515 approximates a plurality of texture images by combining texture images having similar spatiotemporal displacement variables into one texture image. That is, the approximator 515 may generate a plurality of approximated texture images and a plurality of approximated spatiotemporal position transformation variables to which the plurality of texture images and the plurality of spatiotemporal position transformation variables are approximated. In this case, the approximation unit 515 may calculate the similarity between the spatiotemporal position transformation variables by obtaining correlation characteristics of the texture image signal. In addition, the approximation unit 515 may combine the texture images having values within the preset threshold similarity between the spatiotemporal position transformation variables into one texture image.

The compression unit 516 compresses the plurality of texture images and the plurality of spatiotemporal position conversion variables corresponding to each texture image. Of course, the compression unit 516 may compress the plurality of approximated texture images and the plurality of spatiotemporal position transformation variables. The compressor 516 may compress the plurality of texture images by an image compression method. The compression unit 516 may compress the plurality of spatiotemporal position conversion variables separately by a bitstream compression method. Of course, the compression unit 516 may compress the plurality of approximated texture images by an image compression method, and separately compress the plurality of approximated spatiotemporal position change variables by a bitstream compression method.

The decoder 520 includes a decompressor 521, a matcher 522, a visual texture generator 523, and a visual texture combiner 524. In addition, the decoder 520 may further include a corrector 525.

The decompressor 521 receives the compressed video signal from the encoder 510 and decompresses the compressed video signal. The decompressor 521 decompresses a plurality of compressed texture images and a plurality of spatiotemporal position conversion variables corresponding to each of the texture images.

The matching unit 522 matches the texture images and the spatiotemporal position transformation variables corresponding to the texture images in a one-to-one manner in the plurality of texture images and the plurality of space-time position transformation variables decompressed by the decompressor 521. Of course, the matching unit 522 may match the approximated spatiotemporal position transformation variable corresponding to each approximated texture image and the corresponding approximated texture image.

The visual texture generator 523 generates a visual texture using the matched texture image and the space-time position transformation variable. In detail, the visual texture generation unit 523 generates a visual texture composed of a plurality of frames for the texture image by applying a spatiotemporal position conversion variable that defines a time-dependent movement of feature points, etc., to the texture image. Of course, the visual texture generator 523 may generate a visual texture using the matched approximated texture image and the approximated spatiotemporal position transformation variable.

The visual texture combiner 524 combines the visual textures generated by the visual texture generator 523 corresponding to each texture image. By combining the visual textures, a plurality of frames of the shot unit image are reconstructed as a whole.

The corrector 525 filters and corrects artifacts at the combined boundary of the plurality of combined visual textures. That is, the plurality of visual textures combined by the visual texture combiner 524 are restored as a simple sum, and a defect may occur at a boundary between the visual textures. The correction unit 525 generates a final corrected reconstructed image by performing removal filtering on the defect.

As described above, the multi-texture image-based image processing method and apparatus according to the present invention are not limited to the configuration and method of the embodiments described as described above, but the embodiments may be modified in various ways. All or some of the embodiments may be optionally combined.

500; Multi-texture image-based image processing device
510; Encoding Section
511; A seed image selector 512; Feature point detector
513; Variable calculator 514; Texture Image Definition
515; Approximation section 516; Compression part
520; Decoding unit
521; Decompression unit 522; Matching part
523; Visual texture generator 524; Visual Texture Joint
525; Compensator

Claims (14)

Classifying the input image into a shot unit image and selecting one frame among a plurality of frames of the shot unit image as a seed image;
Detecting a plurality of feature points in the seed image;
Calculating a spatiotemporal position change variable for each of the feature points by tracking the plurality of feature points in the plurality of frames of the shot unit image; And
And defining a plurality of texture images using feature points corresponding to the spatiotemporal position conversion variables. The method according to claim 1,
And compressing the plurality of texture images and the spatiotemporal position transformation variables corresponding to each of the texture images. The method according to claim 2,
The compressing step,
The texture image is compressed using an image compression method, and the spatiotemporal position conversion variable is compressed separately using a bitstream compression method. Claim 1
In the plurality of texture images, the similarity calculated by obtaining correlation characteristics of the texture image signal is approximated by combining texture images having spatiotemporal displacement variables having values within a predetermined threshold. The multi-texture image-based image processing method further comprising the step. The method according to claim 1,
Detecting the plurality of feature points,
And detecting a point having a change amount greater than or equal to a predetermined value as the feature point in the plurality of frames. The method according to claim 2,
Decompressing the plurality of compressed texture images and the spatiotemporal position transformation variables corresponding to the respective texture images;
Matching the texture image and the spatiotemporal position transformation variable corresponding to the texture image;
Generating a visual texture using the texture image and the spatiotemporal position transformation variable; And
And combining the visual textures generated in correspondence to the respective texture images. The method of claim 6,
And filtering and correcting artifacts at the combined boundary of the visual textures. A seed image selection unit classifying the input image into a shot unit image and selecting one frame among a plurality of frames of the shot unit image as a seed image;
A feature point detector configured to detect a plurality of feature points from the seed image;
A variable calculator configured to track the plurality of feature points in the plurality of frames of the shot unit image and calculate a space-time position conversion variable for each feature point; And
And a texture image definition unit defining a plurality of texture images using feature points corresponding to the spatiotemporal position conversion variables. The method according to claim 8,
And a compression unit for compressing the plurality of texture images and the spatiotemporal position transformation variables corresponding to each of the texture images. The method according to claim 9,
The compression unit,
And the texture image is compressed by an image compression method, and the spatiotemporal position conversion variable is separately compressed by a bitstream compression method. The method according to claim 8,
In the plurality of texture images, the similarity calculated by obtaining correlation characteristics of the texture image signal is approximated by combining texture images having spatiotemporal displacement variables having values within a predetermined threshold. The multi-texture image-based image processing apparatus further comprising an approximation unit. The method according to claim 8,
The feature point detector,
The multi-texture image-based image processing apparatus of claim 1, wherein the plurality of frames detect a point having a change amount greater than or equal to a predetermined value as the feature point. The method according to claim 9,
A decompression unit configured to decompress the plurality of compressed texture images and the spatiotemporal position conversion variables corresponding to the respective texture images;
A matching unit matching the texture image and the space-time position conversion variable corresponding to the texture image;
A visual texture generator for generating a visual texture using the texture image and the spatiotemporal position change variable; And
And a visual texture combiner for combining the visual textures generated in correspondence with the respective texture images. The method according to claim 13,
And a correction unit which filters and corrects artifacts at the combined boundary of the visual textures.

Images