JP2003116048A - Shading correction method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a shading correcting method for eliminating a two-dimensional deflection. SOLUTION: Two-dimensional correcting block data in a horizontal direction and a vertical direction is provided, and the two-dimensional correcting block data is subjected to filtering by a two-dimensional filter 5D. A shading correction of a video signal is conducted by using the processed signal, and a memory for storing the state of a vertical filter is provided.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a shading correction method for correcting unevenness due to image pickup conditions such as an optical system of a lens and an image pickup element, and a light source directionality when an image is picked up by a television camera. It is about.

[0002]

2. Description of the Related Art Aberration occurs in an image picked up by a television camera for various reasons. The aberration targeted by the present invention is an aberration called shading, and includes black shading and modulation shading. There are various reasons for the occurrence, such as unevenness in sensitivity of the optical system of the lens and the image pickup device, bias due to directional nonuniformity in the analog processing circuit in the television camera, and unevenness due to image pickup conditions such as the directionality of the light source.

Taking the unevenness due to the directionality of the light source as an example, when a camera is used for image measurement or the like, the object to be measured and the camera are close to each other, and the light source illuminates the subject obliquely. Non-uniformity occurs in the amount of light above. This adversely affects the image as modulation shading. When there is unevenness due to the directionality of the light source, when a uniform white image is displayed on the entire screen, the deviation due to the nonuniformity of the light amount is included in the image as the modulation shading.

FIG. 12 shows an example of a shading signal.
FIG. 12 shows a case where there is a bias only in the vertical direction.
As a method of correcting the modulation shading, conventionally, a one-dimensional correction signal is generated in the vertical direction to correct the aberration.

FIG. 13 shows a conventional configuration and waveform. An image captured by an optical system 1 such as a lens passes through an image pickup element 2 such as a CCD and an image signal processing circuit 3 and becomes an electrical image signal. The waveform at this stage is shown in FIG. In the waveform 7, it is assumed that the level at the screen edge is 50% when the center position of the bias is 100%. This information is measured in blocks. However, the deviation is only in the vertical direction, the horizontal direction is constant, and a specific region in the horizontal direction is representatively measured. The relationship is also shown in FIG.

Next, in order to correct this bias, a correction signal is obtained in block units. The correction signal corresponds to gain. In this example, assuming that the entire level is adjusted to the center of the screen, the correction signal of the block corresponding to the center of the screen, that is, the gain is set to 1
If it is doubled, the gain G of the block corresponding to the screen edge is 50.
% × G = 100%, and G = 2 times. The correction data obtained for each block is stored in the memory 6. The block data is read from the memory 6 in synchronization with the vertical synchronizing signal of the video. The waveform is shown in 9.

Next, smoothing is performed in the analog filter circuit 5 to obtain a correction signal. The waveform of this correction signal is shown in FIG. When this correction signal is multiplied by the video signal in the multiplication circuit 4 ', the signal is uniformly 100% and uniform. Shading correction was performed by such a method.

Now, the characteristics of the smoothing filter will be described. FIG. 14 shows the smoothing filter characteristic of the filter circuit 5 of FIG. In FIG. 14, a curve connecting circles is a filter output, and a staircase waveform is a filter input. One block has N clocks, and the sampling period in block units is Fo.

The frequency components of the block signal are shown in FIG. That is, the sampling frequency of the block signal is set to Fs
Then, Fo = Fs / N, and there is a harmonic component at an integral multiple frequency of Fo at intervals of Fo. Smoothing to remove the block step is nothing but to remove this harmonic component.

[0010]

As described above, the conventional shading correction is one-dimensional processing, whereas the actual bias has a two-dimensional deviation. In particular, "unevenness" due to the directionality of the light source includes a complicated deviation, and the conventional one-dimensional correction method deviation remains, which is insufficient.

An object of the present invention is to provide a shading correction method for removing a two-dimensional deviation.

[0012]

The present invention provides a shading correction method for correcting aberration of a captured image in real time, comprising two-dimensional correction block data in the horizontal and vertical directions, and the two-dimensional correction. The shading correction method is characterized in that the block data for use is smoothed by a two-dimensional filter, and the shading correction of the video signal is performed using the processed signal.

The present invention is the shading correction method, wherein the two-dimensional filter is composed of a vertical filter and a horizontal filter, and a buffer memory for storing the filter state of the vertical filter is provided. .

According to the present invention, the two-dimensional filter is composed of a vertical direction filter and a horizontal direction filter, and the vertical direction filter is updated line by line by the number of taps of the vertical direction filter. The shading correction method is characterized in that the resulting state is written in each tap.

[0015]

FIG. 1 shows a configuration having a shading correction method according to an embodiment of the present invention. An image captured by an optical system 1 such as a lens passes through an image pickup element 2 such as a CCD and an image signal processing circuit 3 and becomes an electrical image signal. The correction block data obtained in block units is stored in the memory 6. The correction block data is read from the memory 6 in synchronism with the horizontal / vertical synchronization signals of the video, smoothed by the two-dimensional digital filter circuit 5D to obtain the correction signal, and the correction signal is input to the subtraction circuit 4 as the video signal. Shading correction is performed by subtracting from.

The present invention is to provide block data in two dimensions in order to remove a two-dimensional deviation. A two-dimensional filter is required to smooth two-dimensional block data. This can be realized in principle by digitally configuring a two-dimensional filter.

FIG. 2 is a block diagram of the first embodiment of the two-dimensional digital filter circuit 5D shown in FIG. In FIG. 2, the two-dimensional digital filter circuit includes a line memory 51.
To 580, a vertical direction filter 59 composed of 80 line memories and a filter coefficient calculation circuit 581.
0, and a horizontal filter 7140 including 134 registers consisting of registers 71 to 7134 and a filter coefficient calculation circuit 7135.

This configuration corresponds to a screen size called XGA when a camera is used for image measurement. If the screen size is called XGA, horizontal sampling is performed. The number is 1344, and the number of lines in the vertical direction is 806. If the block interval is about 5% of the whole, 1344 x 5% in the horizontal direction
= 67, 806 × 5% = 40 lines in the vertical direction. The number of blocks is 20 × 20 = 400 blocks. In constructing the two-dimensional filter, the sampling pitch in the vertical direction is 40 lines, and the number of taps of the filter in the vertical direction is 40 × 2 = 80.

That is, 80 line memories are required. The total capacity of the line memory is 80 × 1344 = 10
It becomes 7520 words and becomes huge. The problem is that not only the capacitance, but also the output of each tap needs to be taken out when constructing the filter. In other words, unlike the field memory, it cannot be integrated into one memory, so 80 memories are required.

Therefore, as an improvement, if the sampling interval is narrowed, the number of taps of the filter is reduced and the memory is also reduced, but the number of blocks is increased. For example, if the sampling interval is 2 lines, the number of taps of the filter in the vertical direction is 2 × 2 = 4 taps, which is a feasible capacity, but the number of blocks is 806/2 lines = 403 blocks, two-dimensional. Then, horizontal × vertical = 20 × 403 = 8060 blocks are required. This increases the capacity of the memory that stores the block data, and also increases the calculation time of the CPU when obtaining the correction value for each block. Further, as the filter characteristic, the sampling pitch is too high, so that the deviation of the low frequency component cannot be taken.

For this reason, the means for narrowing the sampling interval is not effective either. Without narrowing the sampling interval,
A method of using a buffer memory instead of the line memory will be considered as a means for realizing a desired filter characteristic.

The method using this buffer memory realizes a two-dimensional filter by performing buffer processing for storing the state of the filter in the buffer memory, and does not require a line memory.

The capacity of the buffer memory is the number of filter taps × the number of vertical blocks. In the above example, 80 taps × 20 blocks = 1600 words, which is 1
It is about a line memory and has a realizable capacity.

Although it cannot be applied to a general signal that changes in a pixel unit, the object is in a block unit, that is, the signal width is constant for the block width.

FIG. 3 is a diagram showing a configuration and waveforms of a two-dimensional filter using buffer processing. The correction data storage memory 6, the buffer memory 5D-1, the vertical direction filter 5D-2, and the horizontal direction filter 5D-. It consists of 3.
1 corresponds to the correction data storage memory 6, and the digital filter 5D of FIG. 1 corresponds to the buffer memory 5D-1, the vertical direction filter 5D-2, and the horizontal direction filter 5D-. 3, the output of the horizontal direction filter 5D-3 is supplied to the subtraction circuit 4 of FIG. 1, and shading correction is performed by subtracting the output from the input video signal.

With this configuration, a two-dimensional filter can be constructed without using a line memory, so that a system advantageous in terms of mounting, price and power can be constructed. In the example of the screen size XGA (horizontal direction: 1344, vertical direction: 806), 80 line memories are reduced to one.

FIG. 3 shows a waveform example of the output of the vertical direction filter 5D-2. As can be seen from this waveform example, the vertical block data filtering () for one horizontal block width does not require memory,
It can be realized as a one-dimensional filter using a register. This is easy to understand if the deviation is only in the vertical direction and there is no deviation in the horizontal direction.

Here, the vertical and horizontal positions are expressed as (V, H). To simplify the explanation, if the total number of pixels in the horizontal direction is 1024 pixels, the total number of lines in the vertical direction is 256, and the size of the block is 32 pixels in the horizontal direction and 4 lines in the vertical direction, the pixel position on the screen is It is as shown in 7. That is, the pixel positions on the first line are (0,0) (0,1) (0,2) (0,3) (0,4) in order.
... (0,1023) The pixel positions of the second line are (1,0) (1,1) (1,2) (1,3) in order.
(1,4) ... (1,1023).

Let us consider the vertical filter output of the 9th pixel on the 5th line. Since the block size in the vertical direction is 4 lines, the number of taps is considered to be 4 + 1 + 4 = 9 taps. At this time, to obtain the vertical filter output, (0, 8)
(1,8) (2,8) (3,8) (4,8) (5,8)
Information of (6, 8) (7, 8) (8, 8) is required.
In the case of a signal that changes in general pixel units, for each pixel,
That is, since the input signal changes in accordance with the horizontal sampling operation, for example, in order to obtain the vertical filter output of the 10th pixel of the next 5th line, (0, 9) (1, 9)
(2,9) (3,9) (4,9) (5,9) (6,9)
Information of (7, 9) (8, 9) is required. After all, since information for all pixels is required, a line memory for 9 taps is required.

On the other hand, in the case of a block signal, for example, in the above example, since there is no change in the horizontal direction in units of 32 pixels in the horizontal direction, the vertical filter output of the 9th pixel of the 5th line and
It is the same as the vertical filter output of the 10th pixel of the line. That is, as shown in FIG. 8, up to the 32nd horizontal pixel, (0,0) (1,0) (2,0) (3,0) (4
The information of 0) (5,0) (6,0) (7,0) (8,0) can be commonly used. By utilizing this point, a one-dimensional vertical filter can be assembled with a register without using a line memory.

The structure of the two-dimensional filter using the line memory shown in FIG. 2 will be described again. In FIG. 2, there are 80 line memories, the output of which is subjected to coefficient calculation to form the vertical direction filter 590. Vertical filter 590
The signal after passing is input to the horizontal direction filter 7140 composed of 134 registers, and this is passed,
Realizes a vertical and horizontal two-dimensional filter.

The configuration to be realized in the embodiment of FIG. 3 is such that this vertical filter is composed of a register, like the horizontal filter. In the above example of the filter with 9 taps in the vertical direction, the filter operation can be made common in the unit of 32 pixels, but the memory cannot be replaced with the register only by that. It is assumed that 9 memories are replaced with 9 registers. The first tap in the register is populated with data. Considering from the beginning of the screen, first (0,0)
Is set. Since the video input repeats the horizontal operation, the next input signal is (0, 1), which is (0,
0) has the same contents. The next change in data is
It becomes the point of (0, 32). However, this is the operation itself of the horizontal filter.

The vertical filter must be sampled line by line. When sampling in line units, if this sampling point is placed at the horizontal edge, that is, at the 0th pixel, the input data is (0,0) (1,0) (2,0)
(3,0) (4,0) (5,0) (6,0) (7,0)
It becomes (8, 0), but only one line at this end is taken in. Only the ones in Fig. 3 are taken in, and ... are not taken in. This is not a two-dimensional filter.
Therefore, consider once storing the state of the filter in the buffer memory.

FIG. 4 shows the configuration of a vertical direction filter having the buffer interface shown in FIG. The filters are selectors 81, 82, 83, 84, 85, 86, 87, 88 and tap registers 91, 92, 93, 94, 95, 96,
97 and 98, and a filter coefficient calculation circuit 100. Here, the selector 81 and the tap register 91 configure the tap 0, the selector 82 and the tap register 92 configure the tap 1, and the selector 83 and the tap register 93.
The tap 2 is composed of the selector 84 and the tap register 9
4 configures tap 3; selector 85 and tap register 95 configure tap 4; selector 86 and tap register 96 configure tap 5; selector 87 and tap register 97 configure tap 6; The tap 7 is configured by the tap register 98.

Selectors 81, 82, 83, 84, 85,
86, 87 and 88 are used to switch between normal filter processing for shifting each tap data of the filter and fetching of the data of the buffer memory 5D-1 of FIG.

FIG. 5 shows the concept of the address space of the buffer memory 5D-1 shown in FIG. The buffer memory 5D-1 has a configuration capable of storing information of all the registers of the vertical direction filter 5D-2 in block units.

Next, the operation of FIG. 4 will be described. In FIG. 4, first, selectors 81, 82, 83, 84, 85, 8
6, 87, 88 are switched to the buffer memory 5D-1 side in FIG. 3, and data corresponding to the number of taps is loaded. This shows the information of the previous line. After loading, the selector 81 switches to the correction data storage memory 6 side, and the selectors 82, 83, 8
4, 85, 86, 87 and 88 are switched to the normal filter processing side for shifting each tap data. Then, although not shown, a one-shot filter sampling clock is input to the vertical filter to update the filter. After updating the filter,
The values of all the registers in the filter are stored again in the buffer memory 5D-1 of FIG. This operation is performed within a period of 32 pixels or less.

The above processing utilizes the property that information can be commonly used up to the horizontal 32nd pixel. That is, the load process for eight taps requires at least 8 cycles = 8 pixels. Further, in the process of inserting and updating the one-shot filter sampling clock in the vertical filter, one cycle = 1 pixel period is required. Further, after the filter is updated, the process of storing the values of all the registers in the filter into the buffer memory 5D-1 again requires a period of 8 cycles = 8 pixels.

Therefore, a total period of 17 pixels (= 8 + 1 + 8) is necessary, but this is sufficiently possible because it is within 32 pixels. If the block size is set to the number of vertical filter taps × 2 + 1 <the number of horizontal block pixels, the processing can be performed. If the data width of the buffering memory is wide, for example, if 1 tap data is 10 bits,
If the data width of the buffering memory is 40 bits,
The above process is 8/4 + 1 + 8/4 = 5, and
It can be easily understood that the condition of vertical filter tap number × 2 + 1 <horizontal block pixel number is not absolute.

When the processing of one horizontal block is completed,
At the next 32 pixel point, similarly, the state of the previous line is loaded from the corresponding buffer memory address.
That is, the state of the previous line is restored. After that, a one-shot filter sampling clock is input to the vertical filter to update the filter. After updating the filter, the values of all the registers in the filter are stored again in the buffer memory 5D-1.

By repeating this operation block by block, the output of the vertical filter 5D-2 shown in FIG. 3 is obtained. By using such a method, it becomes possible to configure a vertical one-dimensional filter with a register without using a line memory.

The vertical filter 5D-2 shown in FIGS. 3 and 4 will be described again with attention paid to the block rows in FIG. FIG. 6 shows an operation time chart of the vertical filter 5D-2 shown in FIG. First, since load processing is first performed from the buffer memory 5D-1, dust data enters the filter (X, X). Next, selectors 81 and 8
The state of 2, 83, 84, 85, 86, 87, 88 is switched to the normal filter processing operation, the data from the correction data storage memory 6 is taken in, and the filter operation is updated. The data of (0,0) is input to tap 2. This is once stored in addresses 0 to 7 of the buffer memory 5D-1.

After storing, the process moves to the adjacent block.
In other words, move to the block sequence. After the movement, the data of the block is loaded into the vertical direction filter 5D-2 from the addresses 8 to 15 of the buffer memory 5D-1. Similarly, in the first line, after (X, X) loading, the selectors 81,8
The states of 2, 83, 84, 85, 86, 87, 88 are switched, and the filter is updated once. The data of (0, 32) enters the tap 2. And that state as well
It is temporarily stored in addresses 8 to 15 of the buffer memory 5D-1. After storing, move to the processing of the next block.

The above operation is repeated to return to the head in the horizontal direction again. That is, the operation returns to the block sequence of.
First, data at addresses 0 to 7 in the buffer memory 5D-1 is loaded. In this content, tap 2 is (0, 0). Next, the data from the correction data storage memory 6 is fetched and the operation of the filter is updated.

Since the block width in the vertical direction is 4 lines, the same data is stored up to 4 lines, and (0, 0) data is similarly input to the tap 2. Further, the data (0, 0) of tap 2 is sent to tap 4. After that, this content is stored again in the addresses 0 to 7 of the buffer memory 5D-1. Address 0 is (0, 0) Address 1 is (0, 0
Data of 0) is entered.

When such an operation is repeated for 8 lines, tap 2 is (4, 0), tap 4 is (4, 0), tap 6 is (4, 0), tap 8 in the block row.
Is (4,0), tap 10 is (0,0), tap 1
The data of (0,0) is input to 2, the data of (0,0) is input to the tap 14, and the data of (0,0) is input to the tap 16.

In the block row, tap 2 is (4, 32), tap 4 is (4, 32), tap 6
Is (4, 32), tap 8 is (4, 32), tap 10 is (0, 32), tap 12 is (0, 32),
(0, 32) for tap 14 and (0, 3) for tap 16.
The data of 2) is entered. This is exactly the same operation as when the line memory is used.

The method of realizing the vertical filter using the buffer memory has been described above. In the next embodiment, a method for realizing the operation of the filter by reading the correction data storage memory and controlling writing to each tap of the vertical filter without using the buffer memory will be described.

FIG. 9 shows the configuration and the waveform including the two-dimensional filter by the correction data import control according to the embodiment of the present invention. 6 is a correction data storage memory, 5D-2 is a vertical filter, 5D-3 is a vertical filter, 10 is a memory read address control circuit, and the output of the memory read address control circuit 10 is connected to the correction data storage memory 6. It FIG. 10 shows a configuration including the vertical filter 5D-2 in FIG. FIG. 11 shows an operation time chart of the vertical filter of FIG.

The correction of the two-dimensional shading will be described below with reference to FIGS. As shown in the above embodiment, the vertical filter is updated line by line. It is also necessary to switch for each horizontal block. In order to update the filter without using the line memory, in the above-described embodiments, the method of temporarily storing the state of the filter in the buffer memory has been described. In the present embodiment, this is realized by reading control of the correction data storage memory and writing control to each tap of the vertical filter, instead of storing in the buffer memory.

FIG. 10 shows the vertical filter 5D-2 of FIG.
2 is shown, but the difference from FIG. 2 is that the taps are not connected. Originally, the tap of the filter is configured as a shift register, and the shift operation is performed one by one each time the update clock is input. In the present embodiment, the shift operation is performed by devising the reading of the memory instead of the filter and the writing control to each tap. FIG. 10 will be described again.

FIG. 10 shows tap 0 register 111, tap 1 register 112, tap 2 register 113,
Register 114 for tap 3 and register 11 for tap 4
5, tap 5 register 116, tap 6 register 1
17, a tap 7 register 118, a filter coefficient calculation circuit 120, a hold register 121, and each tap write control circuit 122. The output of the correction data storage memory 6 is connected to each tap in parallel.

Here, if the data of the block sequence shown in FIG. 3 of the above-mentioned embodiment is expressed again, as shown in FIG. 8, lines 1 to 4 = (0,0), lines 5 to 8
= (4,0), lines 9-12 = (8,0)
Becomes

For example, the state of each tap of the filter in line 8 is tap 0 = (4,0), tap 1 =
(4,0), tap 2 = (4,0), tap 3 = (4
0), tap 4 = (0,0), tap 5 = (0,0),
Tap 6 = (0,0) and tap 7 = (0,0).

Next, the state of each tap of the filter in the line 9 is as follows: tap 0 = (8,0), tap 1 = (4
0), tap 2 = (4,0), tap 3 = (4,0),
Tap 4 = (4,0), tap 5 = (0,0), tap 6 = (0,0), tap 7 = (0,0).

Furthermore, the state of each tap of the filter in line 10 is tap 0 = (8,0), tap 1 =
(8,0), tap 2 = (4,0), tap 3 = (4
0), tap 4 = (4,0), tap 5 = (4,0),
Tap 6 = (0,0) and tap 7 = (0,0).
This shift operation is performed by memory control.

The operation is shown in the time chart of FIG. Eight taps 0 to 7 during the block 0 period,
The phase data on the target line is read from the correction data storage memory 6 and written in order. After the writing for one block is completed, the filter output is held. By doing this, the output result of the vertical filter 5D-2 as shown in FIG. 9 is obtained at the output of the filter. That is, the same characteristics as when using the line memory can be obtained.

[0058]

According to the present invention, it is possible to obtain a shading correction method for removing a two-dimensional deviation.

[Brief description of drawings]

FIG. 1 is a diagram showing a configuration having a shading correction method according to an embodiment of the present invention.

FIG. 2 is a configuration diagram of a first embodiment of the two-dimensional digital filter circuit of FIG.

FIG. 3 is a second diagram including the two-dimensional digital filter circuit of FIG.
It is a figure which shows the structure and waveform of embodiment of FIG.

FIG. 4 is a diagram showing a configuration of a vertical filter of FIG.

5 is a diagram showing a concept of an address space of the buffer memory of FIG.

FIG. 6 is a diagram showing an operation time chart of the vertical direction filter of FIG.

FIG. 7 is a diagram showing pixel positions on a screen.

FIG. 8 is a diagram showing changes in block data at pixel positions on the screen.

9 is a third diagram including the two-dimensional digital filter circuit of FIG.
It is a figure which shows the structure and waveform of embodiment of FIG.

10 is a diagram showing a configuration including a vertical direction filter in FIG.

11 is a diagram showing an operation time chart of the vertical filter of FIG.

FIG. 12 is a diagram showing an example of a shading signal.

FIG. 13 is a diagram showing a conventional configuration and waveform.

14 is a diagram showing a smoothing filter characteristic of the filter circuit of FIG.

FIG. 15 is a diagram showing frequency components of a block signal.

[Explanation of symbols]

1 ... Optical system, 2 ... Image sensor, 3 ... Video signal processing circuit, 4
... Subtraction circuit, 5D ... Two-dimensional digital filter circuit, 6 ...
Memory, 51 to 580 ... Line memory, 581 ... Filter coefficient operation circuit, 590 ... Vertical filter, 71 to 7134 ... Register, 7135 ... Filter coefficient operation circuit, 7140 ... Horizontal filter, 5D-1 ... Buffer memory, 5D- 2 ... Vertical filter, 5D3 ... Horizontal filter, 81 to 88 ... Selector, 91 to 98 ...
Register, 100 ... Filter coefficient operation circuit, 10 ... Memory read address control circuit, 111 to 118 ... Tap register, 120 ... Filter coefficient operation circuit, 121 ... Hold register, 122 ... Each tap write control circuit.

   ─────────────────────────────────────────────────── ─── Continued front page    F-term (reference) 5B047 AB02 BB06 DA04 DA06                 5C021 PA34 PA36 PA39 PA40 PA78                       PA80 PA82 PA87 RB00 XA67                 5C022 AB51 AC69                 5C072 AA01 BA08 EA05 FB12 UA11                       VA10                 5C077 MM03 PP02 PP06 PQ21 PQ22                       TT09

Claims (3)

[Claims] 1. A shading correction method for correcting aberration of a picked-up image in real time, comprising two-dimensional correction block data in horizontal and vertical directions, and the two-dimensional correction block data is processed by a two-dimensional filter. A shading correction method characterized by performing a smoothing process and performing a shading correction of a video signal using the processed signal. 2. The shading correction method according to claim 1, wherein the two-dimensional filter comprises a vertical direction filter and a horizontal direction filter, and a buffer memory for storing a filter state of the vertical direction filter is provided. A characteristic shading correction method. 3. The shading correction method according to claim 1, wherein the two-dimensional filter is composed of a vertical direction filter and a horizontal direction filter, and the vertical direction filter is provided for each line by the number of taps of the vertical direction filter,
A shading correction method, wherein a state of a result obtained by updating the vertical direction filter is written in each tap.