JP2007025902A - Image processing apparatus and image processing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a technique for detecting a predetermined subject in an image more simply and with high accuracy without requiring an empirical threshold.
SOLUTION: A face probability within a rectangle of a predetermined size arranged at each position on a luminance image is obtained (S106), a probability distribution of each area is obtained based on the face probability of each area, and each probability distribution is obtained. (S109), the second reduced image to the Nth reduced image of the luminance image are generated (S103), and the face of the region in the rectangle arranged at each position on the nth reduced image is generated. A process of obtaining a probability (S106), obtaining a probability distribution of each area based on the face probability of each area, and combining the n-th map data indicating each probability distribution with the (n-1) -th map data. By repeating for n = 2 to N, composite map data is created, and a representative pattern is selected using the composite map data (S115).
[Selection] Figure 2

Description

  The present invention relates to a technique for detecting a predetermined subject in an image.

  An image processing method for automatically detecting a specific subject pattern from an image is very useful, and such an image processing method can be used to detect a human face, for example. Such methods can be used in many areas such as teleconferencing, man-machine interfaces, security, monitor systems for tracking human faces, image compression, and the like. As a technique for detecting a face from such an image, for example, Non-Patent Document 1 discloses various methods. Among them, use some prominent features (two eyes, mouth, nose, etc.) and the unique geometric positional relationship between those features, or symmetric features of human face, human face color A method for detecting a human face by using features, template matching, neural network, etc. is shown.

  For example, the method proposed in Non-Patent Document 2 is a method of detecting a face pattern in an image using a neural network. The face detection method according to Non-Patent Document 2 will be briefly described below.

  First, an image including a face is read into a memory, and a predetermined area to be compared with the face is cut out from the image. Then, a pixel value distribution of each pixel constituting the cut-out area is used as an input to obtain one output by calculation using a neural network.

  At this time, the weights and threshold values of the neural network are learned in advance using a large number of face image patterns and non-face image patterns. If such a neural network is used, for example, if the output of the neural network is 0 or more. It can be determined that the face is non-face.

  Then, the face is detected from the image by scanning the cutout position of the image pattern to be collated with the face which is an input of the neural network, for example, in the vertical and horizontal directions from the entire image.

  Further, in order to cope with detection of faces of various sizes, the read images are sequentially reduced at a predetermined rate, and the above-described face detection scanning is performed on each of the images.

  When a face is detected by the above-described method and a pattern determined to be a face is output, a situation in which patterns are detected by overlapping with adjacent patterns or patterns slightly different in size frequently occurs. In such a case, Non-Patent Document 2 combines a plurality of empirical algorithms, such as a process of narrowing down the size and number of appearances of patterns using a predetermined threshold, and using the positional relationship between pattern overlap and the center of gravity. The correct face is narrowed down.

Thus, the conventional example has a complicated algorithm, and a plurality of parameters need to be set empirically. Therefore, there is a problem that a pattern indicating a face to be detected is missed (undetected) or a pattern other than a face is left (false detection).
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL.24, NO.1, JANUARY 2002, “Detecting Faces in Images: A Survey” IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL.20, NO.1, JANUARY 1998, “Neural network-based face detection”

  The present invention has been made in view of the above problems, and an object thereof is to provide a technique for detecting a predetermined subject in an image more easily and with high accuracy without requiring an empirical threshold. To do.

  In order to achieve the object of the present invention, for example, an image processing apparatus of the present invention comprises the following arrangement.

That is, acquisition means for acquiring an image including a predetermined subject;
Generating means for generating a luminance image composed of luminance components of the image;
First probability calculating means for obtaining a probability that an area in the rectangle indicates a predetermined pattern with the predetermined subject when a rectangle of a predetermined size is arranged at each position on the luminance image;
First probability distribution calculating means for obtaining a probability distribution for each region based on the probability for each region calculated by the first probability calculating unit;
First creation means for creating first map data indicating a probability distribution for each of the regions on the luminance image;
Reduction means for generating a second reduced image to an Nth reduced image by recursively reducing the luminance image;
A second probability calculating means for obtaining a probability that an area in the rectangle indicates the predetermined subject and an oversight pattern when the rectangle is arranged at each position on an nth (2 ≦ n ≦ N) reduced image;
Second probability distribution calculating means for obtaining a probability distribution for each region based on the probability for each region calculated by the second probability calculating means;
Synthesizing means for synthesizing the nth map data indicating the probability distribution for the respective regions on the nth reduced image with the (n-1) th map data;
repetitive means for creating composite map data by repeating the processing by the second probability calculation means, the second probability distribution calculation means, and the synthesis means for n = 2 to N;
And detecting means for detecting a representative pattern from subject pattern candidates based on the composite map data.

  In order to achieve the object of the present invention, for example, an image processing method of the present invention comprises the following arrangement.

That is, an acquisition step of acquiring an image including a predetermined subject;
A generation step of generating a luminance image composed of luminance components of the image;
A first probability calculating step of obtaining a probability that an area in the rectangle indicates the predetermined subject and an opening pattern when a rectangle of a predetermined size is arranged at each position on the luminance image;
A first probability distribution calculation step for obtaining a probability distribution for each region based on the probability for each region calculated in the first probability calculation step;
A first creation step of creating first map data indicating a probability distribution for each of the regions on the luminance image;
A reduction process for generating a second reduced image to an Nth reduced image by recursively reducing the luminance image;
A second probability calculating step of obtaining a probability that an area in the rectangle shows an obtuse pattern with the predetermined subject when the rectangle is arranged at each position on an nth (2 ≦ n ≦ N) reduced image;
A second probability distribution calculating step for obtaining a probability distribution for each region based on the probability for each region calculated in the second probability calculating step;
A synthesis step of synthesizing n-th map data indicating a probability distribution for the respective regions on the n-th reduced image with (n-1) -th map data;
a repetition step of creating composite map data by repeating the processes of the second probability calculation step, the second probability distribution calculation step, and the combination step for n = 2 to N;
And a detecting step of detecting a representative pattern from the subject pattern candidates based on the composite map data.

  With the configuration of the present invention, it is possible to detect a predetermined subject in an image more simply and with high accuracy without requiring an empirical threshold.

  Hereinafter, the present invention will be described in detail according to preferred embodiments with reference to the accompanying drawings.

[First Embodiment]
The image processing apparatus according to the present embodiment is configured by a computer such as a PC (personal computer) or WS (workstation), and an external device via an image input from an imaging apparatus such as a digital camera or a network such as the Internet. A predetermined subject included in an image input in various input forms such as an image downloaded from, an image input by reading from a storage medium such as a CD-ROM or a DVD-ROM is detected. In this embodiment, a human face is used as a subject, but other subjects may be used.

  First, an image processing apparatus according to the present embodiment that performs such processing will be described. FIG. 3 is a diagram illustrating a hardware configuration of a computer applicable to the image processing apparatus according to the present embodiment.

  A CPU 201 controls the entire computer using programs and data stored in the RAM 202 and the ROM 203, and executes each process described later performed by the computer.

  Reference numeral 202 denotes a RAM, an area for temporarily storing programs and data read from the external storage device 207 and the storage medium drive device 208, and for temporarily storing data received from the outside via the I / F 209. Various areas such as an area and a work area used by the CPU 201 to execute various processes can be provided as appropriate.

  A ROM 203 stores a boot program, setting data of the computer, and the like.

  Reference numerals 204 and 205 denote a keyboard and a mouse, respectively, and various instructions can be input to the CPU 201 when operated by a computer operator.

  A display unit 206 includes a CRT, a liquid crystal screen, and the like, and displays the processing result by the CPU 201 using characters, images, and the like.

  Reference numeral 207 denotes an external storage device, which is a large-capacity information storage device such as a hard disk drive device, and stores an OS (Operating System) and programs and data for causing the CPU 201 to execute each process described later performed by the computer. These are read out to the RAM 202 as appropriate under the control of the CPU 201.

  A storage medium drive device 208 reads out programs and data recorded on a storage medium such as a CD-ROM or DVD-ROM, and outputs them to the RAM 202, the external storage device 207, or the like. A part of the program or data stored in the external storage device 207 may be recorded on the storage medium. In that case, when using the stored program or data, The storage medium drive device 208 reads out programs and data recorded on the storage medium and outputs them to the RAM 202.

  Reference numeral 209 denotes an I / F (interface), to which a digital camera, the Internet, a LAN network line, or the like can be connected.

  A bus 210 connects the above-described units.

  In addition, the input form of the image to the computer is not particularly limited, and various forms are conceivable.

  FIG. 1 is a block diagram showing a functional configuration of a computer applicable to the image processing apparatus according to the present embodiment. As shown in the figure, the image processing apparatus according to the present embodiment includes an image input unit 10, an image memory 20, an image reduction unit 30, a collation pattern extraction unit 40, a luminance normalization unit 50, a face discrimination unit 60, a face candidate list. The storage unit 70, the face probability distribution generation unit 80, the face probability map storage unit 90, the representative pattern detection unit 100, and the face area output unit 101 are configured.

  The image input unit 10 receives image data output from a device such as a digital still camera or a film scanner, and outputs the image data to the subsequent image memory 20. As described above, the image input form is not particularly limited.

  The image memory 20 is a memory for storing image data output from the image input unit 10.

  The image reduction unit 30 first generates a luminance image composed of luminance components of image data received from the image memory 20. Then, a plurality of reduced images are generated by recursively reducing the generated luminance image. Each generated reduced image (if the original luminance image generated based on the image data received from the image memory 20 is also interpreted as a 1/1 reduced image, this original can also be included in the reduced image). The data is sequentially output to the subsequent matching pattern extraction unit 40.

  When the collation pattern extraction unit 40 receives the reduced image from the image reduction unit 30, the pixel group included in the rectangle is sequentially extracted as a “collation target pattern” while moving a rectangle of a predetermined size on the reduced image, This is output to the luminance normalization unit 50 in the subsequent stage. Such processing is performed for each reduced image received from the image reduction unit 30.

  The luminance normalization unit 50 normalizes the luminance distribution of the pixel group constituting the pattern to be collated received from the collation pattern extraction unit 40.

  The face discriminating unit 60 obtains the probability that the collation target pattern normalized by the luminance normalization unit 50 is a face pattern. If the probability that the collation target pattern is a face pattern is not 0, the image input unit 10 The position of the pattern to be collated on the image input to (the position on the image input to the image input unit 10 so that the positional relationship between the reduced image from which the pattern to be collated is extracted and the pattern to be collated is maintained. (Position) and size are registered in the face candidate list storage unit 70, and data indicating the probabilities obtained for the matching target pattern is output to the face probability distribution generation unit 80 in the subsequent stage.

  The face candidate list storage unit 70 stores a set of positions and sizes of the face pattern and the target pattern to be compared on the image input to the image input unit 10. This set of data is sequentially registered in the list data. Therefore, this list data consists of one or more sets of data.

  The face probability distribution generation unit 80 obtains a probability distribution for the verification target pattern, and updates the map data stored in the probability map storage unit 90 using the obtained probability distribution data.

  The probability map storage unit 90 stores map data. The map data will be described later.

  The representative pattern detection unit 100 uses the map data stored in the face probability map storage unit 90 and the list data registered in the face candidate list storage unit 70 on the image input to the image input unit 10. A process for detecting a representative face pattern (hereinafter referred to as a representative pattern) is performed. The representative pattern will be described later.

  The face area output unit 101 outputs the representative pattern detected by the representative pattern detection unit 100.

  Each of the above units can be configured with, for example, the CPU 201, the RAM 202, the external storage device 207, and the like.

  Next, a process performed by the CPU 201 operating as each unit illustrated in FIG. 1, that is, a process for detecting a subject included in an image will be described below with reference to FIG. 2 showing a flowchart of the process. explain. Note that a program and data for causing the CPU 201 to execute the processing according to the flowchart of FIG. 10 are stored in the external storage device 207 (or a storage medium readable by the storage medium drive device 208). Accordingly, the computer executes each process described below by loading the data into the RAM 202 as needed and the CPU 201 executing the process using this.

  When image data is input from the outside via the external storage device 207 or the I / F 209, the CPU 201 temporarily stores it in an area corresponding to the image memory 20 in the RAM 202 (step S101). If an image input to the computer is compressed, the image is decompressed and temporarily stored in the RAM 202.

  In the present embodiment, it is assumed that each pixel constituting the input image data is represented by R, G, and B. Accordingly, the CPU 201 determines, based on the image data stored in the RAM 202 in step S101, an image (luminance image) composed of the luminance components of this image, that is, the value of each pixel constituting this image as the luminance value of this pixel. The image converted into is generated (step S102). However, if each pixel constituting the image data stored in the RAM 202 in step S101 is expressed by YCrCb, a luminance image is generated using only the Y component in step S102.

  Next, the CPU 201 generates a plurality of reduced images by recursively reducing the generated luminance image (step S103). For example, a reduced image 2 obtained by multiplying the vertical and horizontal sizes of the luminance image generated in step S102 (hereinafter referred to as reduced image 1) by 1 / 1.2 is generated, and then the vertical and horizontal sizes of the reduced image 2 are generated. A plurality of reduced images are generated, such as generating a reduced image 3 obtained by multiplying 1 / 1.2. This is because when detecting a face in the subsequent processing, detection is sequentially performed on image data of a plurality of sizes in order to support detection of faces of various sizes. The number of reduced images to be generated is not particularly limited.

  In step S104 and subsequent steps, each generated reduced image is processed. That is, the processing after step S104 is repeatedly performed for the number of generated reduced images.

  In the following description, it is assumed that the generated reduced images are referred to as reduced image 1, reduced image 2,... Reduced image N in descending order of size. Note that the order of selection as processing targets is not particularly limited.

  First, the CPU 201 arranges a rectangle of a predetermined size on the reduced image 1, and extracts a pixel group in the rectangle as a verification target pattern (step S104). When this rectangle is arranged at each position on the reduced image 1, it is for obtaining the luminance distribution within the rectangle at each position. For example, this rectangle is initially arranged at the upper left corner of the image.

  Next, a process of normalizing the luminance distribution of each pixel in the verification target pattern extracted in step S104 is performed (step S105). For example, brightness correction such as histogram smoothing is performed. This is for suppressing deterioration in accuracy of subject collation because the luminance distribution of the subject pattern to be captured changes depending on the illumination condition.

  Next, a process of obtaining a probability (face probability) that the collation target pattern (luminance pattern) whose luminance distribution has been normalized in step S105 is a face pattern (face and face pattern) is performed (step S106).

  FIG. 5 is a diagram showing the operation of the neural network for identifying the pattern in the predetermined area. In the figure, R represents an area to be identified on the image, for example. In this embodiment, as shown in the figure, this area R is further divided into three areas by each of the neurons (indicated by N). ) As a receptive field. Then, the luminance distribution of the divided area is input to each neuron, and an output in the intermediate layer is obtained. Then, the final output is obtained by using the output of each neuron as the input of the neuron in the output layer.

  Here, in each neuron, the product-sum operation of the weight and the luminance distribution obtained by learning in advance and the operation by the sigmoid function as a result are performed. In this embodiment, the output value of the neuron in the output layer is used as the face probability (for details of the neural network and the learning method, see Non-Patent Document 2 above). Note that the processing for obtaining the probability (face probability) that the pattern to be verified whose luminance distribution is normalized in step S105 is a face pattern is not limited to this, and for example, Proceedings of the IEEE Conference on Computer Vision and Pattern The AdaBoost method proposed in a report by Viola and Jones entitled “Rapid Object Detection using a Boosted Cascade of Simple Features” in Recognition, 2001 may be used.

  FIG. 4 is a diagram for explaining processing for detecting a face pattern for reduced images of various sizes (in the case of the present embodiment, reduced image 1, reduced image 2,... Reduced image N). . When a rectangle of the same size is arranged at each position on each reduced image, in order to determine whether or not the area within the rectangle at each position is a face pattern, first, as shown on the left side of FIG. Then, a rectangle is arranged at the upper left corner of the reduced image, and the position of the rectangle is moved from top to bottom from there to the right side. Each time it is moved, the pixel group in the rectangle is used as a collation pattern for discrimination of the face pattern.

  Returning to FIG. 2, next, the probability that the pattern to be collated (luminance pattern) whose luminance distribution has been normalized in step S105 is a face pattern (face and face pattern) is greater than 0. If it is determined that the pattern is a face pattern, the process proceeds to step S108 via step S107, and the size of the pattern to be verified and the position of the pattern to be verified on the image acquired in step S101 are set. It is stored (registered) in the face candidate list storage unit 70 provided in the RAM 202 or in the external storage device 207 (step S108).

  Next, based on the face probability obtained in step S106 for the collation target pattern extracted in step S104, a probability distribution (face probability distribution) for this collation target pattern is obtained and stored in the RAM 202 or the external storage device 207. In the map data stored in the provided face probability map storage unit 90, a process of updating the map data is performed in order to add the obtained face probability distribution to the data portion corresponding to the matching target pattern (step S109). ).

  Here, the process in step S109 will be described in more detail.

  First, when the center position of the pattern to be collated extracted in step S104 is set as the origin, a probability distribution is obtained such that the peak value is at this origin and the probability value decreases as the distance from the origin increases. For example, a two-dimensional Gaussian function having the peak value at the center position of the verification target pattern as the face probability and the diagonal length of the verification target pattern as an extension is obtained. Then, using the obtained two-dimensional Gaussian function, a probability value corresponding to each pixel position in the verification target pattern is obtained. Thereby, the probability value corresponding to each pixel constituting the verification target pattern can be obtained.

  Here, the map data is data of an image (map image) having a predetermined size, and the pixel value of each pixel constituting the map image is initialized to 0.

  Therefore, in step S109, the pixel value of each pixel constituting the region (second region) on the map image corresponding to the region (first region) from which the pattern to be collated is extracted on the reduced image 1 is calculated. The map data is updated by performing a process of adding the probability values of the pixels corresponding in position among the pixels constituting the first region.

  Here, if the map image and the reduced image 1 have the same size, the first area and the second area have the same size. In this case, the probability value of the pixels constituting the first area is set to S (x , Y) {x, y are coordinate values on the reduced image 1}, the probability value S (x, y) with respect to the probability value M (x, y) of the pixel located at the coordinate (x, y) in the map image The map data is updated by adding y).

  Next, the process proceeds to step S110, and it is checked whether there is a rectangular movement destination on the reduced image 1 (step S110). That is, when the position of the rectangle on the reduced image 1 is moved and the process of extracting the portion (pixel group) in the rectangle at the next position as the verification target pattern is performed, if there is no destination, for example, the current rectangle If the position is already at the lower right corner of the reduced image 1, the rectangle cannot be moved anymore. On the other hand, if the position of the current rectangle is not already the position of the lower right corner of the reduced image 1, the rectangle can be moved.

  Accordingly, if there is a destination, the process proceeds from step S110 to step S111, and the position of the rectangle on the reduced image 1 is moved (step S111). When the movement of the rectangle is completed, the process proceeds to step S104, the pattern to be collated in the moved rectangle is extracted, and the subsequent processes are performed.

  On the other hand, if there is no rectangular movement destination, the process proceeds to step S112 to determine whether or not the above processing has been performed for all the reduced images (step S112), and there is a reduced image that has not yet been processed. In this case, the process proceeds to step S113, the position of the rectangle to be arranged on the reduced image is initialized (for example, returned to the position of the upper left corner of the reduced image) (step S113), and the processing after step S104 is performed for the next reduced image. I do.

  In the present embodiment, since processing is currently performed on the reduced image 1, processing is performed on the reduced image 2 next. Therefore, in this case, a rectangle is arranged at the position of the upper left corner on the reduced image 2 (step S113), and the processing after step S104 is performed on the reduced image 2.

  Therefore, by performing the processing from step S104 on on the reduced image n (n ≧ 2), an area (m-th area) on the map image corresponding to the area (k-th area) from which the pattern to be collated is extracted on the reduced image n. Updating the map data by performing a process of adding the probability value of the pixel corresponding to the position among the pixels constituting the kth region to the probability value of each pixel constituting the region become.

  According to this, since a pattern is detected by overlapping adjacent patterns or patterns with slightly different sizes that occur in an area that is originally a face, probability values are sequentially added. A relatively high probability is obtained in the data. In addition, when a pattern that is not originally a face happens to be identified as a face, it may be isolated and a high probability may be output, but since a high probability value is not added, the map after the matching pattern scan There is a relatively low probability in the data.

  Note that the above probability distribution does not need to be calculated for all the patterns to be collated. For example, the probability distribution is generated only for a collation pattern that outputs a face probability of 0 or more (identified as a face pattern).

  FIG. 6 is a diagram showing a state of probability distribution generation. A represents a pattern discriminated as a face pattern, and B represents a probability distribution generated from one pattern and represents a higher probability value as it becomes darker. C represents map data obtained by adding the probability distributions.

  Note that the method of generating the face probability map is not limited to this. For example, as shown in FIG. 13, the weight of the pixel inside the rectangle L indicated by the face pattern to be collated is set to 1, and the other portions are set to 0. There is a method of adding a value to the corresponding face probability map pixel. The numbers in the face probability map B in FIG. 13 represent weights. Even when this method is used, if a pattern is detected by overlapping adjacent patterns or patterns with slightly different sizes that occur in an area that is originally a face, a probability value as shown in FIG. Are sequentially added, and the position of the representative face pattern in the image A has a relatively high probability in the face probability map B. Therefore, by designating a region where the map value is maximum or a region above a certain threshold value It is possible to detect a representative pattern. For example, in FIG. 15, when a region Q having the maximum map value is taken, a rectangular region indicated by P in the input image is obtained.

  Returning to FIG. 2, if the above processing is performed for all the reduced images, the processing proceeds to step S <b> 114.

  In step S114, the probability value of each pixel constituting the map image indicated by the map data is normalized to 0 to 1 (step S114). For example, the maximum value and the minimum value are obtained by scanning the probability values of all the pixels constituting the map image, and normalized by linear transformation so that the maximum value is 1 and the minimum value is 0.

  Then, using the map data generated in this way, a process of detecting a representative pattern from the image acquired in step S101 is performed (step S115).

  FIG. 7 is a flowchart showing details of the processing in step S115.

  First, the list data held in the face candidate list storage unit 70 provided in the RAM 202 or in the external storage device 207 is read (step S301). Next, referring to the probability value of each pixel constituting the map image indicated by the map data, a process of specifying the pixel position having the largest probability value is performed (step S302). FIG. 8B shows an example of a map image. In FIG. 8A, A indicates the position of a pixel having the maximum probability value. It is assumed that this map image is obtained when the image 801 shown in FIG. 8A is input in step S101.

  Next, in step S303, the following processing is performed. First, each position registered in the list data (the position of the face pattern and the target pattern to be compared in the image acquired in step S101) is acquired. Then, of the acquired positions, the one closest to the position specified in step S302 is selected. The verification target pattern at the selected position is the representative pattern.

  The process in step S303 will be described with reference to FIG. In FIG. 8A, let B be one of the positions registered in the list data. Further, an area having “the size registered in the list data as a set with the position B” (that is, the pattern to be collated at the position B) with the position B as the center is P. Here, among the positions registered in the list data, when this position B is closest to the position A shown in FIG. 8B (that is, when the image 801 and the image 802 are overlapped with the same size) If the position B is closest to the position A), in step S303, the verification target pattern P is selected as a representative pattern.

  Returning to FIG. 7, next, a process of updating the map data is performed (step S304). For example, in the case of FIG. 8B, in the map data, data corresponding to the region corresponding to the pattern to be verified P on the map image 802 is set to zero. FIG. 9 is a diagram illustrating an example of a map image after the processing in step S304 is performed on the map image 802.

  Next, when the number of selected representative patterns reaches a predetermined number, or when there is no pixel having a probability value greater than or equal to the predetermined value with reference to the updated map data in step S304, the present processing is performed. If the number of selected representative patterns has not reached the predetermined number and the map data after the update in step S304 is referred to and there is a pixel having a probability value greater than or equal to the predetermined value, the process is performed. The process proceeds to step S302 via step S305, and the subsequent processing is repeated.

  When the second and subsequent representative patterns are selected by such processing, a region corresponding to the previously selected matching target pattern is avoided in the map image, and the pixel position having the second largest probability value is accelerated. Can be searched.

  Note that the determination conditions used in the determination process in step S305 are not limited to this, and the number of representative patterns to be selected is not particularly limited. In step S305, the map data updated in step S304 is referred to, and the predetermined pattern is determined. If there is no pixel having a probability value equal to or greater than the value, the present process is terminated, and the updated map data in step S304 is referred to. If a pixel having a probability value equal to or greater than the predetermined value exists, the process is performed. You may make it progress to S302.

  Since the representative pattern can be selected as described above, returning to FIG. 2, next, image data of this representative pattern is output (step S116). Note that the representative pattern to be output may be the one before correction in step S105 or the one after correction.

  The output destination is not particularly limited, but may be a predetermined area in the RAM 202 or an external device capable of data communication via the external storage device 207 or the I / F 209. .

  Further, since the updated map image is as shown in FIG. 9, the comparison target pattern selected as the representative pattern in the processing in step S303 is compared with the verification target pattern detected as the representative pattern before, If the overlap is greater than or equal to a predetermined area, a process of not outputting as a representative pattern can be considered, which further increases the accuracy. At this time, the update process is performed in step S304 on the area once detected as the representative pattern.

  In step S115 described above, the method of using the distance between the center of the pattern to be verified and the pixel having the maximum probability value has been described. However, the method of determining the representative pattern is not limited to this. For example, the representative pattern may be a rectangular pattern centered on a pixel having the maximum probability value on the map image and inscribed in an area having a value equal to or greater than a predetermined threshold.

  That is, in the map image 802 as shown in FIG. 8B, when the pixel at the position A is the pixel position having the maximum probability value, as shown in FIG. 10, the map image 802 passes through the position A and x The probability distribution in the plane H parallel to the axis is as shown in the graph in the lower part of the figure. At this time, in the figure, the length of the side parallel to the x axis of the rectangular pattern centered on the position of the pixel having the maximum probability on the map image 802 and inscribed in the region having the probability value equal to or greater than the predetermined threshold th is m.

  Similarly, the probability distribution in the plane I passing through the position A and parallel to the y axis is as shown in the graph of FIG. At this time, in the figure, the length of the side parallel to the y-axis of the rectangular pattern inscribed in the region having the probability value equal to or greater than the predetermined threshold th is n.

  In this manner, the length of each side of the rectangular pattern inscribed in an area having a probability value equal to or greater than a predetermined threshold centered on the pixel position having the maximum probability on the map image 802 is determined, as shown in FIG. Then, an area on the input image (image acquired in step S101) corresponding to the rectangle R shown on the map image 802 as the representative pattern P is detected.

  Further, the pattern extracted here is not limited to the rectangular pattern, and may be a circular pattern centered on the pixel position A having the maximum probability.

[Other Embodiments]
Also, an object of the present invention is to supply a recording medium (or storage medium) in which a program code of software that realizes the functions of the above-described embodiments is recorded to a system or apparatus, and the computer (or CPU or Needless to say, this can also be achieved when the MPU) reads and executes the program code stored in the recording medium. In this case, the program code itself read from the recording medium realizes the functions of the above-described embodiment, and the recording medium on which the program code is recorded constitutes the present invention.

  Further, by executing the program code read by the computer, not only the functions of the above-described embodiments are realized, but also an operating system (OS) running on the computer based on the instruction of the program code. It goes without saying that a case where the function of the above-described embodiment is realized by performing part or all of the actual processing and the processing is included.

  Furthermore, after the program code read from the recording medium is written into a memory provided in a function expansion card inserted into the computer or a function expansion unit connected to the computer, the function is based on the instruction of the program code. It goes without saying that the CPU or the like provided in the expansion card or the function expansion unit performs part or all of the actual processing and the functions of the above-described embodiments are realized by the processing.

  When the present invention is applied to the recording medium, program code corresponding to the flowchart described above is stored in the recording medium.

1 is a block diagram showing a functional configuration of a computer applicable to an image processing apparatus according to a first embodiment of the present invention. It is a flowchart of the process for detecting the to-be-photographed object contained in the image. FIG. 2 is a diagram illustrating a hardware configuration of a computer applicable to the image processing apparatus according to the first embodiment of the present invention. It is a figure explaining the process which discriminate | determines whether the collation target pattern is a face pattern about the reduced image of various sizes. It is the figure shown about the operation | movement of the neural network for identifying the pattern in a predetermined area | region. It is a figure which shows the mode of probability distribution generation. It is a flowchart which shows the detail of the process in step S115. It is a figure which shows an example of the image acquired by step S101, and a map image. It is a figure which shows the example of the map image after performing the process in step S304 with respect to the map image 802. FIG. In the map image 802 as shown in FIG. 8B, when the pixel at the position A is the pixel position having the maximum probability value, the map image 802 passes through the position A in the plane H parallel to the x-axis. It is a figure which shows probability distribution. It is a figure which shows the probability distribution in the surface I which passes the position A and is parallel to a y-axis. It is a figure which shows the area | region on the input image corresponding to the rectangle R shown on the map image. It is a figure explaining the production | generation method of a face probability map. It is a figure explaining the production | generation method of a face probability map. It is a figure explaining the production | generation method of a face probability map.

Claims (6)

Acquisition means for acquiring an image including a predetermined subject;
Generating means for generating a luminance image composed of luminance components of the image;
First probability calculating means for obtaining a probability that an area in the rectangle indicates a predetermined pattern with the predetermined subject when a rectangle of a predetermined size is arranged at each position on the luminance image;
First probability distribution calculating means for obtaining a probability distribution for each region based on the probability for each region calculated by the first probability calculating unit;
First creation means for creating first map data indicating a probability distribution for each of the regions on the luminance image;
Reduction means for generating a second reduced image to an Nth reduced image by recursively reducing the luminance image;
A second probability calculating means for obtaining a probability that an area in the rectangle indicates the predetermined subject and an oversight pattern when the rectangle is arranged at each position on an nth (2 ≦ n ≦ N) reduced image;
Second probability distribution calculating means for obtaining a probability distribution for each region based on the probability for each region calculated by the second probability calculating means;
Synthesizing means for synthesizing the nth map data indicating the probability distribution for the respective regions on the nth reduced image with the (n-1) th map data;
repetitive means for creating composite map data by repeating the processing by the second probability calculation means, the second probability distribution calculation means, and the synthesis means for n = 2 to N;
An image processing apparatus comprising: detecting means for detecting a representative pattern from subject pattern candidates based on the composite map data. The detection means includes
Among the regions on the image acquired by the acquisition unit corresponding to the region having the probability calculated by the first probability calculation unit greater than 0, the pixel position having the largest value on the image indicated by the composite map data is displayed. First means for outputting a region at the closest position as the region of the predetermined subject;
In the composite map data, the composite map data is updated by setting the data corresponding to the region corresponding to the region detected by the first means on the image indicated by the composite map data to 0. Means,
The image processing apparatus according to claim 1, further comprising: a third unit that repeats the processing by the first and second units for the composite map data updated by the second unit.   3. The method according to claim 2, wherein the third means repeats the processing by the first and second means if the largest value on the image indicated by the composite map data is equal to or greater than a predetermined threshold value. Image processing device. An acquisition step of acquiring an image including a predetermined subject;
A generation step of generating a luminance image composed of luminance components of the image;
A first probability calculating step of obtaining a probability that an area in the rectangle indicates the predetermined subject and an opening pattern when a rectangle of a predetermined size is arranged at each position on the luminance image;
A first probability distribution calculation step for obtaining a probability distribution for each region based on the probability for each region calculated in the first probability calculation step;
A first creation step of creating first map data indicating a probability distribution for each of the regions on the luminance image;
A reduction process for generating a second reduced image to an Nth reduced image by recursively reducing the luminance image;
A second probability calculating step of obtaining a probability that an area in the rectangle shows an obtuse pattern with the predetermined subject when the rectangle is arranged at each position on an nth (2 ≦ n ≦ N) reduced image;
A second probability distribution calculating step for obtaining a probability distribution for each region based on the probability for each region calculated in the second probability calculating step;
A synthesis step of synthesizing n-th map data indicating a probability distribution for the respective regions on the n-th reduced image with (n-1) -th map data;
a repetition step of creating composite map data by repeating the processes of the second probability calculation step, the second probability distribution calculation step, and the combination step for n = 2 to N;
And a detection step of detecting a representative pattern from the subject pattern candidates based on the composite map data.   A program causing a computer to execute the image processing method according to claim 4.   A computer-readable storage medium storing the program according to claim 5.

Images