KR20100001684A - System and method for finding a family using face image analysis - Google Patents

Abstract

The present invention discloses a family retrieval system and method for finding a family of a separated mother or adopted child using facial image analysis. The family search system of the present invention includes a first range image of a search subject photographed at a first age and a second face image and second age of the search subject family photographed at the same or similar age as the first age. An input unit configured to receive a third face image of a family photographed at an age within the family; An image analyzer configured to calculate a similarity value between the second face image and the first face image; A virtual face generator configured to generate a virtual face image of the search subject of the second age using the calculated similarity value; And a search unit that provides information about a family using the generated virtual face image. According to the present invention, by searching for a family by using the face image of the separated family, it is possible to easily provide a family search result based on the similarity of faces between families. In addition, the accuracy of face prediction can be improved by utilizing the similarity result between the past Mia and the family face image for morphing the past Mia's face.

Description

System and method for finding a family using face image analysis}

The present invention relates to a family retrieval system and method using facial image analysis, and more particularly, to find a lost infant (adopted child) or the lost infant (adopted child) by using the similarity between faces of the family. The present invention relates to a family search system and method for finding a separated family.

Existing family finder sites posted stories about finding separated families on bulletin boards, responded to posts, or contacted by email. In some cases, a function to register a photo may be provided. In addition, some sites provide text-based keyword search functions such as full name, nickname, situation at the time of separation, family name to find, and year of birth to make it easier to find relevant information about the family. However, if the lost child (adopted child) can not remember his information, it is difficult to find it in this way.

In the case of a family search TV program, a story is requested by telephone, Internet bulletin board, or a letter, and the program crew selects and broadcasts the story. In this case, there is an advantage in that it is possible to see the actual figure of people who are looking for a family, but it is limited in that it should be viewed at a predetermined broadcast time and only information about the selected people is disclosed.

As can be seen from the general family, the faces of parents or siblings have a lot in common. Mia and adoptive children may not be able to remember information about the situation or family at that time, but they can see that the similarity of their faces with their parents, siblings is maintained even when they grow up in different circumstances. However, no service has been disclosed to efficiently find a large number of lost children (adopted children) with only the minimum information known by the lost children, such as gender, age, past or present.

Korean Patent Laid-Open Publication No. 10-2003-85443 discloses a system for providing separated family information through a complex analysis process of video interview content, virtual face conversion prediction program, DNA test, etc. of family seekers. However, the patent does not disclose specific methods for predicting the current face of a separated family.

In view of the above-described problems of the prior art, the present invention provides an infant's face image at the time of an infant's (adopted infant) occurrence, a child's childhood facial image, the current age of the infant (adopted baby), and the current age of the infant (adopted baby) A family search system and method for generating a virtual face image of a current lost child (adopted child) using a face image of a family corresponding to and searching for a family or a lost infant using the generated virtual face image It aims to provide.

The family search system of the present invention for achieving the above object of the present invention includes a first face image of a search subject photographed at a first age, and a second search result family photographed at an age within a predetermined range from the first age. An input unit configured to receive a face image and a third face image of a family photographed at an age within a predetermined range from a second age greater than the first age; An image analyzer configured to calculate a similarity value between the second face image and the first face image; A virtual face generator configured to generate a virtual face image of the search subject of a second age using the calculated similarity value; And a search unit for searching for a family of the search target by using the generated virtual face image, or searching for the search target for the family.

Family search method of the present invention for achieving another object of the present invention described above comprises the steps of receiving a first face image of the search target photographed at a first age; Receiving a second face image of the searched person family photographed at an age within a predetermined range from the first age; Calculating a similarity value between the second face image and the first face image; Receiving a third face image of a family photographed at an age within a predetermined range from a second age greater than the first age, and using the first face image, the third face image, and the calculated similarity value, the second age; Generating a virtual face image of the search target of the; And searching for a family of the search subject using the generated virtual face image or searching for the search subject by the family.

According to the present invention, a virtual face image of a current Mia (adopted child) face is generated through similarity analysis between a past face image of a lost child (adopted child) and a family face image, and a similarity analysis using the generated virtual face image is performed. Family search results based on the similarity of faces between family members can be obtained. In addition, according to the present invention, the accuracy of face prediction can be improved by utilizing the similarity result between the past Mia and the family face image for morphing the face of the past Mia.

Hereinafter, a family retrieval system and method using face image analysis and a recording medium on which a program for performing the method on a computer according to the present invention will be described in detail with reference to the accompanying drawings and embodiments. In the following description and the accompanying drawings, substantially identical components are denoted by the same reference numerals, and thus redundant description will be omitted. In addition, in the following description of the present invention, if it is determined that a detailed description of related known functions or configurations may unnecessarily obscure the subject matter of the present invention, the detailed description thereof will be omitted.

1 is a block diagram showing a family search system and a terminal for information input or reception of the present invention.

The information input computer 10 is a computer for registering information for family search.

The information retrieval side is provided with information retrieval means such as the information receiving computer 50 or the mobile terminal 20, the SoIP video terminal 30, the IPTV 40, and the like, and can retrieve the family search results. The family search system 60 receives the information of the search subject and the family information received through the information input computer 10, searches for the family of the search subject, or searches the search subject, and then returns the searched result to the information search side. Provided to the inquirer through the terminal. Here, the search target means, for example, a lost child or adopted child who has been separated from her family for a certain period of time.

2 is a block diagram illustrating a family search system according to an embodiment of the present invention. The family search system 100 illustrated in FIG. 2 includes an input unit 110, an image analyzer 120, a virtual face generator 130, a searcher 140, a general information DB 150, and a family face image DB ( 160, the search subject face image DB 170 and the virtual face image DB 180.

The input unit 110 receives input information for family search from the user terminal. Here, the user terminal has an information input computer capable of inputting general information including image information, for example. In this embodiment, the information input to the input unit 110 includes a first face image of the search subject photographed at a first age, a second face image of the search subject family photographed at an age within a predetermined range from the first age, and And information on the third face image of the family photographed at an age within a predetermined range from the second predetermined age.

In more detail, the first face image refers to a face image of a lost child photographed at an adjacent point in time at which the lost child occurs. The first age refers to a lost age at the time of photographing the first face image. In addition, in the present embodiment, based on facial similarity between the search subject and the family, the second face image and the first face image of the family photographed within a predetermined range from the first age for the comparison of faces of the same or similar age group are included. Compare. Second age refers to, for example, the current age of a lost child or adopted child. The third face image refers to a face image of a family photographed at the same or similar age group as Mia's current age.

The face image information input to the input unit may be input in the form of a file obtained by scanning an original image file or a photo. When receiving a face picture itself, not an image file, the input unit may further include a scanning unit (not shown) for obtaining an image file from the face picture. In addition, the input unit may further receive general information on the search target as well as the above-described face image information. The concept of general information will be described later.

The image analyzer 120 calculates a similarity value between the second face image and the first face image. The image analyzer 120 may further include a feature point extractor 122 and a similarity calculator 124. The feature point extractor 122 extracts feature points according to a predetermined criterion from the first to third face images input through the input unit, respectively.

The similarity calculator 124 calculates a similarity value between the first and second face images by using the extracted positions of the feature points or the distance between the feature points. For example, the similarity calculator may calculate a cosine distance between the first face image and the second face image as a similarity value. In addition to the cosine distance, Euclidean distance, Mahalanobis distance, etc. may be used as similarity values.

The virtual face generator 130 generates a virtual face image of the search subject of the second age by using the calculated similarity value. In particular, the virtual face generator 130 may generate a virtual face image of the search target through morphing using the similarity value calculated by the image analyzer 120 and the first and third face images. Create

The virtual face generator 130 may further include a feature point coordinate calculator 132 and an image deformer 134. The feature point coordinate calculation unit 132 may calculate the coordinates of the feature points extracted from the first face image and the feature points extracted from the third face image according to a predetermined weight determined according to the similarity value calculated by the image analyzer 120. Calculate the coordinates by summing the coordinates. The image deformer 134 generates the virtual face image by deforming the first face image according to the coordinates calculated by the feature point coordinate calculator.

When there are a plurality of families to be searched, the virtual face generator generates a virtual face image of the person to be searched according to each family by using similarity values calculated for each family, and mixes the generated virtual face images by mixing them. The apparatus may further include a mixing unit (not shown) for generating a virtual face image.

The searcher 140 searches for the family of the search target by using the generated virtual face image, or the family searches for the search target, and provides a search result to a searcher through a search result inquiry terminal (not shown). do.

The searcher 140 selects the most similar virtual face image by comparing the virtual face of the lost Mia generated by the image transform unit 134 with the virtual face stored in the virtual face image DB 180 ( And an information provider (not shown) for reading family information according to the selected face image from the general information DB 150 and transmitting the read result to a search result inquiry terminal. The search result inquiry terminal may include, for example, an information receiving computer, a mobile terminal, a SoIP video terminal, an IPTV, and the like.

General information DB 150 stores the general information input through the input unit 110. General information here includes, for example, general information about the lost child and general information about the family. General information about the lost child includes the current age, sex, place of occurrence, time of occurrence, age at the time of occurrence, and the name of the lost child. General information about the family includes the name of the child (adopted child), the age of the family, the address of the family at the time of the loss, the family's sex, and information about the brother or sister.

The family face image DB 160 stores information on the face image of the family input through the input unit 110. It stores face image information of each family member input by a family related to the lost or adopted child. In particular, the family face image DB is a second face image photographed at the same age or the same age as the child's age at the time of the loss, and a photographing point at the same or similar age as the current child's age (the second age). Stores information on the third face image. Similar age group here means, for example, a second age group and an age group of 5 to 10 years old.

The search target face image DB 170 stores face image information of the search target input through the input unit 110. In particular, the search target face image DB stores a face image (corresponding to a first face image) of a lost child photographed at a time adjacent to the time of occurrence of the lost child.

The virtual face image DB 180 stores the virtual face image information generated by the image transformation unit 134.

The family search system shown in FIG. 2 is used to find a lost child, Mia, who is a search target, or a lost family.

When the lost child (or adopted child) wants to search for her own family, the lost child inputs the input information necessary for the family search to an information input computer which is an external device, and the input information is inputted through the input unit 110 to the family search system. Is entered. Here, the input information includes general information about Mia, current face image information (including photographs) of Mia, and general information about Mia's family. The virtual face image DB stores virtual face image information that has already been generated based on general information and face image information provided by a family seeking a lost child. The searcher 140 searches for a face image that matches the face image of Mia among the face images stored in the virtual face image DB, and outputs family information according to the found face image as a search result.

When a family of a lost child (or adopted child) wants to search for a lost child, the lost family inputs necessary input information for searching for the lost child into an information input computer that is an external device, and the input information is inputted through the input unit 110. Is entered into the search system. Here, the input information includes general information about Mia and facial image information (including photographs) about Mia and Mia's family. The virtual face generator 130 performs prediction on the current face of the lost child based on the input information. The searcher 140 may be a Mia generated by the virtual face generator 130 from among face images stored in the face image DB 170 of a person to be searched, which stores information about a recent face image of Mia provided by Mia who is looking for a family. Search for a face image that matches the virtual face image of, and output information of the lost child according to the found face image as a search result.

3 is a conceptual diagram illustrating a concept of calculating a similarity value in the similarity calculator 124 of FIG. 2. The similarity calculator 124 calculates a similarity value between a lost face image (first face image) and a face image of each family member (second face image) at a time when the lost child is adjacent to the lost child. In particular, the second face image should be a face image of a family photographed at the same or similar age group as the child's age at the time of photographing the first face image. FIG. 3 compares Mia's face image with childhood face images of father, mother, sibling 1, and sibling 2, and shows another similarity value calculation result.

4 is a conceptual diagram illustrating a concept of generating a virtual face in the virtual face generator 130 of FIG. 2. The virtual face generator 130 may be a face image of the lost child (first face image) at the time of occurrence of the lost child and a family face image of the same age as or similar to the current lost child's age when the current lost child is in her 20s. 3 face image) and morphing considering the similarity value. FIG. 4 illustrates an example of generating a virtual face image of a lost child by generating four virtual face images corresponding to each of father, mother, brother 1, and brother 2 and averaging the generated four virtual face images.

5 is a flowchart illustrating a family search method according to an embodiment of the present invention. The flowchart shown in FIG. 5 includes the following steps performed time series in the family search system 100.

In step 202, the input unit 110 receives a lost face image (first face image) at the time of occurrence of the lost child. The lost face image input here is preferably input in the form of digitized face image information, but may be input in the form of a photograph. In this case, a process of transforming a photo into facial image information through scanning is further required.

In step 204, the input unit 110 receives a family face image (second face image) photographed at the same or similar age as the age of the lost child when the child is lost.

In operation 206, the feature point extractor 122 extracts facial feature points representing the feature of the face well from the first face image and the second face image.

In operation 208, the similarity calculator 124 calculates a similarity value between the contrasted images by comparing feature points extracted from the first face image and the second face image. For example, the similarity calculator may calculate the similarity value using the length and the ratio between the feature point positions.

In operation 210, the feature point extractor 122 extracts facial feature points from a family face image (third face image) photographed at the same age as the current mother's age.

In operation 212, the feature point coordinate calculator 132 calculates deformation coordinates according to each feature point for transforming the first face image to the third face image in consideration of the similarity value calculated in step 208.

In operation 214, the image transformation unit 134 generates a virtual face image by morphing the first facial image according to the deformation coordinates calculated in operation 212.

FIG. 6 is a reference diagram for describing a concept of calculating deformation coordinates in operation 212 of FIG. 5. 6 illustrates feature points extracted from a face of a lost child (or a family) input through an input unit. The feature points are extracted from both the first face image and the third face image, and in particular, the red feature points are extracted according to a predetermined feature extraction algorithm to represent the features of eyes, nose, mouth, and ears. In determining facial similarity, the upper part of the forehead is excluded. The outer vertices are the blue feature points that connect the forehead and chin lines.

As illustrated in FIG. 6, the face image may be divided into a plurality of divided regions having the extracted feature points as vertices. All vertices belonging to the face image may be represented by the following equation (1).

[Equation 1]

V = x T 1 + y T 2 + z T 3

Here, T 1, T 2 and T 3 are vectors representing feature points T 1, T 2 and T 3, and V is a vector representing vertices specified by T 1, T 2 and T 3.

In the present embodiment, a virtual face is generated by dividing a face image into divided regions and morphing each region to generate a virtual face. Morphing is to transform the source image into a target. In this embodiment, the source is the first face and the target is the third face. The degree of morphing is adjustable according to the strain rate. In this embodiment, the deformation rate is adaptively adjusted according to the similarity value (degree of similarity) calculated in step 208. The relationship between the similarity value and the deformation rate may be determined in various ways by the system designer. For example, when the similarity value between the first face and the third face is high, the ratio (for example, 20:80) may be determined to have a higher weight of the target than the source, and when the similarity value is low, The ratio (eg, 70:30) can be determined so that the weight is high.

For example, when a source and a target are to be transformed to 50:50, for example, the feature point or vertex Ta ' of the modified source may be expressed as Ta' = ( Ta + Tb ) / 2. Here, Ta is a vector representing the feature point of the source, Tb is a vector representing the feature point of the target, and Ta ' is a vector representing the feature point of the modified source. As another example, when it is modified to 3: 2, it can be expressed as Ta ' = (3 Ta + 2 Tb ) / 5.

Computing the modified feature points ( T 1 ', T 2', T 3 ') for all the feature points ( T 1, T 2, T 3) constituting the vertex V , the modified vertex V is expressed as Can be.

[Equation 2]

V ' = x T 1' + y T 2 '+ z T 3'

By modifying all the feature points of the face image by the above-described method, the deformation coordinates for each feature point can be calculated.

In operation 216, the searcher 140 calculates a similarity value between the contrasted images through a comparison between the virtual face image generated in operation 214 and the lost face image stored in the searched face image DB 170.

In operation 218, the searcher 140 provides information about the lost child or the family of the lost child having the highest similarity value as a result of operation 216.

In this embodiment, a family searching method using face similarity between family members is disclosed. Further contemplated with this embodiment is a family search method using general information about a child (adopted child) or family. General information here includes, for example, the age at the time of the loss of the child (adopted child), the current age, gender, where it occurred, the time of occurrence, the name of the child, the name of the family, and the address of the family. Further use can improve the accuracy of the search.

On the other hand, the family search method using the face analysis of the present invention can be implemented in a computer-readable code on a computer-readable recording medium. The computer-readable recording medium includes all kinds of recording devices in which data that can be read by a computer system is stored.

Examples of computer-readable recording media include ROM, RAM, CD-ROM, magnetic tape, floppy disks, optical data storage devices, and the like, which may be implemented in the form of a carrier wave (for example, transmission over the Internet). Include. The computer readable recording medium can also be distributed over network coupled computer systems so that the computer readable code is stored and executed in a distributed fashion. In addition, functional programs, codes, and code segments for implementing the present invention can be easily inferred by programmers in the art to which the present invention belongs.

So far I looked at the center of the preferred embodiment for the present invention. Those skilled in the art will understand that the present invention can be embodied in a modified form without departing from the essential characteristics of the present invention. Therefore, the disclosed embodiments should be considered in descriptive sense only and not for purposes of limitation. The scope of the present invention is shown not in the above description but in the claims, and all differences within the scope should be construed as being included in the present invention.

The present invention can be used as a family retrieval system for finding lost or adopted infants who have long separated. In addition, the family retrieval system of the present invention enables more accurate and efficient retrieval of separated family information through face similarity analysis and face prediction for face image information input by a family or a lost child (adopted child) who is looking for a separated family. have.

1 is a block diagram showing a family search system and a terminal for information input or reception of the present invention.

2 is a block diagram illustrating a family search system according to an embodiment of the present invention.

3 is a conceptual diagram illustrating a concept of calculating a similarity value in the similarity calculator 124 of FIG. 2.

4 is a conceptual diagram illustrating a concept of generating a virtual face in the virtual face generator 130 of FIG. 2.

5 is a flowchart illustrating a family search method according to an embodiment of the present invention.

FIG. 6 is a reference diagram for describing a concept of calculating deformation coordinates in operation 212 of FIG. 5.

Claims (12)

A first face image of the search subject photographed at a first age, a second face image of the searched family member photographed at an age within a predetermined range from the first age, and a second age greater than the first age An input unit configured to receive a third face image of a family photographed at an age; An image analyzer configured to calculate a similarity value between the first face image and the second face image; A virtual face generator configured to generate a virtual face image of the search subject of the second age by using the first face image, the third face image, and the calculated similarity value; And And a search unit for searching for information about the family of the search subject or searching for information about the search subject by using the generated virtual face image. The method of claim 1, The virtual face generator A morphing is performed by using the first face image as a source and targeting the third face image, wherein the morphing is performed in consideration of the similarity value calculated by the image comparator. . The method of claim 2, The image analyzing unit A feature point extracting unit configured to extract feature points according to a predetermined criterion from the first to third face images, respectively; And a similarity calculator configured to calculate a similarity value between the first and second face images by using the extracted feature points or distances between the feature points. The method of claim 3, wherein The virtual face generator calculates coordinates obtained by adding the coordinates of the feature points extracted from the first face image and the coordinates of the feature points extracted from the third face image according to a predetermined weight determined according to the similarity value calculated by the image comparator. And transforming the first face image according to the calculated coordinates to generate a virtual face image. The method of claim 1, The search target is a child or adopted child, the first age is the age of the search target at the time of occurrence or adoption, the second age is a family search system, characterized in that the search target age at the time of family search. The method of claim 5, wherein When there are a plurality of families to be searched, the image comparison unit calculates a similarity value according to each family, The virtual face generator generates a virtual face image of a search target according to each family by using the calculated similarity value, and generates a mixed virtual face image by mixing the generated virtual face images. The search unit searches for the information on the family of the search target or the information on the search target using the mixed virtual face image. The method of claim 1, The search target is a lost or adopted child, Further comprising a virtual face image DB for storing the virtual face image of the search target predicted in advance according to the family looking for the lost or adopted child, The input unit receives a face image of the search subject of the second age, When the search target is to search for a family, the search unit searches for a virtual face image having a predetermined similarity with the input target face image of the second age from among the virtual face images stored in the virtual face image DB, Family search system, characterized in that to provide information about the family according to the searched virtual face image as a search result. The method of claim 1, The search target is a lost or adopted child, And further including a search subject face image DB previously obtained from the baby or the adopted child. When the family of the search target is to search for the search target, the search unit may display a face image having a predetermined similarity with the virtual face image generated by the virtual face generator from among the face images stored in the search target face image DB. And search and provide information on a search target according to the searched face image as a search result. a) receiving a first face image of a search subject photographed at a first age; b) receiving a second face image of the searched person family photographed at an age within a predetermined range from the first age; c) calculating a similarity value between the second face image and the first face image; d) receiving a third face image of a family photographed at an age within a predetermined range from a second age greater than the first age, and using the first face image, the third face image, and the calculated similarity value; Generating a virtual face image of a search subject of two ages; And e) searching for a family of the search subject or searching for the search subject using the generated virtual face image. The method of claim 9, wherein step c) c1) extracting feature points according to predetermined criteria from the first and second face images, respectively; And c2) calculating a similarity value between the first and second face images using the positions of the extracted feature points or the distances between the feature points. The method of claim 9, Generating the virtual face image of the search subject in step d) generates a virtual face image of the search subject through morphing using the calculated similarity value, the first face image and the third face image. Family search method characterized in that. A computer-readable recording medium having recorded thereon a program for performing a family search method of any one of claims 9 to 11 on a computer.

Images