JP4553667B2 - Utterance identification method and password verification device using the same - Google Patents

Description

  The present invention relates to an utterance identification method for identifying utterance contents from the movement of a lip of a speaker, and more particularly to an utterance identification method for reducing identification errors of utterance contents. The present invention also relates to a password verification device that verifies a password recognized from an utterance state using a speech identification method of the present invention and a password registered in advance.

The following identification method has been proposed as a conventional method for identifying a speaker's Japanese utterance content using lip movement (see, for example, Non-Patent Document 1).
In the conventional method, as shown in FIG. 5, the upper lip reference point is A1, the lower lip reference point is A2, the left and right lip reference points are B1 and B2, and the lower jaw reference point is A3. ~ B2), by measuring the vertical width H (A1 to A2) and the distance HX (A1 to A3) from the upper lip to the lower jaw, and calculating the change state of the horizontal width W, the vertical width H and the distance HX when speaking. Identify the utterance content. For example, when uttering “A” and “I” continuously, three variables when uttering “A” are (W1, H1, HX1), and three variables when uttering “I” are (W2, H2, HX2), the utterance pattern of two consecutive vowels is associated with six variables (W1, H1, HX1) and (W2, H2, HX2). Identify the continuous utterance pattern.
Watanabe, “Dual vowel recognition by lip reading”, Transactions of the Japan Society of Mechanical Engineers, Volume C, 55, 509, January 1989

By the way, the conventional utterance identification methods including the above-mentioned vowel identification method treat each utterance content such as “A”, “I”, “1”, “2” as different ones, and identify the utterance of the speaker. Are taking the way.
However, in the conventional method for identifying each utterance content by treating each utterance content as different, for example, the utterance content has a low identification error rate in a utterer with a stable manner of utterance of five types of vowels. The utterance content identification error rate is high for a speaker whose utterances are difficult to distinguish. As described above, the conventional method has a problem that the error rate of identification largely depends on the individual difference of each speaker.

  The present invention has been made paying attention to the above problems, and an object of the present invention is to provide an utterance identification method capable of reducing the identification error rate without being influenced by how each utterer utters. Another object of the present invention is to provide a password verification apparatus using this utterance identification method.

For this reason, the utterance identification method according to the first aspect of the present invention is based on at least the lip vertical width and horizontal width change patterns for each utterance content of the speaker, and the same utterance content with high similarity in the change patterns is different. As a group, an utterance identification method that classifies each utterance content as a separate group with different utterance content and low similarity of the change pattern, and identifies each utterance content of the individual speaker based on the group classification A plurality of utterance contents are uttered at a certain number of times, and the change pattern is measured at a certain number of times for each of the plurality of utterance contents, and an arbitrary number of the measurement data is measured for each utterance content. Data is extracted as group classification creation data, and the remaining measurement data is used as evaluation data. Based on the extracted group classification creation data, The overlap rate of each utterance content with the other utterance content is calculated based on each statistical distribution state of each utterance content calculated in the above, and the similarity is determined by comparing the calculated overlap rate with a preset threshold value. The utterance contents are grouped to create the group classification, the evaluation data is identified based on the created group classification, the identification error rate is calculated, and the calculated identification error rate is a preset allowable value. The group classification created as described above is determined as OK in the following cases.

Each utterance of the speaker, as in claim 2, it is preferable to be detected with a small portion of the temporal change in the vertical width and the horizontal width of the lips in the change pattern.
As in claim 3 , the utterance content of claim 2 is a number.

According to a fourth aspect of the present invention, there is provided a password verification device according to the present invention, comprising: a detection unit that detects an utterance state of a password uttered by a password registrant; and a change pattern of at least the vertical and horizontal widths of the lips from the utterance state detected by the detection unit Based on the change pattern measurement unit for measuring the password, the database in which the group classification data regarding each utterance content for each password registrant is registered in advance, the change pattern measured by the change pattern measurement unit, and the registration data in the database The utterance identification unit for recognizing the password uttered by the password registrant using the utterance identification method described in Item 1 or 2 , the password recognized by the utterance identification unit is collated with a pre-registered password, and match / mismatch And a collation unit that generates the determination output.

In such a configuration, when the detection unit detects the utterance state of the password uttered by the password registrant, the change pattern measurement unit measures, for example, the change pattern of the vertical and horizontal widths of the lips from the detected utterance state. The utterance identification unit recognizes the password spoken by the password registrant using the utterance identification method of the present invention based on the change pattern measured by the change pattern measurement unit and the registered data in the database. Matching unit is a determination of match / mismatch by matching previously registered password that speech recognition unit has recognized.

As in claim 5 , at the time of password registration, based on the group classification data registered in the database related to the password registrant, the matching probability of the password to be registered is calculated and the validity of the password to be registered is determined. It is good to do. In this case, as in claim 6 , when the calculated matching probability is higher than a predetermined value, it is preferable to instruct the change of the registered contents of the password so that the matching probability is equal to or lower than the predetermined value.

  As described above, according to the utterance identification method of the present invention, different utterance contents that are highly similar to each other and are difficult to identify are not identified as the same group. It is possible to reduce utterance content identification errors without being affected by individual differences.

  In addition, according to the password verification device of the present invention, it is possible to reduce password identification errors, thereby improving the reliability of the authentication device that detects the utterance state and recognizes the registrant.

Hereinafter, an embodiment of an utterance identification method according to the present invention will be described.
The utterance identification method according to the present invention is based on the change pattern of at least the vertical width W and the horizontal width H of the lips with respect to each utterance content of the individual utterance (such as Japanese syllabary and numbers). Each utterance content is classified into a group so that higher ones are in the same group, and different utterance content is low in change pattern similarity. Then, each utterance content of the speaker is selected and identified by group classification. The “utterance” in the present invention does not necessarily have to be accompanied by voice.

  For example, for five numbers “1”, “2”, “3”, “4”, “5”, a speaker speaks “1”, “2”, “3”, “4”, “5”. The lip change pattern when the utterance is stable is different, and another speaker has a high similarity in the lip change pattern when uttering “1” and “2”, and is difficult to identify. It is assumed that the similarity of the lip change pattern when “4” and “5” are spoken is low and easy to identify. In this case, for a certain speaker, “1” to “5” are different groups. For different speakers, “1” and “2” are in the same group, and “3” to “5” are in separate groups. Then, the utterance content is identified by determining to which group the utterance content of the speaker belongs.

  For example, when the password is “1234”, when “1”, “2”, “3”, “4”, “5” are speakers classified into different groups, “1234” is spoken. It is determined that the password matches. Also, in the case of a speaker in which “1” and “2” are the same group and “3”, “4” and “5” are classified into different groups, “1” and “2” are the same group and the same Even if “1234”, “2234”, “2134”, and “1134” are spoken, “1” and “2” are not identified and the passwords are determined to match.

  Therefore, according to the utterance identification method of the present invention, the utterance contents are classified into groups according to each utterer, and the utterance contents that are difficult to identify are not forcibly identified as the same group. It is possible to reduce utterance content identification errors without being influenced by individual differences in patterns.

Next, an embodiment of a method for creating a group that achieves a predetermined utterance content identification rate in the utterance identification method of the present invention will be described with reference to the flowchart of FIG.
In step 1 (indicated by S1 in the figure, the same shall apply hereinafter), the utterer is caused to utter the k utterance contents M times, and the change pattern of the width W and the height H of the lips at that time is measured to determine the utterance data. And
In step 2, the speech data measured in step 1 is registered.
In step 3, an initial value of the overlap rate P is set as a threshold value used in the group classification stage of the utterance content described later. In this case, the initial value of the overlap rate P is set to a large value so that different utterance contents with high similarity are classified into different groups.

In step 4, N pieces are randomly sampled from the M pieces of utterance data of the k kinds of utterance contents registered in step 2 to obtain group classification creation data. Here, the remaining N ′ (M−N) × k utterance data is used as evaluation data for the created group classification.
In step 5, a statistical distribution (probability density) of each utterance content is calculated from each N utterance data of each k utterance content extracted in step 4, and each utterance content is statistically compared with other utterance content. The overlap ratio Q between the distributions is calculated.

  FIG. 2 shows an example of statistical distribution. For example, the utterance contents for calculating the overlap ratio Q between the statistical distributions are A and B, FA (x) in the figure is the statistical distribution of the utterance contents A, and FB (x) is the statistical distribution of the utterance contents B. . The Qab portions where the distributions overlap each other indicate the probability that the utterance content A is erroneously identified as the utterance content B, and the other Qba portions indicate the probability that the utterance content B is erroneously identified as the utterance content A. Yes. Qab and Qba are calculated as the overlapping rate Q.

In step 6, the overlap rate Q calculated in step 5 and the overlap rate P set in step 3 are compared, and k different utterance contents are grouped based on the comparison result. For example, referring to the example of FIG. 2, the groups are grouped as follows according to the magnitude relationship between Qab, Qba and P.

In step 7, using the group classification created in step 6, (N ′ × k) pieces of evaluation data are identified, and an identification error rate is calculated.
In step 8, the discrimination error rate obtained in step 7 is compared with a predetermined tolerance value of a desired discrimination error rate. If the identification error rate calculated in Step 7 is equal to or less than the allowable value, the group classification created in Step 6 is OK, and the process proceeds to Step 9 to determine the group classifications relating to the k utterance contents measured in Step 1. On the other hand, if the identification error rate calculated in step 7 is larger than the allowable value, the process proceeds to step 10.

  In step 10, the setting value of the overlap rate P, which is a threshold value, is reduced for the utterance content having a high identification error rate, and the number of groups to be classified is decreased so that the utterance content with low similarity is also the same group. Step 4 and subsequent processes are repeatedly executed. In this way, a group classification is set so that an undesired identification error rate is obtained.

  As described above, the utterance identification method of the present invention classifies the utterance contents into groups for each utterer, and identifies the utterance contents of each utterer based on the group classification set for each utterer. Compared with the method of distinguishing and identifying all utterance contents, the identification error rate of the utterance contents can be reduced. In other words, the utterance content identification rate can be increased.

Next, an example of a method for measuring the lateral width W and the vertical width H of the lips when measuring the speech data in Step 1 will be described using an example of numerical speech.
Here, a case where the utterance content is “0 (ZERO)” will be described.

FIG. 3 shows a change pattern of the vertical width H and the horizontal width W of the lips when “0 (ZERO)” is spoken. In the figure, H (i) indicates the change pattern of the vertical width, W (i) indicates the change pattern of the horizontal width, and D (i) indicates the change amount of the change amount of the vertical width and the change amount of the horizontal width. Show. Note that i = 1, 2,... Here, H (i) is defined as H (i) = H ′ (i) −, where H (0) is a value at the time of lips before utterance and H ′ (i) is a measured value at the time of utterance. H (0). Similarly, W (i) is W (i) = W ′ (i) −, where W (0) is the value when the lips are closed before speaking and W ′ (i) is the measured value when speaking. W (0). The amount of change D (i) is D (i) = ((H ′ (i) −H (i−1)) ² + (W ′ (i) −W (i−1)) ² ) ^1/2 . is there.

  In addition, the portion with a small amount of change before utterance (a portion in the figure) shows a lip closed state, and the next portion with a small amount of change (b portion in the figure) utters “0 (ZERO)”. The utterance state of “E” of “ZE” indicates the utterance state of “O” of the utterance “RO”. A portion with a small amount of change after utterance (“d” portion in the figure) indicates a lip closed state and is for detecting completion of the utterance.

  As described above, for example, the vowels “A”, [I], “U”, [E], [O] and the lip state have a small change amount D (i) and a stable change pattern at the time of utterance. Therefore, when measuring each utterance data of “ZE” and “RO” of the utterance content “ZERO”, the “b” and “c” portions with a small amount of change are measured, and “ZE” and “RO” are detected and the utterance is detected. Data.

  As in this embodiment, when measuring a portion with a small amount of change such as a vowel as the measurement data of the vertical width H and the horizontal width W of the lips when each utterance content is uttered, the utterance data on the same utterance content Variations can be reduced, and identification errors due to variations for each utterance of the same speaker can be reduced. Also, in this embodiment, since the utterance content is identified from the change pattern of the vertical and horizontal widths of the lips, an image including only the periphery of the lips may be used, and it is necessary for identification as compared with the conventional identification method using the position of the lower jaw. The image range can be narrowed. Therefore, there is an advantage that a necessary amount of image data can be reduced as compared with the conventional method.

Next, the password verification device of the present invention using the speech identification method of the present invention will be described.
FIG. 4 is a block diagram showing an embodiment of the password verification device of the present invention.
4, the password verification apparatus according to the present embodiment includes an input unit 1 that inputs personal data of a speaker, a camera 2 as an imaging unit that images the speaker, and image processing that processes an image captured by the camera 2. Unit 3, utterance identification unit 4 that recognizes the utterance content based on the image processing data of image processing unit 3 and recognizes the input password, database 5 that stores utterance registration data of each speaker registered in advance, It comprises a memory 6 for storing the password of each speaker registered in advance, and a collation unit 7 for collating the input password identified by the utterance identification unit 4 and the registered password stored in the memory 6.

The input unit 1 is for inputting personal information of a pre-registered speaker at the time of authentication.
The camera 2 detects a speech state by capturing a face image of a speaker who is going to be authenticated, and corresponds to a detection unit.
The image processing unit 3 extracts an image around the lips from the image captured by the camera 2 and measures a change pattern of the vertical width H and the horizontal width W of the lips. Therefore, it corresponds to a change pattern measurement unit.
The utterance identification unit 4 reads registration data corresponding to the personal data input by the input unit 1 from the database 5. Further, based on the read registration data and the change pattern data input from the image processing unit 3, the input utterance content is identified using the above-described utterance identification method to recognize the input password.

The database 5 stores group classification data for each speaker classified based on the group classification setting method at the time of registration in association with personal data of each speaker as registration data.
The memory 6 stores a registered password of each speaker registered in advance in association with the personal data of the speaker.
The collation unit 7 reads a registered password corresponding to the personal data input by the input unit 1 from the memory 6, collates it with the input password recognized by the utterance identification unit 4, and generates a match / mismatch determination output.

Next, the operation of the password verification device of this embodiment will be described.
A speaker who wants to receive authentication inputs personal data registered in advance using the input unit 1 and speaks his / her registered password in front of the camera 2. The camera 2 captures the speaker and transmits the captured image to the image processing unit 3. The image processing unit 3 extracts an image around the lips from the input image, measures a change pattern from the movement of the lips, and transmits the measurement data to the utterance identification unit 4. The utterance identification unit 4 reads in advance from the database 5 group classification data for utterance identification of a speaker who is to be authenticated based on personal data input from the input unit 1. When the measurement data is input from the image processing unit 3, the input utterance content is identified based on the read group classification data, the input password is recognized, and the recognized input password is transmitted to the verification unit 7. The verification unit 7 reads in advance from the memory 6 a registered password of the speaker who is to be authenticated based on the personal data input from the input unit 1. When the input password is input from the utterance identification unit 4, it is checked against the read registered password, and if it matches, an authentication OK determination output is generated, and if they do not match, an authentication rejection determination output is generated.

  For example, in the change pattern of the utterance content of a speaker who has previously registered “1234” as the registration password, “1” and “2” are the same group, “3” and “4” are different groups and “ It is assumed that the groups “1” and “2” are classified into different groups. In this case, when the speaker speaks “1234”, even if the speech identifying unit 4 identifies “1134”, “2134”, and “2234” in addition to “1234”, “1” and “2” "Is in the same group, the input password to be transmitted to the collation unit 7 is input to the collation unit 7 as" (1 or 2) (1 or 2) 34 ", and it is determined as a match. Thereby, the probability of matching determination in the matching unit 7 can be increased.

  By the way, in the case of the method of identifying the utterance contents by grouping according to the similarity of the change pattern of the lip movement in each utterance content as in the present invention, depending on the manner of utterance of each utterer, for example, as a password Effective amount of information is different. A person who speaks with a large number of utterance contents classified into the same group reduces the amount of information effective as a password.

Specifically, for example, when described with numbers 0 to 9, {0}, {1,2}, {3}, {4}, {5}, {6}, {7}, {8}, {9 } And 9 types of numerical utterances with different stable utterances, {0, 3, 4, 6}, {1, 2}, {5, 9}, {7}, {8} For example, in the case of a person who utters five different types of numbers, the date of 10000 years from January 01, 0000 to December 31, 9999 (3.65 × 10 ⁶ ways) Assume that a password is selected, for example, an 8-digit sequence of October 25, 1989 (19891025) is selected as the password. In the former case, (19891025), (19891015) (29891015) (29891025) are recognized as the same, and the probability of matching is 4 / (3.65 × 10 ⁶ ), which is about 10 ⁻⁶ . On the other hand, in the latter case, 2 (thousands of the year) x 2 (hundreds of the year) x 1 (tenths of the year) x 2 (first of the year) x 1 (tenths of the month) ) × 4 (1st digit of the month) × 2 (10th digit of the day) × 2 (1st digit of the day) = 128 patterns are recognized as the same, and the probability of matching is 128 / (3.65 × 10 ⁶ ) ^Is about 3.5 × 10 ⁻⁵ . In other words, in the former case, there are nine types of valid information as passwords, but in the latter case, where the number of group classifications is smaller than the former, valid information as passwords is reduced to five types and can be entered arbitrarily. The probability that the entered 8-digit number matches the password is increased.

  Therefore, in the case of the password verification apparatus shown in FIG. 4 that employs the above-described utterance identification method of the present invention, an attempt is made to register at the time of password registration based on the group classification data obtained by measuring how the password registrant speaks. It is desirable to calculate the password matching probability and determine its validity. In addition, if the match probability is higher than the desired match probability as a result of calculating the match probability, it is instructed to change the registered content of the password to be registered so that the registered password has a match probability equal to or lower than the predetermined value. It is desirable to do.

  For example, as described above with respect to the numbers 0 to 9, when the number of group classifications is 5 people compared to the number of group classifications 9 types, any date within 10,000 years is registered as a password The probability of matching the arbitrarily entered 8-digit number with the password increases, and the effectiveness of the password is low. In such a case, for a person with 5 types of group classification, in order to reduce the password matching probability when registering a password, for example, a registrant such as an employee number or telephone number in addition to the date Guidance is given to register a password with a number that is easy to remember. By doing so, the reliability of the password verification device of the present invention can be improved.

  In the above embodiment, the lip content group classification is performed using the change pattern of the vertical and horizontal width of the lips at the time of speaking, but the distance from the upper lip to the lower jaw is also added to the vertical and horizontal widths of the lips. Needless to say, it may be used.

The flowchart explaining one Embodiment of the group classification confirmation method of the speech identification method based on this invention Figure showing an example of statistical distribution The figure which shows an example of the change pattern of the vertical width H and the horizontal width W of the lip at the time of the number utterance The block diagram which shows one Embodiment of the password verification apparatus which concerns on this invention Illustration of conventional utterance identification method

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Input part 2 Camera 3 Image processing part 4 Speech identification part 5 Database 6 Memory 7 Collation part

Claims (6)

Based on at least the lip height and width change patterns for each utterance content of the speaker, different utterance contents with high similarity of the change patterns are grouped together, and different utterance contents have similarities of the change patterns. low is a speech identification method classified into groups each utterance as a separate group, identifying on the basis of the speech content of the speech individuals in the group classification,
A plurality of utterance contents are uttered at a fixed number of times, the change pattern is measured a fixed number of times for each of the plurality of utterance contents, and an arbitrary number of data from the measurement data is obtained for each utterance content. Extracted into group classification creation data, and the remaining measurement data as evaluation data, with respect to each utterance content based on each statistical distribution state of each utterance content calculated based on each extracted group classification creation data Calculate the overlap rate with other utterance content, compare the calculated overlap rate with a preset threshold value, determine the similarity, group each utterance content, and create the group classification Based on the group classification, the evaluation data is identified to calculate an identification error rate, and the group created when the calculated identification error rate is equal to or less than a preset allowable value Class of speech identification method is characterized in that so as to determine a group classified as OK. Each utterance of the speaker is speech identification method according to claim 1 which is adapted detected using a temporal portion of a small change in the vertical width and the horizontal width of the lips in the change pattern. The utterance identification method according to claim 2 , wherein the utterance content is a number. A detection unit for detecting the utterance state of the password spoken by the password registrant;
A change pattern measuring unit that measures a change pattern of at least the vertical and horizontal widths of the lips from the utterance state detected by the detection unit;
A database in which group classification data related to each utterance content for each password registrant is registered in advance;
Based on the change pattern measured by the change pattern measurement unit and the registration data of the database, the utterance identification unit that recognizes the password spoken by the password registrant using the utterance identification method according to claim 1 or 2 ,
A collation unit that collates the password recognized by the utterance identification unit with a pre-registered password, and generates a match / mismatch determination output;
A password verification device characterized by comprising: During password registration, based on the group classification data to be registered in the database of passwords registrant, claim 4 which calculates the probability of match password to be registered and to determine the validity of the password to be the registered The password verification device described in 1. The password verification device according to claim 5 , wherein when the calculated matching probability is higher than a predetermined value, a change in password registration content is instructed so that the matching probability is equal to or lower than the predetermined value.

Images