JP2003331214A - Character recognition error correction method, apparatus and program - Google Patents

Abstract

An object of the present invention is to provide a method for correcting an error of a Japanese character recognition device to which two or more different notations such as a kanji notation and a kana notation are simultaneously given. SOLUTION: The present invention, for example, uses a kanji notation and a kana notation such as an address and a name in a handwritten form.
When different notations of more than one kind are given at the same time, the same content is used in the kanji notation and the kana using the language model, the character recognition device model, and an algorithm that searches for the optimal word string while associating the kanji notation with the kana notation. By utilizing the redundancy that is expressed in the notation, an error that cannot be corrected by only one of the kanji notation and the kana notation is corrected.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a technique for correcting an error that occurs in handwritten character recognition, and more particularly to reading a character pattern as a character recognition result and inputting it using a language model that expresses the connection before and after the character. The present invention relates to a method and apparatus for correcting an error in a Japanese character recognition device that detects an error portion of a generated character pattern and presents a correct answer candidate.

[0002]

2. Description of the Related Art Generally, in a handwritten form such as an application form, when writing an address or a name in Kanji, it is customary to give a Kana in Kana. This can be achieved when humans process this form data by using the constraint that the Kanji notation and the Kana notation represent the same content, or the redundancy that the same content is represented by the Kanji notation and Kana notation. Has the effect of preventing mistakes.

For example, in the case of manually processing a handwritten form, if it is not possible to determine from the kanji notation whether it is "Mizada" or "Nagata", if the kana notation can be discriminated as "Nagata", the kanji notation is " You can see that it is Nagata. Similarly,
If the kana notation can be determined to be “Nagata” or “Nagata” and the kanji notation is “Nagata”, then the kana notation is “Nagata”. In addition, if there is a possibility that the kanji notation is "Paddy" and "Nagata" and there is a possibility that the kana notation is "Nakata" and "Nagata",
Since "Mikata" cannot be read as "Nagata", it is understood that the only valid interpretation is the combination of the Kanji notation "Nagata" and the Kana notation "Nagata".

However, in the case of automatically processing a form using a character recognition device, the conventional character recognition error correction method handles the kanji notation and the kana notation at the same time, so that there is a constraint existing between them. It was not possible to realize error correction while effectively utilizing (or redundancy). The reasons are the following two (1) and (2).

(1) Since it is virtually impossible to register all the sets of kanji and kana notations that may appear in the dictionary, the correct set of kanji and kana is entered even if they are not in the dictionary. You must consider the possibility of appearing in.
However, a method of giving a probability to such an event has not been devised.

(2) When there are a plurality of character recognition candidates for Kanji and Kana respectively, the possibility of correspondence between Kanji and Kana becomes extremely large. However, no algorithm has been devised to systematically check this.

In the conventional error correction method of character recognition, the character ng is input to one input character string (kanji or kana notation).
ram, that is, when n represents an integer of 2 or more,
Statistical data such as a character ngram model that represents an n-chain consisting of n characters, or a word ngram model that represents an n-chain consisting of n words when n represents an integer of 2 or more. The mainstream method is to perform error correction using a language model.

As an example of using the character ngram model, Sugimura and Saito: "Unreadable character determination processing using character concatenation information-application to character recognition-" IEICE Transactions Vol.J68-D No. 1, pp. 64-71, 1985. As an example of using the word ngram model, Takao and Nishino: “Realization and evaluation of post processing of Japanese document reader”
IPSJ Transactions Vol.33 No.5, pp.664−670, 1992 Ito and Maruyama: “Error Detection and Automatic Correction of OCR Input Japanese Sentences” IPSJ Transactions Vol.33, No.5, pp.664
−670, 1992 Nagata: “Japanese Character Recognition Error Correction Method Using Character Similarity and Statistical Language Model” IEICE Transactions (DI)
I) Vol.J81-D-II, No.11, pp.2624-2634, 1998.

In contrast to these methods, in recent years, a method of performing error correction by simultaneously using kanji notation and kana notation is based on a statistical language model in which a basic unit is a single kanji and its reading set. Nagata, M .: Synchronous Morphological Analysis of
Grapheme and Phonemefor Japanese OCR, Proceedings
^{of the 38 th Annual Meeting of the} Association for
Computational Linguistics, pp.384-391, 2000. In the method proposed in this document, for example, when inputting a set of character recognition results of kanji and kana notation "Fukuzawa Yukichi" and "Fukuzawa Yukichi",
Based on the probability of appearance of a single Kanji and its reading group such as “Fuku / Fuku”, “Fuku / Fuku” and “Sawa / Sawa /
The error correction is performed by obtaining the correspondences such as "Sawa", "Satoshi / Yu", and "Kichi / Kichi".

A method of using a single kanji and its reading as a basic unit of a language model is a short character string such as a name (the kanji notation is 3
This is an effective method when there are a large number of different readings for one Chinese character (about -5 characters).

[0011]

However, when a long character string (Kanji notation is about 10 to 15 characters) such as an address is targeted, if a single Kanji and its reading are the basic units of the language model, a search is performed. There is a problem that the number of power combinations becomes huge and the amount of calculation becomes large. In particular, the number of candidates for Kanji notation retrieved from Kana notation becomes a problem.

[0012] For example, in the case of inputting a combination of the kanji notation "Hikarinooka, Yokosuka City, Kanagawa Prefecture" and the kana notation "Kanagawa Ken Yokosuka Shihikari no Oka," the kanji that may be read as "ka" is as shown below. At least 21
There are four.

[0013]

[Table 1] Next, there are at least 17 Chinese characters that can be read as "Kana", as shown below.

[0014]

[Table 2] In addition, at least 6 kanji that may be read as "na"
There are 6 and there are at least 15 Chinese characters that can be read as "Naga".

Similarly, the reading of the kanji "God" is Shin Jin Kami Kan Kaka Kana Duck Bear bear There are at least 14 like.

In the case of error correction of character recognition, not only the first candidate of the recognition result but also the lower candidates have to be considered, so that the search space becomes larger and the calculation cannot be ignored in a long character string such as an address. It becomes the amount.

When a word is used as a basic unit of a statistical language model, a large number of combinations of characters included in a character matrix (that is, a list in which character candidates are arranged in descending order of character recognition score at each character position of an input sentence). Since the parts do not form words, fewer words are matched against the dictionary. However, on the contrary, the situation does not improve because the number of unknown word candidates, that is, word candidates that are not matched with the dictionary increases.

If an unknown word model that gives a high probability to a combination of a kanji character string and a kana character string that is likely to form a word can be devised, the calculation amount can be reduced by narrowing down the unknown word candidates. Therefore, it is possible to perform error correction while associating a pair of relatively long character strings such as Kanji and Kana notation of an address by associating them with each other. However, conventionally, such an unknown word model has not been proposed.

In view of the above-mentioned problems of the prior art, the present invention provides a first comparatively long first address such as Kanji and Kana.
In the set of the character string of the notation and the character string of the second notation, the first notation and the second notation are handled by simultaneously handling the corresponding set of character recognition results of the first notation and the second notation. It is an object of the present invention to provide a character recognition error correction method that realizes error correction using a constraint or redundancy existing between the characters.

Another object of the present invention is to provide an apparatus for implementing such a character recognition error correction method.

A further object of the present invention is to provide a program that causes a computer to implement such a character recognition error correction method.

[0022]

SUMMARY OF THE INVENTION According to the present invention, a language model is provided when a first notation and a second notation such as a kanji notation and a kana notation are given at the same time as an output of a character recognition device.
It is said that the same content is expressed in the first notation and the second notation by using the character recognition device model and an algorithm for searching for an optimum word string while associating the first notation with the second notation. By using the redundancy, an error that cannot be corrected by only one of the first notation and the second notation is corrected.

According to the present invention, a character string having an arbitrary first notation,
Regarding the character string of the second notation, the simultaneous probability of the character string of the first notation and the character string of the second notation, that is, the character string of the first notation makes the character string of the second notation the first notation. And a language model that gives the probability that the character string of the second notation is the character string of the first notation and the arbitrary character string , A character recognition device model that gives a probability that one character is erroneously recognized by the other character, and a word string with the highest probability based on the language model and the character recognition device model, that is, the first notation with the highest probability. By using the optimum word string search means for obtaining a set of the word string of and the word string of the second notation, by associating the character string of the first notation with the character string of the second notation,
Provided is a character recognition error correction method for correcting an error that cannot be corrected by only one of the character string of the first notation or the character string of the second notation.

The invention according to claim 1 is the first character string and the second character string according to the first character string obtained by character recognition of the character string of the first character string and the character string of the second character string. A character recognition error correction method for a second set of character strings according to claim 1, wherein the character string according to the first notation is a character string representing the character string according to the second notation in the first notation, and The language model that gives the word coincidence probability in which the character string of the notation is the character string of the first notation expressed in the second notation, and two characters for each of the first notation and the second notation A character recognition device model which gives a character confusion probability that one of the characters is erroneously recognized as the other character is prepared in the storage means, and the storage is performed for the first character string and the second character string set. Using the language model and the character recognition device model prepared for the means Obtaining a most probable set of the character string of the first notation and the character string of the second notation by extracting the word string candidates and selecting the word string with the highest probability from the word string candidates. Is a character recognition error correction method characterized by:

The invention according to claim 2 is the first character string and the second character string according to the first character string obtained by character-recognizing the character string of the first character string and the character string of the second character string. A method for character recognition error correction of a second set of character strings according to claim 1, which uses the procedure of setting a model in the storage means and the model set in the storage means, From the extracted word string candidates using the procedure of extracting the character string and the word string candidates of the second character string set in the second notation and the model set in the storage means, By choosing the word sequence with the highest probability,
And a procedure for obtaining a set of the most probable character string of the first notation and the character string of the second notation, and the procedure of setting the model in the storage means is the first notation representing the same word. And a word dictionary storing a second set of notations are stored in the storage means, and the word chain model that gives the appearance probability of two or more chains of the first notation and the second set of words is stored in the storage unit. An unknown word model stored in the storage means and giving a probability that an arbitrary character string in the first notation and a character string in the second notation that are not registered in the word dictionary form a word; For each of the notation 1 and the second notation, including a procedure of storing in the storage means a character recognition device model that gives a character confusion probability that one of two characters is erroneously recognized as the other character, The procedure for extracting a column candidate is the first character string described above. A procedure of searching the word dictionary for a word candidate that completely matches a possible combination of the second character strings, and a combination of the first character string and the second character string are possible. From the procedure of searching the word dictionary for similar word candidates that are similar-matched with the set, and the set that is possible as a combination of the first character string and the second character string, the unknown word that is not registered in the word dictionary is unknown. A procedure for generating word candidates,
The procedure for acquiring the set of the character string of the notation and the character string of the second notation is, for the combination of the word candidate, the similar word candidate, and the unknown word candidate, the word dictionary, the word chain model, and Using the unknown word model, the word in the first notation is the word representing the word in the second notation in the first notation, and the word in the second notation is the word in the first notation. The word joint probability that is the word expressed in 2 is obtained,
Using the above character recognition device model, obtain the character confusion probability,
It is a character recognition error correction method characterized by including a procedure of selecting a word string that maximizes a product of the word coincidence probability and the character confusion probability.

According to the third aspect of the present invention, the unknown word model that gives a probability that a character string in the first notation and a character string in the second notation which are not registered in the word dictionary form a word. Is to classify any word into any of the word types defined based on the type of characters that make up the first notation of the word, calculate the appearance probability of the word type from the appearance frequency of the word type, From the average word length of the first notation of the type and the average word length of the second notation, the joint probability of the length of the first notation and the length of the second notation is calculated, and two or more of the first notation are calculated. The first word of the word is calculated by calculating the probability of appearance of the character string of the first notation and the probability of appearance of the character string of the second notation from the character chain of 2 and two or more characters of the second notation.
The set of the character string of the first notation and the character string of the second notation is used to estimate the probability of forming a word for each word type classified based on the type of the character that constitutes the notation.

The invention according to claim 4 is the first character string and the second character string according to the first character string obtained by character recognition of the character string of the first character string and the character string of the second character string. A character recognition error correction device for a second set of character strings according to claim 1, wherein the character string according to the first notation is a character string representing the character string according to the second notation in the first notation, and The language model that gives the word coincidence probability in which the character string of the notation is the character string of the first notation expressed in the second notation, and two characters for each of the first notation and the second notation The character recognition device model which gives a character confusion probability that one of the characters is erroneously recognized by the other character, and the set of the first character string and the second character string are prepared in the storage means. Using the language model and the character recognition device model,
An optimal word for obtaining a most probable set of the character string of the first notation and the character string of the second notation by extracting the word string candidates and selecting the word string with the highest probability from the word string candidates. A character recognition error correction device comprising: a column search means.

According to a fifth aspect of the invention, the first character string and the second character string according to the first character string obtained by recognizing the character string of the first character string and the character string of the second character string are recognized. A character recognition error correction device for a second set of character strings according to claim 1, which uses a storage unit for storing a model and the model set in the storage unit,
From the extracted word string candidates by using the means for extracting the word string candidates of the second character string group according to the second notation and the model set in the storage means. , An optimal word string searching means for acquiring a most probable set of the character string of the first notation and the character string of the second notation by selecting the word string with the highest probability, The stored model is a word dictionary that stores a first notation and a second notation set that represent the same word, and the occurrence of two or more chains of the first notation and the second notation set of words. A word chain model that gives a probability; an unknown word model that gives a probability that a character string in the first notation and a character string in the second notation that are not registered in the word dictionary form a word;
And a second notation, and a character recognition device model that gives a character confusion probability that one of the two characters is erroneously recognized by the other character, and means for extracting the word string candidate , A word collating unit that searches the word dictionary for word candidates that completely match a possible combination of the first character string and the second character string, the first character string, and the second character string. A similar word matching means for searching the word dictionary for similar word candidates to be similar-matched with a possible combination of the character strings, and a combination of the first character string and the second character string. An unknown word candidate generating means for generating an unknown word candidate that is not registered in the word dictionary from a certain set, and the optimum word string searching means includes the word candidate, the similar word candidate, and the unknown word. For the combination of candidates Then, using the word dictionary, the word concatenation model, and the unknown word model, the word in the first notation is the word representing the word in the second notation in the first notation, and the second notation Is a word in which the word in the first notation is expressed in the second notation is obtained, the character confusion probability is obtained using the character recognition device model, and the word simultaneous probability and the character confusion probability are calculated. The character recognition error correction device is characterized by selecting a word string that maximizes the product.

According to the invention of claim 6, the unknown word model giving the probability that a character string of the first notation and a character string of the second notation which are not registered in the word dictionary constitute a word. Is a word type determination means for classifying an arbitrary word into any one of word types defined based on the type of characters forming the first notation of the word, and the first of each word type.
A word length probability calculation means for calculating a joint probability of the length of the first notation and the length of the second notation from the average word length of the notation and the average word length of the second notation; Word notation probability calculation means for calculating the appearance probability of the character string of the first notation and the appearance probability of the character string of the second notation from the chain of at least two characters and the chain of two or more characters at the second notation And an unknown probability for estimating the probability that a set of the character string of the first notation and the character string of the second notation configures a word for each word type classified based on the type of characters that make up the first notation of the word. And word probability calculation means.

The invention according to claim 7 is the first character string and the second character string according to the first character string obtained by character-recognizing the character string of the first character string and the character string of the second character string. A character recognition error correction program for a second set of character strings according to claim 1, wherein the character string according to the first notation is a character string representing the character string according to the second notation with the first notation, and A language model that gives a word joint probability in which the character string of the notation is the character string of the first notation represented by the second notation,
For each of the first notation and the second notation, a character recognition program model that gives a character confusion probability that one of two characters is erroneously recognized as the other character, the first character string, and the second character Using the language model and the character recognition program model prepared for the storage function for the set of character strings, the word string candidates are extracted, and the word string with the highest probability is selected from the word string candidates. A character recognition error correction program characterized by causing a computer to realize an optimum word string search function for acquiring a most probable set of a character string of the first notation and a character string of the second notation. .

The invention according to claim 8 is the first character string and the second character string according to the first character string obtained by character-recognizing the character string of the first character string and the character string of the second character string. Is a character recognition error correction program for a second set of character strings according to the first character string according to the first notation using a memory function for storing a model and the model set in the memory function. And using the function of extracting the word string candidates of the second character string set according to the second notation and the model set in the storage function, the most probable probability among the extracted word string candidates. By selecting a word string with high
The model that allows the computer to realize the most probable first word string search function and the optimum word string search function for acquiring the second word string set, and stores the same word in the model A word dictionary storing a first notation set and a second notation set to be expressed, a word chain model giving an appearance probability of two or more chains of the first notation set and the second notation set of the word, and the word For each of the first notation and the second notation, an unknown word model that gives the probability that a character string of the first notation and a character string of the second notation that are not registered in the dictionary form a word And a character recognition program model that gives a character confusion probability that one of the two characters is erroneously recognized by the other character, and the function of extracting the word string candidates is performed by the first character string and the second character string. Possible combinations and combinations of character strings A word matching function that searches for a matching word candidate from the word dictionary, and a similar word candidate that performs similar matching with a possible combination of the first character string and the second character string from the word dictionary A similar word matching function, and an unknown word candidate generation function for generating an unknown word candidate that is not registered in the word dictionary from a possible combination of the first character string and the second character string. , The optimal word string search function, for the combination of the word candidate, the similar word candidate, and the unknown word candidate, using the word dictionary, the word chain model and the unknown word model, The word in the notation 1 is the word representing the word in the second notation in the first notation, and the word in the second notation is the word representing the word in the first notation in the second notation. Find the word probability and Seeking character confusion probabilities using character recognition program model, the product of the word joint probability and the character confusion probability is chosen word strings that maximizes a character recognition error correction program, characterized in that.

According to the invention of claim 9, the unknown word model giving the probability that a character string of the first notation and a character string of the second notation which are not registered in the word dictionary form a word. Is a word type determination function that classifies an arbitrary word into any of word types defined based on the type of characters that form the first notation of a word, and a first type of each word type.
A word length probability calculation function that calculates a joint probability of the length of the first notation and the length of the second notation from the average word length of the notation and the average word length of the second notation; A word notation probability calculation function for calculating the appearance probability of the character string of the first notation and the appearance probability of the character string of the second notation from the chain of two or more characters in the second notation And an unknown probability for estimating the probability that a set of the character string of the first notation and the character string of the second notation configures a word for each word type classified based on the type of characters that make up the first notation of the word. And a word probability calculation function.

[0033]

1 is a block diagram of a character recognition error correction system according to a first embodiment of the present invention. The character recognition error correction system of the present embodiment has a Japanese kanji notation,
A character recognition device 1 for recognizing a Japanese kana notation, and a character recognition error correction device 100 for correcting a character recognition error depending on a result of recognition of a set of a Japanese kanji notation and a kana notation by the character recognition device 1. , Including. The first aspect of the present invention
Since the examples assume character recognition in the address field of a handwritten form such as an application form, the kanji notation may include kanji, katakana, hiragana, romaji, alphanumeric characters, etc. , Kana in the furigana field may include Roman characters and alphanumeric characters in addition to Katakana or Hiragana. In the following description of the embodiments, for the sake of simplicity, it is assumed that the kanji notation is composed of only kanji and the kana notation is composed of only katakana.

The character recognition error correction device 100 inputs the character matrix in kanji and the character matrix in kana as a result of character recognition from the character recognition device 1. The character matrix is a list in which character candidates are arranged in descending order of character recognition score at each input character position, and the list is arranged by the number of characters. In addition, hereinafter, a character string formed by selecting one character at a time from each character position in the character matrix from the list of character candidates at that character position is referred to as a “character string included in the character matrix”.

The character recognition error correction device 100 includes an optimum word string search unit 2 which receives a character matrix in kanji notation and a character matrix in kana notation from the character recognition device 1, and a word storing a set of kanji notation and kana notation of a word. Register in the dictionary 7, the word collation unit 3 that searches the word dictionary 7 for words that completely match a set of kanji strings included in the character matrix of kanji notation and kana strings included in the character matrix of kana notation, and the word dictionary 7. From an unknown word model 8 that gives the probability that an arbitrary set of Kanji and Kana sequences that form a word does not exist, and a set of Kana strings included in the Kanji character matrix and Kana string contained in the Kana character matrix. , An unknown word candidate generation unit 4 for generating an unknown word candidate using the unknown word model 8,
For any two characters, a character recognition device model 10 that gives a probability that one character is erroneously recognized by the other character, that is, a character confusion probability, and a kanji string and kana characters included in a kanji character matrix The similar word matching unit 5 that searches the word dictionary 7 for words to be similarly matched with the set of kana strings included in the matrix, and n chains of pairs of kanji notation and kana notation of words, that is, the occurrence probability of ngram is given. The word ngram model 6, the word ngram model 6, the word dictionary 7, and the unknown word model 8 are included. For any kanji string and kana string, the kanji string is the kanji notation of the kana string and the kana string is the kanji string. The language model 9 provides a probability of kana notation, that is, a joint probability of kanji notation and kana notation.

Further, the optimum word string searching unit 2 receives a word that completely matches with the word dictionary 7 from the word matching unit 3, receives an unknown word candidate from the unknown word candidate generating unit 4, and receives from the similar word matching unit 5. Receiving similar words that have undergone similar matching with the word dictionary 7, a word string with the highest probability, that is, a set of kanji notation and kana notation is obtained based on the language model 9 and the character recognition device model 10.

FIG. 2 is a flow chart for explaining the character recognition error correction method in the character recognition error correction system according to the first embodiment of the present invention.

The character recognition error correction method is performed by the character recognition device 1.
A character matrix for kanji and a character matrix for kana obtained by recognizing a set of kanji and kana are used as input.

In step 1, the word collation unit 3 completely matches a word in the word dictionary 7, that is, a set of a kanji character string included in the kanji character matrix and a kana character string included in the kana character string, that is, a word in the word dictionary 7. , Identify matching words.

In step 2, the unknown candidate generator 4
Generates a candidate of a word that is not registered in the word dictionary 7, that is, an unknown word, in a set of a kanji character string included in the kanji character matrix and a kana character string included in the kana character matrix. .

In step 3, the similar word matching unit 5
Identifies a word in the word dictionary 7 that is similar-matched with a set of a kanji character string included in a kanji character matrix and a kana character string included in a kana character matrix, that is, a similar word.

Finally, in step 4, the optimum word string searching unit 2 uses the language model 9 and the character recognition model 10 to select a word string having the highest probability among the matching word, the unknown word and the similar word, that is, , Find a set of kanji and kana notation.

In the above description, step 1, step 2,
Although the processing is performed in the order of step 3, the order of executing step 1, step 2, and step 3 is not limited to such order, and any order may be performed.

According to the configuration of the character recognition error correction system of the first embodiment of the present invention as described above, it is possible to associate the Kanji notation and the Kana notation, and it is possible to use only one of the Kanji notation and the Kana notation. Correct the uncorrectable error, and even if the input includes a word that is not registered in the dictionary, the probability of occurrence of a set of kanji and kana strings is estimated based on the unknown word model 8, and the correct characters are included in the character matrix. Even if not, the error correction candidate can be presented by the similar word matching.

FIG. 3 is a schematic block diagram of a character recognition error correction system according to the second embodiment of the present invention. Next, the configuration of the second embodiment of the present invention will be described with reference to FIG.
In FIG. 3, components designated by the same reference numerals as those of FIG. 1 are the same or similar components as the corresponding components of FIG.

The character recognition error correction system according to the second embodiment of the present invention comprises a character recognition device 1, an optimum word string search unit 2,
A word matching unit 3, an unknown word generation unit 4, a similar word matching unit 5,
A language model 9 and a character recognition device model 10 are included.
The word model 9 includes a word bigram model 60, a word dictionary 7, and an unknown word model 8. Word bigram
The model 60 includes a word bigram frequency table 61,
The word bigram probability calculation unit 62 is included. The unknown word model 8 includes an unknown word probability calculation unit 81 and a word type determination unit 8
2, a word type definition storage unit 83, a word length probability calculation unit 84, an average word length table 85, a word notation probability calculation unit 86, and a character bigram frequency table 87. The character recognition device model 10 includes a character confusion probability calculation unit 1
1 and a character recognition device correct answer rate table 12.

Here, the word bigram represents a two-chain consisting of two words, and the character bigram represents a two-chain consisting of two characters.

The optimum word string search unit 2 receives the set of the kanji notation matrix and the kana notation matrix output from the character recognition device 1 for the input set of the kanji notation and kana notation, and inputs each of the two character matrices. About, the dynamic programming (dynamic programm)
ing), the joint probability of the word string, that is, the word b
A word string that maximizes the product of the igram probability and the character confusion probability is obtained.

Therefore, the optimum word string searching unit 2 includes the perfect match word from the word matching unit 3, the unknown word candidate from the unknown word candidate generating unit 4, and the similar word candidate from the similar word matching unit 5. , Given as a word candidate. The character confusion probability calculation unit 11 gives the word candidates the character confusion probabilities that form the kanji notation and kana notation of the word. In addition, the word bigram probability is calculated by the word bigram probability calculation unit 6
Given by 2.

The word collating unit 3 collates all combinations of the character strings included in the character matrix of the Kanji notation and the character strings included in the character matrix of the Kana notation with the word dictionary 7 and regards the collated ones as a complete match word. , To the optimum word string search unit 2. Here, at each character position of the character matrix, a character string formed by selecting one character at a time from the list of character candidates at that character position is called a “character string included in the character matrix”.

The unknown word candidate generating unit 4 selects, as an unknown word, a combination of the character strings included in the character matrix of the Kanji notation and the character strings included in the character matrix of the Kana notation that is not matched with the word dictionary 7. Then, a predetermined number of unknown words in the descending order of unknown word probabilities obtained by the unknown word model probability calculation unit 81 are given to the optimum word string search unit 2 as unknown word candidates.

The similar word collating unit 5 collates all combinations of the character strings included in the character matrix of the Kanji notation and the character strings included in the character matrix of the Kana notation with the word dictionary 7, and the collated word is the similar word. It is given to the optimum word string search unit 2 as a candidate. As a distance measure for similar word matching, it is possible to use the edit distance (that is, the percentage of matching characters) that indicates the number of insertions, deletions, and substitutions required to convert one character string into another. it can.

The word bigram probability calculation unit 62 uses the word bigram added to the word bigram frequency table 61.
The appearance probability of the word bigram is calculated from the ram frequency.

The word type determination unit 82 determines the word type of the unknown word from the types of characters forming the Kanji notation of the unknown word candidate based on the word type definition 83.

The word length probability calculation unit 84 calculates the Kanji notation length and the Kana notation length of the unknown word candidate from the average word length of the Kanji notation and kana notation of each word type stored in the average word length table 85. Find the joint probability.

The word notation probability calculation unit 86 uses the character bigr.
am bi character in kanji notation stored in the frequency table 87
From the gram frequency and the character bigram frequency of the kana notation characters, the simultaneous probability of the kanji notation character string and the kana notation character string of the unknown word candidate is obtained.

The character confusion probability calculation unit 11 calculates a character confusion probability from the first candidate correct answer rate and the cumulative correct answer rate stored in the character recognition device correct answer rate table 12.

Thus, the optimum word string searching section 2 uses the word b.
A word string (a set of kanji notation and kana notation) that maximizes the product of the igram probability and the character confusion probability can be obtained.

In the following, first, the "information theoretical interpretation of character recognition error correction" which is the theoretical basis of the present invention will be described, and then a language model, an unknown word model, a character recognition device model, an optimum word string search. The means and the similar word matching means will be described in this order.

[1] Information Theoretical Interpretation of Character Recognition Error Correction In the third embodiment of the present invention, the relationship between the input and output of the character recognition device is determined by a model of a noisy communication channel (noisy channel mo
del) formalize. The first input kanji (graphemes) and the second input kana (phonem notation)
es) are G and P, respectively, and the outputs of the character recognition device for them are G'and P '. The error correction of the character recognition of the present invention is a kanji notation that maximizes the posterior probability P (G, P | G ', P').

[0061]

[Equation 1] And kana notation

[0062]

[Equation 2] Result in the problem of seeking. Furthermore, using Bayes' theorem, a set of kanji and kana notation that maximizes P (G, P) P (G ', P' | G, P)

[0063]

[Equation 3] You can see that

[0064]

[Equation 4] Here, P (G, P) is a language model, and P (G ', P'
| G, P) is called a character device model.

[2] Language model Kanji notation and kana notation, respectively Character string of length l G = α₁α_Two． ． ． α_l as well as Character string of length P = β₁β_Two． ． ． β_m Consists of In addition, kanji notation and kana of words
By associating the notations, a set of kanji and kana notations
Is a word string of length n (G, Ρ) = ((g₁, P₁), (G_Two，
p_Two) ,. ． ． , (G_n, P_n)) Suppose it is divided into

For example, a combination of the kanji notation “Hikarinooka, Yokosuka City, Kanagawa Prefecture” and the kana notation “Kanagawa Ken Yokosuka Shihikari no Oka” is a word string ((Kanagawa Ken, Kanagawa Ken),
(Yokosuka City, Yokosukashi), (Hikarinooka, Hikarinooka)) are associated and divided.

In the present invention, the language model P (G, P) is
The Chinese character notation G and the kana notation P are approximated with the joint probability of the most likely word string P (G, Ρ). Further, the joint probability P (G, Ρ) of this word string is set to the word bigra described later.
Approximate with m model.

[3] Word bigram model In general, the word N-gram represents an N chain consisting of N words when N represents an integer of 2 or more. As an example, the word bigram is a word chain consisting of two words. Word bigram model, the word as in the following equation bigram probability _{P (g i, p i |} g i-1, p
The product of _i-1 ) approximates the joint probability of word strings.

[0069]

[Equation 5] Here, <bos> and <eos> are special symbols representing the beginning and end of a sentence.

The method for obtaining the word bigram probability (that is, word bigram probability calculation means) is as follows.

First, the Japanese text in kanji and kana mixed notation is divided into words, and data with kana reading added is created. Hereinafter, this data will be referred to as learning data.

The kanji notation of words in this learning data
Occurrence frequency C (g_i, P_i) And words
Occurrence probability C of a bigram in a set of kanji and kana notation
(G _i-1, P_i-1, G_i, P_i), The word bi
It is stored in the gram frequency table.

FIG. 4 is an explanatory diagram of an example of word appearance frequency.
FIG. 5 is an explanatory diagram of an example of the word bigram appearance frequency.
In the illustrated example, a set of a kanji string and a kana string separated by '/' represents one word.

Next, based on these appearance frequencies, the relative frequency f (g _i , p _i ) of the word and the relative frequency f of the word bigram.
(G _i , p _i | g _i-1 , p _i-1 ) is calculated.

[0075]

[Equation 6] Next, these relative frequencies are linearly interpolated to obtain the word bigra.
Find m sets of probabilities. Where the linear interpolation coefficient α is the training data
The maximum probability of data is determined. P (g_i, P_i| G_i-1, P_i-1) =   (1-α) f (g_i, P_i) + Αf (g_i, P_i| G_i-1, P_i-1) (5) [4] Unknown word model The unknown word model is a kanji notation not registered in the word dictionary.
In the calculation model for obtaining the appearance probability of the set of
is there. This constitutes the unknown word (g, p) Length l_gKanji character string g = cg₁． ． ． cg_lg When, Length l_pKana notation character string p = cp₁． ． ． cp_lp Is defined as the joint probability distribution P (g, p) of

In one embodiment of the present invention, a plurality of word types are defined based on the types of characters that make up the Kanji notation of a word, and the probability of unknown words is estimated for each word type.

FIG. 6 is an explanatory diagram of an example in which unknown Japanese words are classified into seven word types. The definition of the word type is described in Backus Naur Form (BNF), where [...] Indicates matching with any one character in the character set. A character range is written by writing a-between two characters. JIS-X-0208 is assumed for the character code. * Represents 0 or more repetitions, + represents 1 or more repetitions.

<Sym>, <num>, <alpha>
>, <Hira>, <kata>, and <kan>
Represents a symbol string, a numeral string, an alphabet string, a hiragana string, a katakana string, and a kanji string, respectively. Character strings composed of multiple character types other than these are all <mi
sc>.

In the third embodiment of the present invention, the word type probability, that is, the appearance probability P (<WT>) of each word type <WT> in the unknown word, and the kanji notation and kana of the unknown word for each word type. Simultaneous probability P (g, p | <WT
The probability of appearance of an unknown word (simultaneous probability of kanji notation and kana notation of an unknown word) P (g, p) is obtained from the product of>). P (g, p) = P (<WT>) P (g, p | <WT>) (6) Word type probability classifies low-frequency words (words with an appearance frequency of 1) in learning data into word types. Then, each is calculated from the relative frequency of the word type.

Unknown word appearance probability P (g,
p | <WT>) is a joint probability P (l _g , l _p | <WT>) of the length of the kanji notation and the length of the kana notation for each word type,
It is approximated by the product of the appearance probability P (g) of the character string in Kanji and the appearance probability P (p) of the character string in Kana.

[0081]

[Equation 7] Below are the Kanji notation lengths and Kana notation by word type.
Joint probability of length P (l _g, L_p｜ <WT>) word length
Call the rate.

The word length probability is approximated by the product of the distribution of the length of kanji notation for each word type and the distribution of the length of kana notation for each word type. , Average kanji character length by word type λ
_{g, <WT>} and average character length of kana notation λ _{p, <WT>}
It is approximated by a Poisson distribution with as a parameter.

[0083]

[Equation 8] The average character length λ _{g, <WT>} of the kanji notation for each word type and the average character length λ _{p, <WT>} of the kana notation are obtained from the low frequency words in the learning data. FIG. 7 is an explanatory diagram of an example of average character lengths of kanji notation and kana notation for each word type.

The appearance probability P (g) of a character string in Kanji is
It is approximated by the character bigram, which is the character used for Kanji notation.

[0085]

[Equation 9] The appearance probability P (p) of a character string in kana notation is approximated by a character bigram model of characters used in kana notation.

[0086]

[Equation 10] Similar to the word bigram probability, the character bigram probability in kanji and kana notation can be obtained by linearly interpolating the relative probabilities from the character appearance frequency and the character bigram appearance frequency obtained from the learning data. . FIG. 8 shows an example of the appearance frequency of the character bigram in kanji, and FIG. 9 shows the character bigram in kana.
It is a figure which shows the example of the appearance frequency of.

[5] Character recognition device model In the third embodiment of the present invention, the character recognition device model P
Regarding (G ', P' | G, P), Kanji notation G and kana table
Notation P is recognized independently, and each character in Kanji notation cg _i
And each character in kana notation cp_jAre independently recognized
It

[0088]

[Equation 11] Generally, in a character recognition device, a probability P (c _j | c _i ) that an input character c _i is recognized as a character c _j is called a character confusion probability. The character confusion probability is basically a character confusion matrix (characte) which is frequency data of input and output pairs of a character recognition device.
r confusion matrix). However, the character confusion matrix is not versatile because the character recognition method largely depends on the quality of the input image. In addition, since there are more than 3000 types of characters in Japanese, it is not possible to collect a sufficiently large number of character recognition results for all characters.

Therefore, in the third embodiment of the present invention, the correct answer rate p ₁ of the first candidate output by the character recognition device and the cumulative correct answer rate p _n up to the nth candidate (n is output by the character recognition device). The character confusion probability is approximated as follows using the number of character candidates) as a parameter.

[0090]

[Equation 12] Here, | C | is the size of the character set to be recognized. For example, in the case of Chinese characters, | C _g | = 6879, and in the case of Kana, | C _p | = 87.

In equation (12), if the character is the first candidate, a constant value (average correct answer rate of the first candidate) p ₁ is assigned as the character confusion probability regardless of the character. If the character is included in the candidate characters after the second candidate, the remainder obtained by subtracting the first candidate correct answer rate from the cumulative correct answer rate is equally allocated. If the character is not a character candidate, the value obtained by subtracting the cumulative correct answer rate from 1 is equally assigned to the character set other than the candidate character.

[6] Optimal word string search The procedure (optimal word string searching means) for obtaining a set of kanji notation and kana notation maximizing the posterior probability shown in the equation (1) is shown below.

The input Kanji and Kana notation
Each Character string of length l G = α₁α_Two． ． ． α_l as well as Character string of length P = β₁β_Two． ． ． β_m And

The character position in the Kanji notation and the character position in the Kana notation are respectively expressed as x (0≤x≤l) and y (0≤y).
≤ m), a word string of length n (G, Ρ) = ((g ₁ , p ₁ ), (g ₂ ,
p ₂ ) ,. ． ． , (G _n , p _n )) is a sequence of word boundary coordinates (length n + 1) (x ₀ , y ₀ ), (x ₁ , y ₁ ), (x ₂ ,
y ₂ ) ,. ． ． , (X _n , y _n ). Here, the character position (x _i , y
_i ) are the ending positions of the words (g _i , p _i ), respectively (x ₀ , y ₀ ) = (0, 0) and (x _n , y _n ) =
(L, m).

The joint probability P (g ₁ , p ₁ , ..., G _i , p _i ) of the word string from the beginning to the i-th word, and the kanji and kana notation characters that make up each word When the maximum value of the product with the character confusion probability is defined as φ (g _i , p _i ), the following relationship is established from Expression (2).

[0096]

[Equation 13] Here, q and r are kanji notation g_iStart and end positions of
Where s and t are kana notation p_iStart and end positions of
Represents That is, g_i= Cg_{q + 1}． ． ． cg_r p_i= Cp_{s + 1}． ． ． cp_t Is. Also, cg '_jAnd cp '_kIs cg_jAnd c
p_kIs a character recognition result corresponding to.

Expression (13) represents the following relationship. lead
To the i-th word and the kanji notation for each word
Maximum value of the product of the character confusion probabilities of the characters that make up the kana notation
φ (g_i, P_i) Is from the first to the i-1th word
Synchronous probability and each sentence that composes kanji and kana notation of each word
The maximum value of the product of the character confusion probability_i-1，
p_{i- 1}), And the word of the i-1st word and the i-th word
bigram probability P (g_i, P _i| G_i-1，
p_i-1) The maximum value of the product of
Multiplied by the product of the character confusion probabilities of the characters that make up the kana notation
It is a thing. Using this relationship, φ (g
_i, P_i), The maximum probability from the beginning to the end
Value φ (g_n, P_n) Can be asked.

FIG. 10 is a flow chart for explaining the operation of optimum word string search. The optimum word string search realizes the calculation of Expression (13) using a two-dimensional dynamic programming method. Here, φ (g _i , p _i ) is called a partial analysis probability, and a table storing φ (g _i , p _i ) is called a partial analysis table.

The optimum word string search operation will be described below with reference to FIG.

The optimum word string search starts from the beginning of the kanji notation and kana notation, and the current analysis position of each character advances toward the end by one character. In step S11, the start position of the search is set to the beginning (0, 0) of the kanji and kana notations.

In step S12, it is determined whether or not the search has reached the end of the Chinese character notation. If the end has been reached, the optimum word string search ends. If not, the following processing is performed at each character position of the Kanji notation.

In step S13, it is determined whether the search has reached the end of the kana notation. If the end has been reached, the process proceeds to step S30. If not, the following processing is performed at each character position in the kana notation.

In step S14, all the word strings are searched from the partial analysis table until the combination of the current character position in Kanji and the current character position in Kana is reached, and one of them is searched for in the current partial analysis (word Select as a column.

In step S15, it is determined whether all word strings have been examined. If so, in step S29, the search proceeds to the next character position in the kana notation. Otherwise, the following process is performed for each word string.

In step S16, a list of all kanji character strings included in the kanji character matrix starting from the current kanji character position is created.

In step S17, a list of all kana character strings included in the character matrix of kana notation starting from the current character position of kana notation is created.

In step S18, a word list composed of all the combinations of the kanji character string list created in step S16 and the kana character string list created in step S17 is created. Any word in this list that does not match the word dictionary is considered an unknown word.

In step S19, the list of kanji character strings created in step S16 and the word in the word dictionary that is similar to the list of kana character strings created in step S17 are added to the word list.

In step S20, one word is selected from the word list.

In step S21, it is determined whether all the words have been examined. If not, the process proceeds to step S28. Otherwise, the following process is performed for each word.

In step S22, it is checked whether or not the current word (word string from the beginning with the current word as the last word) is registered in the partial analysis table. If so, proceed to step S24. If not,
In step S23, this word is registered in the partial analysis table, the probability of partial analysis (word string) is initialized to 0, and then the process proceeds to step S24.

In step S24, the probability of a new word string based on the combination of the current word string and the current word is calculated. The probability of a new word string is expressed by the following equation.

[0113]

[Equation 14] In step S25, it is checked whether the probability of the new word string is greater than the probability of the previous word string in which the last word is the same. If so, the probability of the new word string is stored in the partial analysis table in step S26, and the process proceeds to step S27.

In step S27, the next word is selected, and the process returns to step S21.

In step S28, the next word string is selected,
Return to step S15.

In step S29, the search is advanced to the character position next to the kana notation, and the process returns to step S13.

In step S30, the search is advanced to the next character position in the Chinese character notation, and the process returns to step S12.

[7] Similarity Matching of Words A method of similarity matching of words in step S9 of optimum word search will be described below. The character position of the current Kanji character string is x, and the character position of the kana character string is y.

First, a word similarity search is performed using a kanji character string as a key.

(1) Search for all words in the word dictionary that match the kanji notation, starting from the character position x in the current kanji notation, and the elements of the list of all kanji character strings included in the kanji notation character matrix. To create a similar word candidate list.

(2) For each element of the similar word candidate list, the edit distance (match) between the kana notation and all kana character strings included in the character matrix of the kana notation starting from the character position y of the current kana notation Find the minimum value of the number of characters that is not used.

(3) From the similar word candidate list, words having a relative edit distance (ratio of non-matching characters to the length of the character string) of 0.5 or less are taken out, arranged in order of appearance frequency, and a maximum of 5 words are similar. Generate as a word candidate.

Next, a word similarity search is performed using the kana character string as a key.

(4) Search for all words in the word dictionary whose kana notation matches the element of the list of all kana character strings included in the character matrix of the kana notation, starting from the current character position y of the kana notation. To create a similar word candidate list.

(5) For each element of the similar word candidate list, its kanji notation and the editing distance (corresponding to all the kanji character strings included in the character matrix of the kanji notation starting from the character position x of the current kanji notation) Find the minimum value of the number of characters that is not used.

(6) From the similar word candidate list, words having a relative edit distance (ratio of non-matching characters to the length of the character string) of 0.5 or less are taken out, arranged in order of appearance frequency, and a maximum of 5 words are similar. Generate as a word candidate.

(7) Finally, the set sum of the word candidates generated using the Kanji character string as a key and the word candidates generated using the kana character string as a key (collecting duplicate word candidates into one) The final candidate for similar words is used.

The editing distance threshold value 0.
5 and the maximum number of candidates of 5 are examples of the set values of the parameters, and the optimum value is experimentally determined, for example. Also, in the above example, first the Kanji character string is used as a key to perform a word similar search, and then the kana character string is used as a key to perform a word similarity search. It is also possible to perform a similar search for a word, and then perform a similar search for a word using the Kanji character string as a key.

[0129]

EXAMPLE Finally, a processing example according to an embodiment of the present invention will be shown. FIG. 11 is a diagram illustrating an example of an optimum word string search for a set of character matrices output by the character recognition device for the kanji notation “Itozaki-cho, Fukui-shi, Fukui Prefecture” and the kana notation “Fukuiken Fukuishitozakicho”. .

In this processing example, up to the second candidate is used as the character matrix. For example, the first candidate and the second candidate for the Chinese character “Fuku” are “Fuku” and “Ran”, respectively, and the first candidate and the second candidate for the Kana “Fu” are “Fu” and “Fu”, respectively. It is "ku".

In the optimum word string search, at each character position, all combinations of the word string reaching the word and the words starting from it are examined, and the joint probability of the word string at the ending position of the starting word is updated.

On the left side of FIG. 11, the character position in Kanji notation is represented by x (horizontal axis), the character position in kana notation is represented by y (vertical axis), and a word string reaching a certain character position (6, 9) and starting 5 shows the positional relationship between the words, that is, the starting position of the last word of the word string reaching the character position (6, 9) and the ending position of the word starting from the character position (6, 9).

On the right side of FIG. 11, the last word of the word string reaching the character position (6, 9) and the character position (6, 9).
It shows how to examine all the combinations of words starting from 9). Each word is represented by "Kanji character string / kana character string", and the coordinates of the start position and end position of the word are shown at the top of the box corresponding to the word. When the word is an unknown word, the word type is shown at the bottom of the box.

There are three types of word candidates, one that completely matches the word dictionary, one that is similar-matched to the word dictionary, and one that is not matched to the word dictionary (unknown word). For example,
The words "Fukui City / Fukuishi" from character positions (3,5) to character positions (6,9) are exact matches, and the kanji character string "Fukui City" and the kana character string "Fukuishi" are in the character matrix. Yes, and "Fukui City / Fukuishi" is in the word dictionary.

From character position (6,9) to character position (8,1)
The word “Itozaki / Itozaki” in 3) is a similar collation, and the kanji character string “Itozaki” is in the character matrix, but the kana character string “Itozaki” is not in the character matrix, and the kanji character string is the key word. It was retrieved from a dictionary. The word candidate “city / ishi” from the character position (5, 7) to the character position (6, 9) is an unknown word candidate pair, and the kanji character string “city” and the kana character string “ishi” are in the character matrix. However, this word "city / ishi" is not in the dictionary.

FIG. 11 shows a state of processing at a certain character position (6, 9), but such processing starts from the origin (0, 0) and ends at (9, 16) on the plane. By carrying out at all grid points, it is possible to obtain a word string that maximizes the product of the joint probability of kanji and kana notation and the probability of character confusion.

The character recognition error correction method according to the above embodiment of the present invention can be constructed by software (program), and the character recognition according to the embodiment of the present invention can be performed by executing this program by the CPU of the computer. An error correction device can be realized. The constructed program is recorded in a disk device or the like and installed in the computer as needed, stored in a portable recording medium such as a flexible disk, a memory card, or a CD-ROM, and installed in the computer as necessary. Alternatively, it is installed in the computer via a communication line or the like and executed by the CPU of the computer.

Although the representative embodiments of the present invention have been described above, the present invention is not limited to the above embodiments, and various modifications and applications are possible within the scope of the claims.

[0139]

As described above, according to the present invention, a language model which gives a simultaneous probability of a kanji character string and a kana character string, a character recognition device model which gives a character confusion probability, a language model and a character recognition device model. Based on, the optimum word string search means for finding the word string with the highest probability for the character recognition result set of the input kanji character string and kana character string is used to associate the kanji notation with the kana notation. By using the redundancy that the same contents are expressed in Kanji and Kana notation, it is possible to correct the character recognition error that cannot be corrected only by either Kanji or Kana notation. The method can be realized.

[Brief description of drawings]

FIG. 1 is a configuration diagram of a character recognition error correction system according to a first embodiment of the present invention.

FIG. 2 is a flowchart of a character recognition error correction method according to a first embodiment of the present invention.

FIG. 3 is a configuration diagram of a character recognition error correction system according to a second embodiment of the present invention.

FIG. 4 is an explanatory diagram of an example of a word dictionary.

FIG. 5 is an explanatory diagram of an example of the appearance frequency of the word bigram.

FIG. 6 is an explanatory diagram of an example of defining a word type.

FIG. 7 is an explanatory diagram of an example of an average length of kanji notation and an average length of kana notation for each word type.

FIG. 8 is an explanatory diagram of an example of the appearance frequency of a character bigram in Kanji notation.

FIG. 9 is an explanatory diagram of an example of the appearance frequency of the characters bigram in kana notation.

FIG. 10 is a flowchart of an optimum word string search process according to the third embodiment of the present invention.

FIG. 11 is a diagram showing an example of optimum word string search.

[Explanation of symbols]

1 character recognition device 2 Optimal word string search unit 3 Word matching unit 4 Unknown word candidate generator 5 Similar word matching unit 6-word ngram model 7 word dictionary 8 Unknown word model 9 word model 10 Character recognition device model 100 character recognition error correction device

Claims (12)

[Claims] 1. A first character string in the first notation and a second character string in the second notation obtained by character-recognizing the character string in the first notation and the character string in the second notation. In the character recognition error correction method of the set, the character string in the first notation is a character string in which the character string in the second notation is represented in the first notation, and the character string in the second notation is A language model that gives a word coincidence probability that is a character string in which the character string in the first notation is expressed in the second notation, and one of the two characters is the other for each of the first notation and the second notation. And a character recognition device model that gives a character confusion probability of being erroneously recognized as a character in the storage means, and the character prepared in the storage means for the set of the first character string and the second character string. A word string candidate is extracted using the language model and the character recognition device model, and the word string is extracted. By choosing the most probable word sequence from the complement to obtain a set of most probable first notation character string and the string of the second representation, character recognition error correction method characterized by. 2. A first character string in the first notation and a second character string in the second notation obtained by character-recognizing the character string in the first notation and the character string in the second notation. A method of setting a model in a storage means, and using the model set in the storage means, the first character string and the second character string according to the first notation. A procedure for extracting a word string candidate of the second character string set by the notation and a word string having the highest probability among the extracted word string candidates by using the model set in the storage means. By selecting the most probable set of the character string of the first notation and the character string of the second notation, and the procedure of setting the model in the storage means is the same word. A word dictionary storing a set of the first notation and the second notation is stored in the storage means. An arbitrary first notation that is stored in the storage means and that stores a word chain model that gives the probability of occurrence of two or more chains of the first notation and the second notation set of words An unknown word model that gives the probability that the character string of the above and the character string of the second notation form a word are stored in the storage means, and one of two characters for each of the first notation and the second notation is stored. Includes a step of storing a character recognition device model that gives a character confusion probability of being erroneously recognized as the other character in the storage means, and the step of extracting the word string candidate includes the first character string and the second character string. A procedure for searching the word dictionary for word candidates that completely match a possible combination of character strings, and similar matching with a possible combination of the first character string and the second character string. Searching for similar word candidates from the word dictionary, A procedure for generating an unknown word candidate that is not registered in the word dictionary from a possible combination of a first character string and the second character string, the most probable first notation The procedure for acquiring the set of the character string of the above and the character string of the second notation is, for the combination of the word candidate, the similar word candidate, and the unknown word candidate, the word dictionary, the word chain model, and the unknown Using the word model, the word in the first notation is the word representing the word in the second notation in the first notation, and
The word joint probability that the word represented by is the word in which the first notation is represented by the second notation is obtained, the character confusion probability is obtained using the character recognition device model, and the word joint probability and the character confusion are obtained. A method for character recognition error correction, including a procedure for selecting a word string that maximizes the product of probabilities. 3. The unknown word model which gives a probability that a character string of the first notation and a character string of the second notation which are not registered in the word dictionary form a word is the first notation of the word. Arbitrary words are classified into any of the word types defined based on the type of characters that make up the word type, the appearance probability of the word type is calculated from the appearance frequency of the word type, and the average of the first notation of each word type is calculated. The joint probability of the length of the first notation and the length of the second notation is calculated from the word length and the average word length of the second notation, and a chain of two or more characters of the first notation and the second notation Notation 2
By calculating the appearance probability of the character string of the first notation and the appearance probability of the character string of the second notation from a chain of at least two characters, based on the type of characters that make up the first notation of the word The character recognition error correction method according to claim 2, wherein a set of a character string of the first notation and a character string of the second notation is used to estimate the probability of forming a word for each classified word type. . 4. A first character string in the first notation and a second character string in the second notation obtained by character-recognizing the character string in the first notation and the character string in the second notation. In the character recognition error correction device of the set, the character string in the first notation is a character string in which the character string in the second notation is represented in the first notation, and the character string in the second notation is A language model that gives a word coincidence probability that is a character string in which the character string of the first notation is represented by the second character, and one of the two characters is the other for the first character and the second character. Character recognition device model which gives a character confusion probability of being erroneously recognized as a character, and the language model and the character recognition prepared in the storage means for the set of the first character string and the second character string. The device model is used to extract word string candidates, and the most probable word string candidate is extracted. Character recognition error characterized by having an optimum word string search means for acquiring a most probable set of a character string of the first notation and a character string of the second notation by selecting a word string having a high rate. Correction device. 5. A first character string in the first notation and a second character string in the second notation obtained by character-recognizing the character string in the first notation and the character string in the second notation. A character recognition error correction device of a set, wherein a storage unit for storing a model, the model set in the storage unit is used, and the first character string and the second notation according to the first notation are used. A means for extracting a word string candidate of the second character string set by the above and a model set in the storage means are used to select a word string with the highest probability from the extracted word string candidates. And a most suitable word string search means for acquiring a set of the most probable character string of the first notation and the character string of the second notation, and the model stored in the memory means is the same word. A word dictionary storing a set of a first notation and a second notation, Word chain model that gives the appearance probabilities of two or more chains of the notation and the second notation set, and a character string of the first notation and a character string of the second notation that are not registered in the word dictionary An unknown word model that gives a probability that a word forms a word, and a character recognition device that gives a character confusion probability that one of two characters is erroneously recognized as the other character for each of the first notation and the second notation And a means for extracting the word string candidates, which searches the word dictionary for a word candidate that completely matches a possible combination of the first character string and the second character string. Word matching means, similar word matching means for searching the word dictionary for similar word candidates to be similar-matched with a possible combination as the combination of the first character string and the second character string; Possibility as a combination of character string and the second character string An unknown word candidate generation unit that generates an unknown word candidate that is not registered in the word dictionary from a set, the optimum word string search unit including the word candidate, the similar word candidate, and the unknown word. For the combination of candidates, using the word dictionary, the word chain model, and the unknown word model, the word in the first notation is the word representing the word in the second notation in the first notation, and Second
The word concurrency is a word in which the word in the first notation is a word in which the word in the first notation is represented in the second notation is obtained, the character confusion probability is obtained using the character recognition device model, and the word concurrency and the character confusion are obtained. A character recognition error correction device characterized by selecting a word string that maximizes the product of probabilities. 6. The unknown word model, which gives a probability that an arbitrary character string of the first notation and a character string of the second notation which are not registered in the word dictionary form a word, is the first notation of the word. A word type determination means for classifying an arbitrary word into any one of word types defined based on the type of characters constituting the, and an average word length of the first notation and an average word of the second notation of each word type Word length probability calculation means for calculating a joint probability of the length of the first notation and the length of the second notation from the length, a chain of two or more characters of the first notation, and two of the second notations A word notation probability calculation means for calculating the appearance probability of the character string of the first notation and the appearance probability of the character string of the second notation from the chain of characters described above, and the type of characters that make up the first notation of the word A character string of the first notation and a sentence of the second notation for each word type classified based on The character recognition error correction device according to claim 5, further comprising unknown word probability calculation means for estimating a probability that a set of character strings forms a word. 7. A first character string in the first notation and a second character string in the second notation obtained by character-recognizing the character string in the first notation and the character string in the second notation. A character recognition error correction program of a set, wherein the character string in the first notation is a character string in which the character string in the second notation is represented in the first notation, and the character string in the second notation is A language model that gives a word coincidence probability that is a character string in which the character string of the first notation is represented by the second character, and one of the two characters is the other for the first character and the second character. Character recognition program model which gives a character confusion probability of being erroneously recognized by the character, and the language model and the character recognition prepared for the memory function for the set of the first character string and the second character string. The word string candidates are extracted using the program model, and the word strings are extracted. By selecting a word string with the highest probability from the candidates, the computer is made to realize an optimum word string search function for acquiring the most probable set of the character string of the first notation and the character string of the second notation. A character recognition error correction program characterized by the following. 8. A first character string in the first notation and a second character string in the second notation obtained by character-recognizing the character string in the first notation and the character string in the second notation. A character recognition error correction program of a set, the memory function for storing a model, the first character string by the first notation and the second notation using the model set in the memory function. A function of extracting a word string candidate of the second character string set by the above, and a model with the highest probability are selected from the extracted word string candidates by using the model set in the storage function. Thus, the optimal word string search function for acquiring the most probable set of the character string of the first notation and the character string of the second notation is realized by the computer, and the model stored in the storage function is Remember a set of first and second notations that represent the same word A word dictionary, a word chain model that gives the appearance probabilities of two or more chains of a first notation and a second notation group of words, and a character string of any first notation registered in the word dictionary And the unknown word model that gives the probability that the character string of the second notation constitutes a word, and one of the two characters is erroneously recognized as the other character for each of the first notation and the second notation A character recognition program model that gives a character confusion probability, and a function of extracting the word string candidates is a word that completely matches a set that is possible as a combination of the first character string and the second character string. A word matching function that searches candidates from the word dictionary, and a similar word that searches the word dictionary for similar words that are similar-matched with a possible combination of the first character string and the second character string The collation function, the first character string and the first character string An unknown word candidate generation function for generating an unknown word candidate that is not registered in the word dictionary from a set that is possible as a combination of the character strings in the optimal word string search function. For the combination of similar word candidates and the unknown word candidates, the word in the first notation is changed to the word in the second notation using the word dictionary, the word chain model, and the unknown word model. And the second word
The word concurrency is a word in which the word in the first notation is the word in which the word in the first notation is represented in the second notation is obtained, the character confusion probability is obtained using the character recognition program model, and the word concurrency and the character confusion are obtained. A character recognition error correction program characterized by selecting a word string that maximizes the product of probabilities. 9. The unknown word model, which gives a probability that a character string of a first notation and a character string of a second notation that are not registered in the word dictionary form a word, is the first notation of the word. A word type determination function for classifying an arbitrary word into any one of word types defined based on the type of characters constituting the word, and an average word length of the first notation and an average word of the second notation of each word type A word length probability calculation function for calculating a joint probability of the length of the first notation and the length of the second notation from the length, a chain of two or more characters of the first notation, and a second notation of 2
A word notation probability calculation function that calculates the appearance probability of the first notation and the appearance probability of the second notation from a chain of more than one character, and the character that composes the first notation of the word. An unknown word probability calculation function for estimating the probability that a set of a character string in the first notation and a character string in the second notation configures a word for each word type classified based on the type. A character recognition error correction program according to item 8. 10. A character recognition error correction device, which receives a result of character recognition of a set of kanji notation and kana notation as an input and corrects a character recognition error during this input, for an arbitrary kanji string and kana string. , A kanji string and kana string simultaneous probability, that is, the kanji string is the kanji notation of the kana string and the kana string is the kana notation of the kanji string. Confusion probability, that is,
Based on the character recognition device model that gives the probability that one character is erroneously recognized by the other character, and based on the language model and the character recognition device model, it is the most Equipped with an optimal character string search means for finding a combination of Kanji and Kana notation with a high probability, and by utilizing the redundancy that the same contents are expressed in Kanji and Kana by associating Kanji and Kana. And a character recognition error correction device characterized by correcting an error in character recognition. 11. A Japanese character recognition error correction device, which receives a result of character recognition of a set of kanji notation and kana notation as an input, and corrects a character recognition error during this input, which comprises the kanji notation and kana notation. A character matrix for each, that is, a list of character candidates arranged in descending order of character recognition score at each character position as an input, a word dictionary that stores pairs of kanji notation and kana notation of words, and kanji notation of words And a word ngram model that gives the probability of ngram occurrence of a set of kana notation, an unknown word model that gives the probability that a combination of an arbitrary kanji string and kana string that is not registered in the word dictionary form a word, and any two characters For, the character confusion probability, that is,
A character recognition device model that gives the probability that one character is erroneously recognized by the other character, and a word dictionary that completely matches the set of kanji strings and kana strings included in the kanji character matrix and kana character matrix. A word matching means for searching words, a similar word matching means for searching words in a word dictionary for similar matching with a set of kanji strings and kana strings contained in a character matrix for kanji notation and a character matrix for kana notation, and kanji notation Unknown word candidate generating means for generating a candidate of an unknown word, that is, an unknown word, from a set of a kanji string and a kana string included in the character matrix and the kana character notation character matrix, and the word matching means. From among the combination of the word searched by the above-mentioned word, the word searched by the similar word matching means, and the unknown word generated by the unknown candidate generating means. Based on Dell and the character recognition device model, it is equipped with an optimal word string search means that finds the word string with the highest probability for the input character matrix pair of kanji and kana notation. By associating, the same contents are expressed in Kanji and Kana notation, and by using the redundancy, errors that cannot be corrected by either Kanji or Kana notation are corrected and the input is registered in the dictionary. It is possible to estimate the occurrence probability of a set of kanji string and kana string based on an unknown word model even if it contains a word that is not included, and to search word candidates by similar word matching even if the correct character is not included in the character matrix. Character recognition error correction device. 12. An unknown word model that gives a probability that a combination of an arbitrary kanji string and a kana string that is not registered in a word dictionary forms a word was defined based on the type of characters that form the kanji notation of the word. A word type determination means that classifies any word into any of the word types, a word type probability calculation means that obtains the occurrence probability of a word type from the appearance frequency of the word type, and an average word in kanji and kana notation for each word type From the long
A word length probability calculating means for obtaining the joint probability of the length of the kanji notation and the length of the kana notation, the character ngram frequency of the kanji notation and the character ngr of the kana notation
word notation probability calculation means for obtaining the probability of occurrence of a character string in kanji notation and the probability of occurrence of a character string in kana notation from am frequency, and the words classified based on the type of characters that make up the kanji notation of the word The character recognition error correction device according to claim 11, further comprising an unknown word model for estimating a probability that a set of a kanji string and a kana string constitutes a word for each type.