JP2001117930A - Document classification device, document classification method, and recording medium - Google Patents

Abstract

(57) [Summary] [PROBLEMS] To provide a document classification device and a document classification method capable of classifying and organizing documents for each topic. In a document classification method for automatically classifying a document set according to the contents of the document, words in each document data of a document set including a plurality of documents are analyzed (S
2) Based on the analysis result, a document feature vector and a word feature vector expressed in the same feature space are generated (S
3) The document set is classified into a plurality of document subsets for each feature dimension based on a value obtained by projecting the generated document feature vector to each feature dimension (S4), and the word group is similarly assigned to each document dimension. Classification into a plurality of word subsets for each feature dimension (S6)
The classification result is displayed (S8), and a document subset for performing a set operation between a plurality of document subsets is designated using the displayed classification result (S9). The calculation is performed by the specified calculation method (S1
0).

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a document classifying apparatus for automatically classifying a group of documents into a plurality of document subsets according to the contents of the documents. The present invention relates to a document classification method and a recording medium that stores a program for executing the method.

[0002]

2. Description of the Related Art With the spread of the Internet and the like, it has become possible to access a large amount of document information.
It is necessary to classify a large amount of collected document information into meaningful groups so that the desired document information can be accessed efficiently and the analysis work of a large set of documents can be performed efficiently. Sex is growing. However, if a user manually classifies a large amount of document information, human / time costs become enormous. Therefore, in recent years, apparatuses capable of automatically classifying a set of documents according to the contents of the documents have been provided. In such automatic classification, for example, by extracting a plurality of words constituting them from a document using natural language processing such as Japanese morphological analysis, the document is classified into a vector of the frequency of occurrence of the plurality of words (document feature). Vector). This technique is called a vector space model of a document, and is widely used. In such a vector space model, the similarity between vectors can be defined by calculating the distance, inner product, cosine, and the like between any two document feature vectors in the space. And various automatic document classification methods have been provided (for example, the invention described in JP-A-7-114572). Hereinafter, such a conventional technique will be described with reference to the drawings.

First, the appearance frequency of each word is counted for each document, and a document feature vector that quantitatively represents the feature of each document is obtained based on the counted frequency. FIG. 10 shows an example of calculating a document feature vector of each document data in a classification example in which 12 document data are classified. The number of components of the vector is the number of types of all words occurring in the group of document data to be classified. Here, the number of components is reduced to a three-dimensional vector using the co-occurrence relation of words. In this document classification method, the document classification is performed by applying a chi-square method, a discriminant analysis method, a cluster analysis method, or the like to such a document feature vector. FIG. 11 shows an example of a classification result in a case where 12 pieces of document data are classified into three document subsets using the document feature vectors of the document data. By applying, for example, the Ward method, which is one of the cluster analysis methods, to the quantitative three-dimensional vectors of the document data, those having similar feature vector values can be grouped and classified into three document subsets. it can. That is, each document data belongs to one of the three document subsets (clusters) as shown in FIG. Note that the representative value shown in FIG. 11A is the average value of the feature vectors of the belonging document data (the center of gravity of the belonging document data). Further, a distance between a feature vector of the document data and a representative value of the document subset to which the document data belongs is determined as a feature value indicating a rank (similarity order) relationship in the document subsets of the document data belonging to each document subset. .
An example of obtaining the distance of the document data 13 belonging to the cluster (document subset) 12 will be described below. ((3.00−2.66) ² + (2.00−2.00) ² + (4.00−3.66)
² ) ^1/2 = 0.48 The distance shown in FIG. 11B was obtained in this way. The smaller the distance from the representative value of the document subset to which it belongs, the higher the similarity with the average document belonging to the document subset. In this way, an automatic document classification technology for collecting documents having high similarity with respect to each individual document (from the beginning to the end of the document) has been established.
In this conventional technique, it is difficult to focus on various topics included in a set of documents to be classified and to classify each topic. In other words, according to the above-described related art, since the documents are classified according to the similarity of the entire document, documents having a combination of a plurality of topics are classified into the same document subset.

[0004]

As described above, the prior art has a problem that it is difficult to classify documents for each topic. An object of the present invention is to solve such a problem of the conventional technology and to provide a document classifying apparatus and the like which can classify and arrange documents by topic.

[0005]

According to the first aspect of the present invention, there is provided a document classification apparatus for automatically classifying a document set according to the contents of the document. Document analysis means for analyzing words in the respective document data, feature vector generation means for generating a document feature vector expressed in a feature space based on the analysis result by the document analysis means, and a generated document feature vector Document classification means for classifying a document set into a plurality of document subsets for each feature dimension based on a value obtained by projecting the document classification into each feature dimension, and a classification result presentation means for presenting a document classification result by the document classification means And a set interaction for designating a document subset for performing a set operation between a plurality of document subsets using the classification result presented by the classification result presentation means. A designation unit, and a set inter-operation means for performing operation between documents subset specified by the predetermined calculation method by the set operation between designating means. Claim 2
In the invention described in the first aspect, the feature vector generation unit is configured to generate a word feature vector expressed in the same feature space as the document feature vector based on the analysis result by the document analysis unit, Word classification means for classifying a group of words into a plurality of word subsets based on a value obtained by projecting the generated word feature vector to each feature dimension, and combining the word classification result by the word classification means with the document classification result. The classification result presenting means is configured to be presented. According to a third aspect of the present invention, in the first or second aspect of the present invention, the inter-set operation specifying means is configured to specify the operation method, and the inter-set operation means is operated to perform the operation by the operation method. Was configured. According to a fourth aspect of the present invention, in the first, second, or third aspect, an inter-set operation is performed such that a document subset is specified by specifying a feature dimension used for document classification. The designation means has been configured. According to a fifth aspect of the present invention, in the invention according to any one of the first to fourth aspects, an operation result is stored as a new operable document subset as a document subset as an operation result by the inter-set operation means. A storage means was provided. According to a sixth aspect of the present invention, in the invention according to any one of the first to fifth aspects, the strength of the commonality of the elements between the plurality of document subsets is calculated for all combinations of the document subsets. It is provided with an inter-commonality calculation means.

[0006] According to the invention of claim 7, according to claim 1,
The invention according to any one of the first to sixth aspects, further comprises a specific subset presentation unit for presenting a document subset including one or a plurality of designated documents. In the invention according to claim 8, in a document classification method for automatically classifying a document set according to the contents of the document, words in each document data of the document set including a plurality of documents are analyzed, and the analysis result is obtained. Generates a document feature vector expressed in the feature space based on the generated document feature vector and classifies the document set into a plurality of document subsets for each feature dimension based on a value obtained by projecting the generated document feature vector to each feature dimension. And present the document classification results,
Using the presented classification result, a document subset for causing a set operation between a plurality of document subsets is specified, and the operation between the specified document subsets is performed by a predetermined calculation method. According to a ninth aspect of the present invention, in the invention of the eighth aspect, a word feature vector expressed in the same feature space as the document feature vector is generated based on the analysis result, and The word group is classified into a plurality of word subsets based on the values obtained by projecting the feature dimensions, and the word classification result is presented together with the document classification result. According to the tenth aspect, in the eighth aspect,
Alternatively, according to the ninth aspect of the present invention, an operation method is specified, and the operation is performed according to the specified operation method. Also,
According to the eleventh aspect, in the eighth, ninth, or tenth aspect, a document subset is specified by specifying a feature dimension used for document classification. According to a twelfth aspect of the present invention, in the invention of any one of the eighth to eleventh aspects, the document subset generated as a result of the inter-set operation is stored as a new operable document subset. . Claim 13
According to the invention described in any one of claims 8 to 12, the strength of the commonality of the elements between the plurality of document subsets is calculated for all combinations of the document subsets. According to a fourteenth aspect of the present invention, in any one of the eighth to thirteenth aspects, a document subset including one or a plurality of designated documents is presented. The invention according to claim 15 relates to a storage medium storing a program, and stores a program for causing a computer to execute the document classification method according to any one of claims 8 to 14. Features.

According to the first and eighth aspects of the present invention, a word in each document data of a document set composed of a plurality of documents is analyzed, and the feature is determined based on the analysis result. A document feature vector expressed in space is generated, and the document set is classified into a plurality of document subsets for each feature dimension based on a value obtained by projecting the generated document feature vector to each feature dimension. When a document classification result is presented and a document subset for causing a set operation to be performed between a plurality of document subsets using the presented classification result is specified, the operation between the specified document subsets is performed by a predetermined operation method. Done in In the invention described in claim 2 and claim 9, in the invention described in claim 1 or claim 8,
A word feature vector expressed in the same feature space as the document feature vector is generated based on the analysis result, and the word group includes a plurality of words based on a value obtained by projecting the generated word feature vector to each feature dimension. It is classified into a subset, and the word classification result is presented together with the document classification result. According to the third and tenth aspects of the present invention, in the first, second, eighth, or ninth aspect of the invention, when an operation method is specified, the operation between the specified document subsets is specified. It is performed by the calculated operation method. Claim 4 and Claim 11
According to the invention described in any one of claims 1 to 3, or 8 to 10, the document subset is designated by designating a feature dimension used for document classification. According to the fifth and twelfth aspects of the present invention, in the first to the fourth aspects or the eighth to the eleventh aspects, the document subset generated as an inter-set operation result can be newly operated. Stored as a document subset. According to the sixth and thirteenth aspects of the present invention, in the first to the fifth aspects or the eighth to the twelfth aspects of the present invention, the strength of the commonality of the elements between a plurality of document subsets is determined by the document part. Calculated for all combinations of sets. According to the seventh and fourteenth aspects of the present invention, a document subset including one or a plurality of designated documents is presented in the first to the sixth aspects or the eight to the thirteenth aspects. You. According to the invention described in claim 15, by reading a program included in a recording medium into a computer, the document classification device according to any one of claims 1 to 7 can be realized, and the document classification method according to claims 8 to 14 can be realized. Can be implemented. Therefore, the recording medium can be easily distributed and sold as a software product independently of the device or as a set with the device.

[0008]

DESCRIPTION OF THE PREFERRED EMBODIMENTS In the present invention, a document feature vector having words included in a set of documents to be classified as feature amounts is used.
By applying singular value decomposition to the document-word matrix, each of the potential semantic spaces generated based on the co-occurrence of words in the document (the property that when word A occurs, word B also occurs), etc. By classifying documents into multiple subsets for each feature dimension, assuming that the dimensions (feature dimensions) represent topics included in the content of the document, the documents are classified for each topic included in the content of the document Then, the user performs a set operation such as sum or product using the document subset obtained as a result of the classification, and generates a new document subset by the set operation, thereby classifying documents based on topics.・ Be able to organize things. That is, in the present invention, each dimension (feature dimension) in the latent semantic space is a conceptual one in which a plurality of words are semantically reduced, and expresses a topic composed of a plurality of words. When classifying a document or word by projecting a document feature vector or a word feature vector onto a certain feature dimension, for example, if the value of a certain feature dimension is a large positive value in the document feature vector or the word feature vector, the document Alternatively, the word is considered to be strongly related to the topic represented by the feature dimension. On the other hand, if the value of the feature dimension is 0
If it is near, it is considered that the relevance between the document or word and the topic represented by the feature dimension is unknown, and if the value of the feature dimension is a large negative value, the document or word has the feature It is considered that there is no relevance to the topic represented by the dimension. In the singular value decomposition, the document-word matrix is decomposed into a plurality of matrices including a matrix L composed of singular values. The singular value is a value generated by singular value decomposition. For example, a document including a large number of words that appear in many documents in common has a high value in the dimension of the matrix L having a high singular value.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. FIG. 1 is a functional block diagram of the document classification device according to the embodiment of the present invention. As shown in the figure, a document classification device according to this embodiment includes a system control unit 1 that controls and manages the entire device, a data processing unit 2 that performs various processes related to document classification, and a document that constitutes a document set to be classified. An input unit 3 for inputting data and instruction information; a storage device (for example, a hard disk device) 4 for storing various data such as the document data and the classification result information;
A display device 5 for displaying classification result information and the like is provided. The system control unit 1 and the data processing unit 2 have a shared or dedicated memory (for example, RAM) for storing programs and various data, and a shared or dedicated CPU that operates according to the programs. A document analysis unit that analyzes words in each document data of a document set including a plurality of documents; a document feature vector and a word feature vector expressed in the same feature space based on an analysis result by the document analysis unit; , A document classifying means for classifying a document set into a plurality of document subsets based on a value obtained by projecting the generated document feature vector to each feature dimension, and a generated word feature vector for each feature dimension. A data classification unit for classifying a group of words into a plurality of word subsets on the basis of the values projected to the document set, and an inter-set operation unit for performing an operation between designated document subsets by a predetermined operation method. Is achieved. A classification result presenting means for presenting the document classification result information by the document classification means and the word classification result information by the word classification means comprises a system control unit 1, a data processing unit 2, and a display device 5, and the presented classification result is displayed. An inter-set operation designating means for designating a document subset for performing a set operation between a plurality of document subsets using information includes a system control unit 1, a data processing unit 2, and an input unit 3.

FIG. 2 shows an operation flow of the first embodiment of the present invention. The operation of this embodiment will be described below with reference to FIG. First, the input unit 3 inputs document data of documents belonging to a set of documents to be classified (S
1). The input unit 3 includes a keyboard, a removable storage medium,
A network interface unit and the like are provided, and a document data group is input using any of them, and the system control unit 1 stores the input document data group in the storage device 4.
It is stored in. Next, the data processing unit 2 operating as a document analysis unit performs a natural language analysis on each of the input document data, and extracts each word (for example, a noun) included in each of the document data, A word identification code (ID) is assigned to the word, and the number of occurrences of the word is counted for each document (S2). Subsequently, the data processing unit 2, which also operates as a feature vector generation unit, generates a vector expressing each document and word as a document feature vector and a word feature vector in the same feature space (S3). For example, by creating a document-word matrix using the analysis result performed in step S2 and applying singular value decomposition (factor analysis, principal component analysis, quantification theory type 3 or the like can be used), A latent semantic space is generated based on the co-occurrence of words in a document, and a document feature vector and a word feature vector expressed in the generated latent semantic space are generated. Although not described here,
There is also a method of using a space to which document co-reference information (information indicating a document that is referred to by a plurality of documents together) is added. In the above, when the number of feature dimensions of the potential semantic space is k,
For example, the document feature vector of document A is expressed as a k-dimensional vector as follows. Document feature vector of document A = (a1, a2, a3, ..., ak) The value of each dimension reflects the co-occurrence of words, and many occurrences occur in topics corresponding to each dimension. A document containing many words to be processed has a large value in that dimension. Also,
The word feature vector of word B is also expressed as a k-dimensional vector as follows. Word feature vector of word B = (b1, b2, b3, ..., bk)

Subsequently, the data processing unit 2 operating as a document classifying unit performs a known operation of projecting a document feature vector on each document into one feature dimension, thereby obtaining one feature dimension of each document (one feature dimension). (Corresponding to the topic) of the document feature vector (one-dimensional vector), and classify the document group (document set) into a plurality of document subsets according to the magnitude of the value of each document (S
4). This classification is performed for all feature dimensions. Classification methods include a clustering method (refer to “Prior Art”), a thresholding method for setting and dividing a threshold, and a one-dimensionally projected document feature vector sorted by the value. For example, a constant processing method for classifying every certain number of documents in descending order is used.
When each feature dimension (that is, for each topic) is classified into a plurality of document subsets from the viewpoint of the strength of association with each topic, the data processing unit 2 obtains the obtained document subset. Information storage device 4
It is stored in the document classification result storage unit (document classification result storage area) (S5), and is managed by assigning an ID (identification code) to each document subset. Subsequently, the data processing unit 2, which also operates as a word classification unit, projects a word feature vector for each word into one feature dimension, and forms a word group using the value of the projected word feature vector (one-dimensional vector). It is classified into a plurality of word subsets (S6). This classification is performed for all feature dimensions. As a classification method, a clustering method, a determination method by threshold processing for setting and dividing a threshold value, and one-dimensionally projected word feature vectors are sorted by the values,
A constant processing method or the like that classifies a certain number of documents in descending order of value is used. When each feature dimension (that is, for each topic) is classified into a plurality of word subsets from the viewpoint of the strength of association with each topic, the data processing unit 2 obtains the obtained word subset. The information is stored in a word classification result storage unit (word classification result storage area) in the storage device 4 (S7), and is managed by assigning an ID (identification code) to each word subset.

Thereafter, the data processing unit 2 stores the document classification result information (document subset information) stored in the storage device 4.
Display device 5 and the word classification result information (word subset information)
Is displayed (presented) (S8). FIG. 3 shows an example of display (presentation) of the classification result information. In this example, when the number of feature dimensions is 50, documents and words are classified into five subsets for each feature dimension. Here, the number of elements (documents or words) included in each subset is shown. In addition, the respective subsets are arranged, for example, in an ascending order of the value. That is, the set 1 is the set with the largest value, and the documents or words belonging to this set have the highest relevance to the topic corresponding to the dimension (feature dimension). When it is desired to browse the elements of each subset, the subset to be browsed is designated by using a mouse (constituting the input unit 3) or the like, so that the document subset or the word as shown in FIGS. You can browse the elements of the subset. FIG. 4 shows a set 1 of feature dimension 1 (in the illustrated example, the set 1 is given an identification code of ID1).
And the left column shows the document ID.
In addition, although the contents (contents) (or a part thereof) of each document are shown as the document contents, the document name may be shown. In this case, the contents are displayed when one document name is designated by a mouse or the like.
In FIG. 5, the word ID is shown in the left column.
The user sequentially displays the document classification result information (document subset information) (see FIG. 4) and the word classification result information (word subset information) (see FIG. 5) of each feature dimension as necessary. While the classification result information shown in FIG. 3 is displayed, an operation between document subsets is designated if necessary (S
9). A plurality of document subsets to be operated on and the operation method of the set operation performed between them (for example, OR or AND)
Is specified. The document subset may be specified on the screen by using a mouse or the like, or the user may directly input the document subset I by using a keyboard.
It may be realized by inputting D. The operation method of the set operation can be specified by describing (displaying) a logical expression using the specified document subset on the screen. An example is shown below. Note that the description of the document subset in this example is, for example, when the user selects a document subset (for example, dimension 1 and set 1) using a mouse or the like, or inputs “&” using a keyboard.
This is realized by the data processing unit 2 displaying it on the screen, for example, at the bottom. (Dimension 1: Set 1 [ID1]) & Dimension 49: Set 1 [ID
246])

For example, in the classification result display example of FIG. 3, when it is desired to calculate the product of all the sets classified by the feature dimension 1 (dimension 1) and the document subset (dimension 2: set 1), On the screen shown in FIG. 3, "dimension 1" is selected, "&" is input from the keyboard, "dimension 2: set 1" is selected on the screen, and the following expression is described. (Dimension 1 & Dimension 2: Set 1 [ID11]) Then, the data processing unit 2 calculates this expression as
Expands into two expressions. (Dimension 1: Set 1 [ID1] & Dimension 2: Set 1 [ID1
1]), (dimension 1: set 2 [ID2] & dimension 2: set 1
[ID11]), (dimension 1: set 3 [ID3] & dimension 2: set 1 [ID11]), (dimension 1: set 4 [ID4] & dimension 2: set 1 [ID11]), (dimension 1: set 5 [ID5] &
(Dimension 2: set 1 [ID11]) Thus, the data processing unit 2 which also operates as an inter-set operation unit performs an operation on the specified document subset using the specified operation method (S10). For example, if the operation method is a logical product, only documents in both of the specified two document subsets are extracted as elements of the new document subset. Then, the data processing unit 2 causes the storage device 4 to store the new document subset information generated as a result of the calculation (S11). Further, the data processing unit 2 causes the display device 5 to display (present) the stored document subset information as the calculated result (S12).
FIG. 6 shows a display example. Here the document subset (dimension 1:
It shows the result of a product operation between a set 1) and a document subset (dimension 39: set 1). Document subset (dimension 1:
It can be seen that three documents are commonly included in the set 1) and the document subset (dimension 39: set 1).

As described above, according to the first embodiment of the present invention, a document subset relating to a user's desired topic can be organized and generated as intended by the user, and therefore, the desired topic can be generated. It is possible to efficiently grasp the contents of a group of related documents and to easily find a desired document. Note that a configuration is also possible in which a word feature vector is obtained, a word subset is generated based on the word feature vector, and no means is provided for displaying word subset information. However, as described above, in the configuration including such means, not only the document subset information but also the word subset information is presented, so that the contents of the subset can be understood well by that amount, and therefore, a more appropriate Document subsets can be generated. Also,
A method is also possible in which the operation method is not specified by previously determining the operation method as one of a logical sum and a logical product. However, as described above, in the configuration in which the calculation method can be specified, the document subset information more matched with the user's intention can be generated. Also, in the above, the operation result by the inter-set operation means is a new operation possible document subset as an operation result storage unit in the storage device 4 (operation result storage area, corresponding to the operation result storage means described in claims).
It is also possible to have a configuration in which the information is stored.

In the second embodiment of the present invention, the data processing unit 2 calculates the degree of commonality of elements between a plurality of document subsets for all combinations of document subsets. It also operates as a means and displays the calculation result. Hereinafter, the operation of this embodiment will be described with reference to FIG. First, the input unit 3 inputs document data of documents belonging to a document set to be classified (S21).
The input unit 3 includes a keyboard, a detachable storage medium, a network interface unit, and the like. A document data group is input using any of them, and the system control unit 1 converts the input document data group. It is stored in the storage device 4. Next, the data processing unit 2 operating as a document analysis unit performs a natural language analysis on each of the input document data, and extracts each word (for example, a noun) included in each of the document data, A word identification code (ID) is assigned to the word, and the number of occurrences of the word is counted for each document (S22). Subsequently, the data processing unit 2 operating as a feature vector generating unit converts each document and word into a document feature vector in the same feature space,
A vector to be expressed as a word feature vector is generated (S23). For example, a document-word matrix is created using the analysis result performed in step S22, and singular value decomposition is applied to generate a potential semantic space based on the co-occurrence of words in the document. The document feature vector and the word feature vector expressed in the latent semantic space are generated. Subsequently, the data processing unit 2 operating as a document classifying unit performs an operation of projecting a document feature vector on each document into one feature dimension, and calculates one feature dimension (corresponding to one topic) of each document. Yes)
, The value of the document feature vector (one-dimensional vector) is obtained, and the document group (document set) is classified into a plurality of document subsets according to the magnitude of the value of each document (S24).
This classification is performed for all feature dimensions. Classification methods include a clustering method, a determination method using threshold processing for setting and dividing a threshold value, and sorting document feature vectors projected one-dimensionally according to their values. A constant processing method or the like that classifies each is used. When each feature dimension (that is, for each topic) is classified into a plurality of document subsets from the viewpoint of the strength of association with each topic, the data processing unit 2 obtains the obtained document subset. The information is stored in the document classification result storage unit (document classification result storage area) in the storage device 4 (S25), and is managed by assigning an ID (identification code) to each document subset.

Subsequently, the data processing unit 2, which also operates as a word classification unit, projects the word feature vector for each word into one feature dimension, and uses the value of the projected word feature vector (one-dimensional vector). The words are classified into a plurality of word subsets (S26). This classification is performed for all feature dimensions. Classification methods include:
A clustering method, a thresholding method for setting and dividing a threshold value into threshold values, a constant process for sorting word feature vectors projected one-dimensionally according to their values and classifying them into a certain number of documents in descending order of value Method is used. Thus, for each feature dimension (ie,
When each word is classified into a plurality of word subsets from the viewpoint of the strength of association with each topic, the data processing unit 2 stores the obtained word subset information in the word classification result in the storage device 4. It is stored in a storage unit (word classification result storage area) (S27), and each word subset has I
D (identification code) is assigned and managed. Thereafter, the data processing unit 2 causes the display device 5 to display (present) the document classification result information (document subset information) and the word classification result information (word subset information) stored in the storage device 4 (S2).
8) (see FIG. 3).

Next, the data processing unit 2, which also operates as an inter-set commonality calculating means, automatically calculates the commonality of elements between a plurality of document subsets for all combinations of document subsets (S29). For example, in order to calculate the degree of commonality of elements between two document subsets, a product set operation may be performed on the two document subsets. Since all document subsets are managed by identification codes (IDs), if there are 50 document subsets, the data processing unit 2 performs
The document subset operation is performed on all combinations of the two subsets for the 50 document subsets. (Document subset ID1 & document subset ID2), (document subset ID1 & document subset ID3), (document subset ID1 & document subset ID4), ..., (document subset ID49 & document subset ID50) Thus, for example, the number of commonly included documents, the ratio of the number of commonly included documents to the number of documents included in the document subset to be subjected to the commonness calculation, and the like are determined as commonality. Further, the data processing unit 2 stores the calculated commonality between the document subsets in the storage device 4 (S
30). Next, the data processing unit 2 displays (presents) the stored commonness calculation result on the display device 5 (S31). FIG. 8 shows a display (presentation) example. In this display example, the ratio of the number of documents included in common to the number of documents included in the document subset for which the common degree is to be calculated is used as the common degree, and displayed in descending order of the common degree. Also, a list of documents included in common and a list of documents included in the target document subset can be displayed. FIG. 8
By referring to the commonality calculation result as shown in (1) and instructing the operation between document subsets as shown in the first embodiment, a more appropriate document subset can be efficiently generated. It is. Further, those having a common degree equal to or higher than a predetermined value may be automatically collected into one document subset without the intervention of the user.

According to a third embodiment of the present invention, a specific part comprising a system control unit 1, a data processing unit 2, an input unit 3, a display device 5, etc. in the first and second embodiments. The system includes a set presentation unit, and can present a document subset including one or a plurality of designated documents. Hereinafter, the operation of this embodiment will be described with reference to FIG. First, the user specifies one or a plurality of documents (S41). For example, the classification result information as shown in FIG. 3 is displayed, and the document subset information as shown in FIG. 4 is displayed, and the document ID is selected by a mouse or the like, or the user directly inputs the document ID. Or specify it. Next, a document subset including the specified document is extracted by searching the specified document from the plurality of document subsets (S42). At this time, when a plurality of documents are specified, a document subset including all the specified documents is extracted. When a document subset including the specified document is extracted, a list of identification codes (ID) of the document subset is displayed (S43). Also,
It is also possible to browse the contents of the displayed document subset. Thus, according to this embodiment, it is possible to understand what topic set the document subset including the document is based on the document, and to determine the relationship between the document subsets through the included document. You can understand. In the above, the document classification method of the present invention has been described for the case of the document classification device having the configuration shown in FIG. 1, but a program programmed according to such a document classification method is stored in, for example, a removable storage medium, and the storage medium Is loaded into an information processing apparatus such as a personal computer which has not been able to classify documents according to the present invention, so that the information processing apparatus can also classify documents according to the present invention.

[0019]

As described above, according to the present invention,
According to the first and eighth aspects of the present invention, a word in each document data of a document set including a plurality of documents is analyzed, and a document feature vector expressed in a feature space is generated based on the analysis result. The document set is classified into a plurality of document subsets for each feature dimension based on a value obtained by projecting the generated document feature vector to each feature dimension, the document classification result is presented, and the presented classification result is presented. When a document subset for causing a set operation to be performed between a plurality of document subsets is specified using, the operation between the specified document subsets is performed by a predetermined calculation method. Can be organized, so that it is possible to efficiently grasp the contents of a document group on a desired topic,
The desired document can also be easily found. According to the second and ninth aspects of the present invention, a word feature vector expressed in the same feature space as the document feature vector is generated based on the analysis result in the first or the eighth aspect of the invention, The word group is classified into a plurality of word subsets based on the value obtained by projecting the generated word feature vector to each feature dimension, and the word classification result is presented together with the document classification result. Understand well,
Therefore, a more appropriate document subset can be generated, the contents of a document group on a desired topic can be grasped more efficiently, and a desired document can be found more easily. Claims 3 and 1
In the invention described in Item No. 0, in the invention described in Claim 1, Claim 2, Claim 8, or Claim 9, when the operation method is specified, the operation between the specified document subsets is performed by the specified operation method. This makes it possible to generate a document subset that more closely matches the user's intention, so that the contents of a document group on a desired topic can be grasped more efficiently, and the desired document can be found more easily. it can.
In the inventions according to claims 4 and 11, in the inventions according to claims 1 to 3, or in claims 8 to 10, the document subset is designated by designating the feature dimension used for document classification. Is specified, the operation for performing the operation between the document subsets is simplified.

According to the fifth and twelfth aspects of the present invention, in the first to the fourth aspects or the eighth to the eleventh aspects of the present invention, the document subset generated as a result of the inter-set operation is Since the document subset is stored as a new operable document subset, the document subset can be used for the subsequent operation between document subsets, and therefore, a document subset that further matches the user's intention can be generated. it can. According to the invention described in claim 6 and claim 13, in the invention described in claim 1 to claim 5 or in claim 8 to claim 12, the strength of the commonality of elements between a plurality of document subsets is reduced. Since the calculation is performed for all combinations of document subsets, the user can instruct the operation between document subsets with reference to the calculation result, and thus efficiently generate a more appropriate document subset. It is also possible to automatically combine documents having a commonality equal to or more than a predetermined value into one document subset without the user's instruction. Also, in the invention described in claim 7 and claim 14, according to claim 1 to claim 6, or claim 8
In the invention according to the thirteenth aspect, a document subset including one or a plurality of designated documents is presented,
Based on the document, it is possible to understand what topic set is included in the document subset including the document, and it is also possible to understand the relationship between the document subsets via the included document. Also, in the invention according to claim 15, a program programmed according to the document classification method according to claims 8 to 14 can be stored in, for example, a removable storage medium.
The effects of the inventions according to claims 8 to 14 can also be realized in an information processing apparatus such as a personal computer in which the document classification according to the inventions of claims 14 to 14 cannot be performed. Can be.

[Brief description of the drawings]

FIG. 1 is a configuration block diagram of a document classification device according to each embodiment of the present invention.

FIG. 2 is an operation flowchart of a document classification method according to the first embodiment of the present invention.

FIG. 3 is a display diagram of a document classification method according to the first embodiment of the present invention.

FIG. 4 is another display diagram of the document classification method according to the first embodiment of the present invention.

FIG. 5 is another display diagram of the document classification method according to the first embodiment of the present invention.

FIG. 6 is another display diagram of the document classification method according to the first embodiment of the present invention.

FIG. 7 is an operation flowchart of a document classification method according to the second embodiment of the present invention.

FIG. 8 is a display diagram of a document classification method according to a second embodiment of the present invention.

FIG. 9 is an operation flowchart of a document classification method according to the third embodiment of the present invention.

FIG. 10 is an explanatory diagram of a document classification method showing an example of a conventional technique.

FIG. 11 is another explanatory diagram of a document classification method showing an example of the related art.

[Explanation of symbols]

1 system control unit (classification result presenting means, inter-set operation specifying means) 2 data processing unit (document analyzing means, feature vector generating means, document classifying means, word classifying means, inter-set operation means, classification result presenting means, between sets Calculation designation means, commonality calculation means between sets) 3 input unit (computation designation means between sets) 4 storage device 5 display device (classification result presentation means)

Claims (15)

[Claims] 1. A document classification apparatus for automatically classifying a document set according to the contents of the document, wherein the document analysis means analyzes words in each document data of the document set including a plurality of documents. A feature vector generating means for generating a document feature vector expressed in a feature space based on the analysis result; and a document set for each feature dimension based on a value obtained by projecting the generated document feature vector onto each feature dimension. A document classification means for classifying the document into a plurality of document subsets; a classification result presentation means for presenting a document classification result by the document classification means; and a plurality of document subsets using the classification result presented by the classification result presentation means. An inter-set operation specifying means for specifying a document subset for performing a set operation between the document subset and a document subset specified by the inter-set operation specifying means. A document classification device comprising: an inter-set operation means for performing an operation by a predetermined operation method. 2. The document classification device according to claim 1, wherein
The feature vector generation unit is configured to generate a word feature vector expressed in the same feature space as the document feature vector based on the analysis result by the document analysis unit, and project the generated word feature vector to each feature dimension. Word classification means for classifying a group of words into a plurality of word subsets based on the values obtained in step (a), and the classification result presentation means is configured to present the word classification result by the word classification means together with the document classification result. Characteristic document classification device. 3. The document classification apparatus according to claim 1, wherein the inter-set operation specifying means is configured to specify an operation method, and the inter-set operation means is configured to perform the operation by the operation method. Document classification apparatus characterized by the above-mentioned. 4. The document classifying apparatus according to claim 1, wherein the inter-set operation specifying means is configured to specify a document subset by specifying a feature dimension used for the document classification. A document classification device characterized by comprising. 5. The document classification apparatus according to claim 1, wherein the operation result storing means for storing a document subset as an operation result by the inter-set operation means as a new operable document subset. A document classification device, comprising: 6. The document classification apparatus according to claim 1, wherein the degree of commonality of elements between a plurality of document subsets is calculated for all combinations of the document subsets. A document classification device comprising a calculation unit. 7. The document classification apparatus according to claim 1, further comprising a specific subset presenting unit for presenting a document subset including one or a plurality of designated documents. Document classification device. 8. A document classification method for automatically classifying a document set according to the content of the document, analyzing words in each document data of the document set including a plurality of documents, and storing the words in a feature space based on the analysis result. Generate a document feature vector to be represented, classify the document set into a plurality of document subsets for each feature dimension based on a value obtained by projecting the generated document feature vector onto each feature dimension, and classify the document The result is presented, a document subset for performing a set operation between a plurality of document subsets using the presented classification result is designated, and the operation between the designated document subsets is performed by a predetermined computation method. A document classification method, characterized in that: 9. The document classification method according to claim 8, wherein
Generates a word feature vector expressed in the same feature space as the document feature vector based on the analysis result, and projects the generated word feature vector onto each feature dimension to form a group of words based on a value obtained. A document classification method characterized by classifying into a subset and presenting a word classification result together with a document classification result. 10. The document classification method according to claim 8, wherein an operation method is designated, and the operation is performed according to the designated operation method. 11. The method of claim 8, 9, or 1,
0, wherein a document subset is specified by specifying a feature dimension used for the document classification. 12. The document classification method according to claim 8, wherein the document subset generated as a result of the inter-set operation is stored as a new operable document subset. Classification method. 13. The document classification method according to claim 8, wherein a strength of commonality of an element between a plurality of document subsets is calculated for all combinations of the document subsets. Document classification method to be used. 14. The document classification method according to claim 8, wherein a document subset including one or a plurality of designated documents is presented. 15. A computer-readable recording medium on which a program for causing a computer to execute the document classification method according to claim 8 is recorded.