KR20230068920A - Hybrid psychological analysis method based on picture and an apparatus - Google Patents

Abstract

The present invention relates to a hybrid psychological test method and device based on paintings. The method comprises the steps of: providing a painting board module to obtain image information and image metadata on the paintings of a counselee; applying an object included in the image information to a CNN-based model to divide the same into a plurality of segments according to morphological characteristics and extracting a spatial attribute for each segment; applying to an RNN-based model based on a line, a coordinate, and time included in the metadata to extract a temporal attribute; and integrating the extracted spatial and temporal attribute to calculate a psychological diagnosis result of the counselee. In addition, the accuracy of a psychological test can be improved by using a hybrid method of CNN analysis of images and RNN analysis of the image metadata.

Description

Psychological test method and apparatus of hybrid method based on picture

The present invention relates to a psychological test method and apparatus that analyzes a picture drawn by a counselee in a hybrid method, and more specifically, provides an online canvas and painting tools to the counselee, analyzes the temporal and spatial properties of the picture drawn by the counselee, provide inspection.

Psychological testing measures the mental and psychological state of an individual to objectively quantify or qualitatively analyze the invisible psychological state, and is used as a means for diagnosing individual psychological symptoms.

Recently, as the number of patients with mental disorders increases socially, the demand for psychological tests is increasing.

Psychological testing is known by using the psychological picture test (HTP), which is a kind of projection test, and questionnaires.

The above-mentioned psychological picture test (HTP) requires counselors with expertise in various fields such as art, psychology, clinical practice, and statistics. There is a downside to being able to do this, and there is a downside of requiring too much time to actually look for numerous books and materials for picture analysis. In addition, if accurate analysis results do not come out depending on the condition of the counselor, or if the counselor suffers from heavy counseling and treatment tasks, he or she cannot concentrate on the examination and suffers a lot of difficulties.

On the other hand, counselees have prejudice against psychological counseling, fear of self-disclosure, burden of counseling costs, or geographical and temporal limitations.

Accordingly, there is a growing demand for a method for enabling non-face-to-face online psychological counseling and solutions without directly visiting a counseling center for psychological counseling, and a solution to the above problems is urgently needed.

For reference, prior art documents related to the present invention include Patent Registration No. 10-1772987 (2017.08.31.).

The present invention automatically diagnoses the counselee's psychology with high accuracy without the help of an expert in a hybrid method in which an image of the counselee is acquired by an electronic device and CNN analysis and analysis of the obtained image is performed.

One embodiment according to the present invention is a hybrid psychological test method based on a picture by an electronic device, the psychological test method comprising the steps of providing a picture board module to obtain image information about a counselee's picture and metadata of the image; dividing an object included in the image information into a plurality of segments according to morphological characteristics by applying a CNN-based model, and extracting spatial attributes for each segment; Extracting temporal attributes by applying an RNN-based model based on lines, coordinates, and time included in the metadata, and extracting temporal attributes; and calculating a psychological diagnosis result of the counselee by integrating the extracted spatial and temporal attributes.

In one embodiment, the step of extracting the spatial attributes performs a local attention algorithm to calculate weights for features appearing in each segment of each segment, and features appearing in the whole rather than part of the object. After performing the global attention algorithm to calculate the weight for , it is characterized in that the first result value (wCNN) is output through a fully connected layer.

In one embodiment, the extracting of the temporal attribute may include performing a plurality of convolution operations on each element of line, coordinates, and time included in the metadata, and then performing LSTM (Long short short term memory) algorithm, local attention algorithm is performed to calculate weights for features in each segment, global attention algorithm is performed to calculate weights for features in the entire object, and softmax operation is applied to calculate weights for features in each segment. It is characterized by outputting the result value (wRNN).

In an embodiment, the step of calculating the psychological diagnosis result by integrating the extracted spatial and temporal attributes includes a first result value calculated based on the spatial attribute and a second result value calculated based on the temporal attribute. It is characterized in that a psychological diagnosis result is output by multiplying and summing each weight by a preset weight.

In one embodiment, the psychological diagnosis result is a diagnosis result of at least one of psychology, personality, disposition, family relationship, sociality, and familialization of the counselee.

In one embodiment, when the object is a tree, the plurality of segments are roots, stems, and leaves, the line includes a drawing direction, line thickness, and line type, and the coordinates include a path, shape, and size, The time is characterized in that it includes the drawing progress duration of each segment.

In one embodiment, the paint module provides an online canvas and a drawing tool for the counselee to draw a picture, and enables the designation of the outline of the object, the color of the object, the thickness of the outline, and the color according to the selection of the drawing tool. to be characterized

According to another embodiment of the present invention, a picture-based psychological testing apparatus of a hybrid method includes a picture acquiring unit that acquires image information about a picture of a counselee and metadata of the image by providing a picture board module; a segment detection module for dividing the object included in the image information into a plurality of segments according to morphological characteristics by applying a CNN-based model; a classification module for extracting spatial attributes for each partitioned segment; an RNN analysis module for extracting temporal attributes by applying an RNN-based model based on lines, coordinates, and time included in the metadata, and extracting temporal attributes; and an integration module for integrating the extracted spatial and temporal attributes to calculate a psychological diagnosis result of the counselee.

According to an embodiment disclosed in the present invention, by automating the psychological test, it not only plays an auxiliary role in the expert's judgment, but also can strengthen the expert's judgment, and can perform public service through the Internet, mobile, etc.

In addition, the accuracy of the psychological test can be improved by a hybrid method of CNN analysis of images and RNN analysis of metadata of images.

1 is a block diagram for explaining the configuration of a hybrid psychological test apparatus based on a picture according to an embodiment of the present invention.
2 is a block diagram for explaining the configuration of a control unit according to an embodiment of the present invention.
3 is a conceptual diagram for explaining the concept of a hybrid psychological test method based on a picture according to an embodiment of the present invention.
4 is a flowchart of a hybrid psychological test method based on a picture according to an embodiment of the present invention.

Since the present invention can make various changes and have various embodiments, specific embodiments will be illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the present invention to specific embodiments, and should be understood to include all modifications, equivalents, and substitutes included in the spirit and scope of the present invention. Like reference numerals have been used for like elements throughout the description of each figure.

Terms such as first, second, A, and B may be used to describe various components, but the components should not be limited by the terms. These terms are only used for the purpose of distinguishing one component from another. For example, a first element may be termed a second element, and similarly, a second element may be termed a first element, without departing from the scope of the present invention. The terms and/or include any combination of a plurality of related recited items or any of a plurality of related recited items.

Terms used in this application are only used to describe specific embodiments, and are not intended to limit the present invention. Singular expressions include plural expressions unless the context clearly dictates otherwise. In this application, the terms "include" or "have" are intended to designate that there is a feature, number, step, operation, component, part, or combination thereof described in the specification, but one or more other features It should be understood that the presence or addition of numbers, steps, operations, components, parts, or combinations thereof is not precluded.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which the present invention belongs. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the related art, and unless explicitly defined in the present application, they should not be interpreted in an ideal or excessively formal meaning. don't Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram for explaining the configuration of a hybrid psychological test apparatus based on a picture according to an embodiment of the present invention.

The hybrid psychological test apparatus 100 based on a picture may include a picture acquisition unit 120, a display unit 130, a control unit 140, and a storage unit 150. The hybrid psychological test device 100 based on the picture may further include an information input unit 110 .

The information input unit 110 converts the user's input operation into an input signal and transmits it to the control unit 140 . The information input unit 110 may be implemented as, for example, a keyboard, a mouse, a touch sensor on a touch screen, a touch pad, a keypad, a voice input, and other input processing devices that are currently available in the past or will be available in the future. For example, the information input unit 110 may transmit the counselor's attitude and basic information to the controller 140 after receiving an input. The basic information may include personal information of the counselee and the type of picture drawn by the counselee.

The personal information may include any one or more of name, gender, age, contact information, and disability, and types of pictures may include cats, trees, fish tanks, and the like.

The picture acquisition unit 120 may include a picture board module. The Paint module is a module to provide an online canvas for the counselee to draw a picture, and through this module, a picture image in the form of electronic data can be drawn without paper or a pencil, and the drawn picture image can be checked. In the paint module, a background to be drawn and a tool to draw a drawing image may be displayed. The counselee can use the tool to designate points, lines, planes, and colors, and the outline of the object, the color of the object, and the thickness of the outline. , specify the color, erase and redraw the drawn object. In this case, all steps of drawing or erasing a picture may be stored and transmitted to the controller 140 . The paint module may acquire metadata such as the shape of the picture as well as the temporal location where the picture is drawn, the thickness of the line, the type of the line, and the duration, and transmit the acquired metadata to the controller 140 .

The picture acquisition unit 120 may be implemented as a device and a terminal separate from the picture-based hybrid psychological test device 100 . In this case, the image acquisition unit 120 and the hybrid psychological test apparatus 100 based on the picture may be wired or wirelessly connected to transmit image information acquired by the picture acquisition unit 120 to the control unit 140 .

The display unit 130 outputs a screen under the control of the controller 140 . The display unit 130 may be implemented as, for example, a liquid crystal display (LCD), a light emitting diode (LED), an organic light emitting diode (OLED), a projector, or other display devices currently available in the past or to be available in the future. The display unit 130 may display, for example, an interface page for inputting information or a page for providing information. Also, the display unit 130 may display, for example, a background to be drawn provided by the drawing board module, a tool to draw a drawing image, and the like.

The control unit 140 controls the overall operation and each component of the hybrid psychological test apparatus 100. In particular, the control unit 140 displays the interface page, the paint background and tools through the display unit 130 through the information input unit 110 and the picture acquisition unit 120, as will be described later, and the information input unit 110 and the picture acquisition unit According to the information received from 120, a psychological diagnosis result of the counselee is derived based on the counselee's picture, and a psychological diagnosis result page is generated and displayed through the display unit 130. of the control unit 140

Operations performed by the control unit 140 may be distributed and processed by several physically separated computing devices. In this case, the control unit 140 may be implemented as an aggregate of physically separated computing devices.

The storage unit 150 stores data under the control of the control unit 140 and transmits the requested data to the control unit 140 . For example, the storage unit 150 stores pictures obtained through the information input unit 110 and the picture acquisition unit 120 . It can also store CNN analysis models and RNN analysis modules.

2 is a block diagram for explaining the configuration of a control unit according to an embodiment of the present invention.

Referring to FIG. 2 , the control unit 140 may include a segment detection module 142, a classification module 144, a recurrent neural network (RNN) analysis module 146, and an integration module 148.

The segment detection module 142 analyzes the picture drawn by the counselee obtained through the picture acquisition unit 120 using an object detection method. For convenience of description, objects include houses, cats, trees, etc. included in the picture drawn by the counselee.

More specifically, the segment detection module 142 divides the object into a plurality of segments, extracts attributes for each segment, and classifies all attributes by summing the attributes for each segment.

The segment detection module 142 applies an object of a target picture to a pre-stored segmentation module and segments it into segments with characteristics. For example, in the case of a house, at least one or more segments of a roof, a wall, a door, and a window may be partitioned. As another example, a tree may be segmented into segments including leaves, pillars, and roots.

The pre-stored segmentation module may be a convolutional neural network (CNN)-based model.

For reference, the CNN-based model is composed of a convolutional layer and a pooling layer. The convolutional layer plays a role in extracting features from the input data, and consists of a filter that extracts features from the convolutional layer and an activation function that changes the value of the filter to a nonlinear value. It consists of

In the convolutional layer, a filter is applied to the input data, and a feature map emphasizing a region of the image similar to the filter is output and passed to the next layer.

The output feature map shows the result of applying a filter to an input such as an input image or another feature map. Through the visualization of a feature map for a specific input image, features of the input are detected or preserved in the feature map.

Here, the filter used is a matrix having a smaller size than the size of the original image, and for example, an edge filter kernel or a blur filter kernel may be used. At this time, if the original image is multiplied while moving the filter, a feature map can be extracted. It includes a plurality of convolutional layers consecutively. After passing through such a continuous convolution layer, it passes through a pooling layer to reduce the amount of data and computation. In this case, the pooling layer compresses the feature map by making data as large as the size of the pooling kernel (eg, n×m) in the feature map into one data.

The CNN-based module may obtain a final image labeled according to a plurality of segments by passing the acquired image through a convolution layer and a pooling layer. That is, according to the CNN-based module, the object of the target picture may be partitioned into a plurality of segments.

The classification module 144 extracts attributes for each partitioned segment from the segment detection module 142 .

In one embodiment, the classification module 144 extracts a feature map of each segment and places local attention on the extracted feature map.

Then, the classification module 144 performs a local attention algorithm and a global attention algorithm for each segment. Here, the local attention algorithm improves the identification result by emphasizing individual features of the segment, and the global attention algorithm improves the identification result by reinforcing the feature of the part that is more important for interpretation among the whole features.

The RNN analysis module 146 calculates an RNN result value by applying the input metadata to a pre-stored RNN model. Metadata may largely include lines, coordinates, and time. Here, the line includes the drawing progress direction, line thickness, line type, etc., the coordinates include the path, shape, and size, and the time includes the drawing progress duration of each segment. For example, the duration may include a duration for drawing roots of a tree, a duration for drawing trunks, a duration for drawing leaves, and the like.

The integration module 148 calculates a psychological diagnosis result by adding preset weights to the first result value of the classification module 144 and the second result value of the RNN analysis module 146 .

3 is a conceptual diagram for explaining the concept of a hybrid psychological test method based on a picture according to an embodiment of the present invention.

Referring to FIG. 3 , when a signal is input from the paint module, the image data input from the segment detection module 142 is divided into a plurality of segments based on a pre-stored segmentation CNN model, and then each segment is classified in the classification module 144. Extract star attributes. A CNN model may be a classification algorithm including a convolutional layer and a fully connected layer (FCL). Each segment can be partitioned by calculating as many results as the number of labels in the output layer through the classification algorithm and deriving the label with the highest score from the calculated results as the final result.

The classification module 144 extracts characteristic attributes for each segment based on the classification CNN model. The classification CNN model performs a local attention algorithm to calculate weights for features appearing in each segment, performs a global attention algorithm to calculate weights for features appearing in the whole rather than parts of an object, and calculates full weights for each segment. A first result value (wCNN) is output by forming a complete connection between layers through a Fully Connected Layer (FCL).

For example, if the object is a tree, each segment is divided into roots, stems, and leaves, and in the properties of the roots, the root characteristics of whether or not the roots are expressed, the uniformity of whether one side of the root is highlighted, and the properties of the stem In, whether the top and bottom of the stem match, whether the bottom is expressed wider than the top, the degree of whether the lines are connected, the pair form of the branches whether or not the branches are paired, and the three dimensions of whether the branches are expressed in three dimensions At least one of the counselee's psychology, personality, disposition, family relationship, sociality, and familialization is determined by spatial attributes, such as expression characteristics and leaf attributes, whether tree leaves are expressed or not, and fruit characteristics whether fruits are displayed. Either one may be determined and the first result value wCNN may be output.

Meanwhile, when a signal is input from the paint module, metadata generated based on the input signal is applied to a pre-stored RNN model in the RNN analysis module 146 to output an analysis result.

The RNN model performs multiple convolution operations on metadata and then goes through a Long Short Term Memory (LSTM) algorithm. At this time, convolution and LSTM algorithm are performed for each element of metadata, local attention algorithm is performed to calculate weights for features in each segment, global attention algorithm is performed to calculate weights for features in the entire object, and A second result value (wRNN) is extracted by applying the softmax operation. For example, if the object is a tree, which part of the root, stem, or leaf was drawn first, which part was drawn the longest (duration), the drawing direction of the root or stem, and the size of the root, stem, or leaf. The second result value (wRNN) may be output by determining at least one of psychology, personality, disposition, family relationship, sociability, and familialization of the counselee according to temporal attributes such as and size ratio.

In one embodiment, the local attention algorithm may include an operation by Equation 1.

[Equation 1]

는 로컬 어텐션 가중치이고,

는 특징값이고,

는 완전 연결 레이어이며,

는 최종 소프트맥스 함수값이다. here,

is the local attention weight,

is a feature value,

is a fully connected layer,

is the final softmax function value.

Also, the global attention algorithm may include an operation according to Equation 2.

[Equation 2]

는 글로벌 어텐션 가중치이고,

는 i번째 엘리먼트(패치)의 특징값이며,

는 i번째 엘리먼트(패치)의 특징에 대한 로컬 어텐션 가중치이다. here,

is the global attention weight,

Is the characteristic value of the i-th element (patch),

is a local attention weight for the feature of the i-th element (patch).

The first result value (wCNN) and the second result value (wRNN) are input to the integration module 148, and the first result value (wCNN) and the second result value are calculated by Equation 3 in the integration module 148. The psychological diagnosis result, which is the final result, is calculated by summing the values obtained by multiplying (wRNN) by different preset weights.

[Equation 3]

4 is a flowchart of a hybrid psychological test method based on a picture according to an embodiment of the present invention. The hybrid psychological test method based on the picture of FIG. 4 may be performed by the psychological test device described in FIGS. 1 to 3 .

In step S110, the psychological test apparatus provides a picture board module to acquire image information and metadata of the counselee's picture.

The counselee can draw a picture on the online canvas by selecting the drawing tool provided by the Paint module and designating the color of the drawing pen, the thickness of the line drawn by the pen, the color of the line, and the color of the drawn object. Through the process described later through the picture drawn on the online canvas, the counselee can receive highly accurate psychological test results non-face-to-face.

As the picture is drawn by the online canvas, the psychological test device creates metadata about the picture.

In step S120, the object included in the image information is applied to a CNN-based model to partition into a plurality of segments according to morphological characteristics, and spatial properties for each segment are extracted. For example, in a tree image, from where to where the roots, stems, and leaves are segmented, and each attribute for the segmented roots, stems, and leaves is extracted. For example, features such as presence or absence of roots, shape of roots, knots on stems, shape of stems, thickness, and presence or absence of leaves or fruits are extracted, and a first result value is calculated by applying a weight to each feature.

In step S130, based on the lines, coordinates, and time included in the metadata, the RNN-based model is applied to extract temporal attributes, and temporal attributes are extracted. For example, a second result value is calculated by extracting temporal features such as from which part of the tree the drawing started, which part of the tree has been painted for a long time, and in which direction the trunk extends, etc., and applying weights. .

In step S140, the psychological diagnosis result of the counselee is calculated by integrating the extracted spatial and temporal attributes.

After multiplying the first result value reflecting the spatial attribute by a predetermined first weight and multiplying the second result value reflecting the temporal attribute by the predetermined second weight value, the final result value is calculated by summing the two values. The final result value is output as a psychological truth diagnosis result.

The above description is merely an example of the technical idea of the present invention, and various modifications and variations can be made to those skilled in the art without departing from the essential characteristics of the present invention. Therefore, the embodiments implemented in the present invention are not intended to limit the technical idea of the present invention, but to explain, and the scope of the technical idea of the present invention is not limited by these embodiments. The protection scope of the present invention should be construed according to the claims below, and all technical ideas within the equivalent range should be construed as being included in the scope of the present invention.

Claims (14)

As a hybrid psychological test method based on pictures by electronic devices,
acquiring image information about a counselee's picture and metadata of the image by providing a drawing board module;
dividing an object included in the image information into a plurality of segments according to morphological characteristics by applying a CNN-based model, and extracting spatial attributes for each segment;
Extracting temporal attributes by applying an RNN-based model based on lines, coordinates, and time included in the metadata, and extracting temporal attributes; and
calculating a psychological diagnosis result of the counselee by integrating the extracted spatial and temporal attributes;
A hybrid psychological test method based on pictures including .
According to claim 1,
The step of extracting the spatial attribute,
After performing a local attention algorithm to calculate weights for features appearing within each segment of each segment, and performing a global attention algorithm to calculate weights for features appearing in the whole rather than parts of an object, , A hybrid psychological test method based on a picture, characterized in that outputting a first result value (wCNN) through a fully connected layer.
According to claim 1,
Extracting the temporal attribute,
After performing multiple convolution operations on each element of line, coordinates, and time included in the metadata, LSTM (Long short term memory) algorithm is performed, and local attention algorithm is performed to In the figure characterized in that weights for features in each segment are calculated, weights for features in the entire object are calculated by performing a global attention algorithm, and then softmax operation is applied to output the second result value (wRNN). A hybrid-based psychological test method.
According to claim 1,
Calculating a psychological diagnosis result by integrating the extracted spatial and temporal attributes,
The first result value calculated based on the spatial attribute and the second result value calculated based on the temporal attribute are multiplied by a preset weight and summed to output a psychological diagnosis result. method of psychological testing.
According to claim 1,
The psychological diagnosis result is a hybrid psychological test method based on a picture, characterized in that the diagnosis result for at least one of the counselee's psychology, personality, inclination, family relationship, sociality, and familialization.
According to claim 1,
If the object is a tree, the plurality of segments are roots, stems, and leaves,
The line includes the drawing progress direction, line thickness, and line type, the coordinates include the path, shape, and size, and the time includes the drawing progress duration of each segment. method of inspection.
According to claim 1,
The drawing board module provides an online canvas and drawing tool for the counselee to draw a picture, and enables designation of the outline of the object, the color of the object, the thickness of the outline, and the color according to the selection of the drawing tool. A hybrid-based psychological test method.
a picture acquiring unit that acquires image information and metadata of the counselee's picture by providing a picture board module;
a segment detection module for dividing the object included in the image information into a plurality of segments according to morphological characteristics by applying a CNN-based model;
a classification module for extracting spatial attributes for each partitioned segment;
an RNN analysis module for extracting temporal attributes by applying an RNN-based model based on lines, coordinates, and time included in the metadata, and extracting temporal attributes; and
an integration module that calculates a psychological diagnosis result of the counselee by integrating the extracted spatial and temporal attributes;
A hybrid psychological test device based on a picture containing a.
According to claim 8,
The classification module,
After performing a local attention algorithm to calculate weights for features appearing within each segment of each segment, and performing a global attention algorithm to calculate weights for features appearing in the whole rather than parts of an object, , A hybrid psychological testing device based on a picture characterized in that it outputs a first result value (wCNN) through a fully connected layer.
According to claim 8,
The RNN analysis module,
After performing multiple convolution operations on each element of line, coordinates, and time included in the metadata, LSTM (Long short term memory) algorithm is performed, and local attention algorithm is performed to In the figure characterized in that weights for features in each segment are calculated, weights for features in the entire object are calculated by performing a global attention algorithm, and then softmax operation is applied to output the second result value (wRNN). A hybrid-based psychological test device.
According to claim 8,
The integrated module,
The first result value calculated based on the spatial attribute and the second result value calculated based on the temporal attribute are multiplied by a preset weight and summed to output a psychological diagnosis result. type of psychological testing device.
According to claim 8,
The psychological diagnosis result is a hybrid psychological examination device based on a picture, characterized in that the diagnosis result for at least one of the counselee's psychology, personality, inclination, family relationship, sociality, and familialization.
According to claim 8,
If the object is a tree, the plurality of segments are roots, stems, and leaves,
The line includes the drawing progress direction, line thickness, and line type, the coordinates include the path, shape, and size, and the time includes the drawing progress duration of each segment. inspection device.
According to claim 8,
The drawing board module provides an online canvas and drawing tool for the counselee to draw a picture, and enables designation of the outline of the object, the color of the object, the thickness of the outline, and the color according to the selection of the drawing tool. A hybrid-based psychological test device.

Images