CN110580516A - An interactive method and device based on an intelligent robot - Google Patents

Abstract

the invention relates to the technical field of human computers, in particular to an interaction method based on an intelligent robot, which comprises the following steps: acquiring interactive data in real time, wherein the interactive data comprises character data and scene data; identifying the character data and the scene data to obtain an identification tag set; based on the identification tag set, performing matching calculation with a knowledge graph to obtain a scene relation corresponding to the identification tag set, and further deducing and obtaining a concept phenomenon corresponding to the scene relation based on the knowledge graph; determining interactive output content based on the conceptual phenomena; and acquiring a corresponding computer interaction instruction according to the interaction output content, and executing. The automatic scene judgment and feedback of the intelligent robot can be realized, and the human input is reduced. In addition, the invention also provides an intelligent robot device and a computer readable storage medium.

Description

An interactive method and device based on an intelligent robot

technical field

The present invention relates to the field of computer information technology, in particular to an interactive method based on an intelligent robot, an intelligent robot device and a computer-readable storage medium.

Background technique

With the rapid development of computer technology, intelligent robots have been widely used in various industries, especially in the field of human interaction.

At present, the interaction form of most robots on the market is voice interaction, which mainly uses pre-set question and answer pairs (question/answer pairing) of the nature of instructions and some PDA entertainment interaction functions (video playback, video games, etc.) to realize the input information. Interactive output.

For such a robot, each scenario based on product positioning requires a large number of experts and programmers to write preset interactive content, but this kind of hard-coded fixed interactive content is relatively rigid and has poor scalability. It is a very unscientific design for industries (such as computers and mobile phones), and the user experience is very bad.

Contents of the invention

In view of the above problems, an embodiment of the present invention provides an interaction method based on an intelligent robot, the method includes the steps of: collecting interaction data in real time, the interaction data including character data and scene data; analyzing the character data and the Scene data is identified to obtain a set of identification labels; based on the set of identification labels, matching calculation is performed with the knowledge map to obtain the scene relationship corresponding to the set of identification labels, and further based on the knowledge map derivation to obtain the corresponding scene relationship Conceptual phenomenon; determining interactive output content based on the conceptual phenomenon; obtaining and executing corresponding computer interactive instructions according to the interactive output content.

The intelligent robot-based interactive method provided by the present invention collects and identifies the data in the interactive scene in real time to obtain a multi-dimensional identification label set, and further performs matching calculations through the identification label set and the knowledge graph to determine the identification The scene relationship mapped by the label set can not only realize the automatic recognition of the scene relationship, but also the recognition result is more accurate; further, after determining the scene relationship corresponding to the recognition label set, it is necessary to further derive the hidden scene relationship. Conceptual phenomenon, so as to provide more accurate interactive output content, so that intelligent robots can automatically judge the psychological needs of characters according to the scene, so as to provide reasonable and intelligent services, reduce manual intervention, and reduce labor costs.

At the same time, the present invention also provides an intelligent robot device, the device includes a data collection module, an interactive output module, and a data processing module respectively connected to the data collection module and the interactive output module, wherein the data The acquisition module is used to collect interaction data in real time, and send the interaction data to the data processing module; the data processing module receives the interaction data, and identifies the interaction data to obtain an identification label set; based on the The recognition label set is matched with the knowledge map to obtain the scene relationship corresponding to the recognition label set, and further based on the knowledge map to derive the concept phenomenon corresponding to the scene relationship, determine the interactive output content, and The interactive output content is sent to the interactive output module; the interactive output module acquires and executes corresponding computer interactive instructions based on the interactive output content.

And, a computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the above-mentioned interaction method based on an intelligent robot is realized.

In an implementation, the interaction data includes multimodal data acquired through audio and video acquisition equipment and GPS.

In one implementation, the identifying the character data and the scene data to obtain the identification label set specifically includes: determining the identity label of the person according to face recognition and voiceprint recognition; The recognition model outputs the emotional label of the character; outputs the action label of the character according to the computer vision and the second deep learning model; derives the dialogue content of the character in the form of text according to speech recognition, and performs character intention recognition based on the dialogue content of the character to obtain the intention label; according to the computer Vision and the third deep learning model output object labels; output event labels according to computer vision and the fourth deep learning model; determine the time when the event occurred based on the robot’s own system time; speculate on the location and scene of the event based on GPS positioning and map information ; And at the same time determine the confidence of each of the identification tags.

In an implementation, the matching calculation is performed with the knowledge map based on the identification tag set to obtain the scene relationship corresponding to the identification tag set, which specifically includes:

Calculate the weight of each of the identification tags in the knowledge map respectively;

Based on the weight and the confidence degree, calculate the matching degree between the recognition label set and the scene relationship in the knowledge graph;

Based on the matching degree, the scene relationship corresponding to the identification tag set is determined.

In one implementation, the method for separately calculating the weight of each of the identification tags in the knowledge graph specifically includes:

The weights of each of the identification tags in the knowledge map are obtained based on the following mathematical formula:

Among them, wij represents the weight of i identification label in j scene relationship; tfij indicates the occurrence number of i identification label in j scene relationship; dfi indicates the number of scene relations containing i identification label; N indicates the total number of scene relations.

In one implementation, the method for calculating the matching degree between the recognition label set and the scene relationship in the knowledge graph based on the weight and the confidence degree specifically includes:

The matching degree of the recognition tag set and the scene relationship in the knowledge map is obtained based on the following mathematical formula:

Wherein, Ps represents the matching degree of the identification tag in the scene relation s; n represents the total number of tag slots in the scene relation s; Xn represents the confidence level corresponding to the tag n in the knowledge map in the recognition tag set.

In one implementation, after determining the scene relationship corresponding to the identification tag set, it further includes the step of: determining a vacant tag slot in the scene relationship based on the identification tag set, and Sets fill the vacant label slots.

In one implementation, after determining the scene relationship corresponding to the identification tag set, it further includes the step of: constructing a character model based on the identification tag set and its mapping relationship in the knowledge graph, and using the graph database way to store.

Description of drawings

One or more embodiments are exemplified by the pictures in the corresponding drawings, and these exemplifications do not constitute a limitation to the embodiments. Elements with the same reference numerals in the drawings represent similar elements. Unless otherwise stated, the drawings in the drawings are not limited to scale.

FIG. 1 shows a flow chart of an interactive method based on an intelligent robot provided in the first embodiment of the present invention;

FIG. 2 is a schematic diagram of the structure of the knowledge graph fragments in the first embodiment of the present invention;

FIG. 3 is a schematic diagram of the structure of the intelligent robot device provided by the second embodiment of the present invention.

Detailed ways

In order to make the purpose, technical solutions and advantages of the embodiments of the present invention more clear, the following will describe each embodiment of the present invention in detail with reference to the accompanying drawings. However, those of ordinary skill in the art can understand that, in each implementation manner of the present invention, many technical details are provided for readers to better understand the present application. However, even without these technical details and various changes and modifications based on the following implementation modes, the technical solution claimed in this application can also be realized.

In the first embodiment of the present invention, an interactive method based on intelligent robots is proposed, which can realize the intelligent reasoning brain of intelligent robots based on the technical framework of deep integration of deep learning algorithms and knowledge graph technology, without investing a large number of experts Preset the interactive content of each scene with programmers, and directly based on the knowledge map technology, it can enable robots to have more intelligent interactive content of the scene, so that service robots can have professional knowledge and professional interactive content.

Please refer to FIG. 1 for details. FIG. 1 is a flowchart of an interaction method based on an intelligent robot provided by the first embodiment of the present invention. As shown in Figure 1, the method specifically includes steps:

Step 101, collect interaction data in real time.

In this embodiment, the multi-modal data of the interactive scene can be obtained in real time through the audio and video collection equipment and GPS, and the collected data can specifically include: images, voice, audio, video, geographic location, system time, etc. interaction data in . The interaction data can be mainly divided into character data and scene data. The specific data collection device is not particularly limited in this embodiment of the present invention.

Step 102, identifying the interaction data to obtain an identification tag.

In order to obtain more accurate information from the collected interaction data, the interaction data can be classified first, and the interaction data can be divided into character data and scene data. The identification of interactive data is actually the identification of character data and scene data.

Specifically, in a specific interaction scenario, character data can include information such as identity information, emotions, actions, intentions, etc., and intelligent robots can identify these information and obtain corresponding labels. Specifically, it can include:

(1) Determine the identity label of the person based on face recognition and voiceprint recognition;

(2) According to computer vision and the first deep learning model, semantic recognition model output character emotion label;

(3) output character action labels according to computer vision and the second deep learning model;

(4) Deriving character dialogue content in text form based on speech recognition, and performing character intention recognition based on the character dialogue content to obtain intent tags.

In this way, a corresponding person identification tag can be obtained based on the person data.

Similarly, scene data can also be identified to obtain corresponding labels, specifically including:

(1) Output object labels according to computer vision and the third deep learning model. The so-called objects refer to physical objects that appear in interactive scenes, such as footballs, cars, musical instruments, etc., which can be used to assist in confirming interactive scenes to improve the accuracy of recognition .

(2) Output event tags based on computer vision and the fourth deep learning model, where events refer to the description of what happened in the interaction scene, such as crossing the road, waiting for a bus, picking up things, dragging luggage, etc., which can also be used to assist in confirming interactions Scenes.

(3) Determine the time at which the event occurs based on the system time of the robot.

(4) According to GPS positioning and map information, infer the scene where the event occurred.

The above-mentioned label recognition based on the deep learning model can also be realized based on the convolutional neural network (CNN) algorithm, and, while outputting the corresponding identification label, the confidence of each label can be output at the same time, please refer to Table 1 Schematic description:

Label identification Identify label 1 (confidence level) Identify Label 2 (Confidence) Identify label 3 (confidence level) … emotional label crying (44%) Angry (23%) Happy (2%) action label Push and pull (74%) Shake (15%) Running (4%) intent label Want to go home (59%) Looking for a mom (28%) Want to eat (5%)

Table 1

The confidence corresponding to each identification tag can be used for subsequent matching calculations. It is worth noting that the data used for matching calculations can include all identification results of each identification tag, and the identification results with higher confidence can also be preliminarily screened out for use in the matching calculation. Subsequent calculations to reduce the amount of calculations.

Step 103, based on the recognition tag set, perform matching calculation with the knowledge map to obtain the scene relationship corresponding to the recognition tag set.

Specifically, the knowledge graph contains multiple scene relationships, and each scene relationship contains a corresponding label slot, for example:

There are different labels in the scene relationship in the knowledge graph

Scenario 1: S2(D), S3(G), S4(I)

Scenario 2: S1(B), S3(F), S5(K)

The content in the label slot (Sn) can correspond to the identified label. For example, they all correspond to character emotional labels. The scene definition of the initial knowledge graph generally does not fill all the slots in order to improve compatibility and applicability ( Sn), so some of the slot contents are empty.

The matching calculation of the knowledge graph may include two processes. First, calculate the weight of each of the identification tags in the knowledge graph, and then, based on the weight and the confidence, calculate the recognition tag set and the scene in the knowledge graph. The matching degree of the relationship, and based on the matching degree, determine the scene relationship corresponding to the identification tag set.

It is understandable that since it is impossible to guarantee that the expression of the identification tags of all recognition outputs is consistent with the expression of the knowledge graph entity, it is necessary to go through coreference resolution before the matching calculation to improve the matching accuracy.

The following will describe the above two processes in detail:

First, the method of calculating the weight of each identification tag in the knowledge map includes:

The weights of each of the identification tags in the knowledge map are obtained based on the following mathematical formula:

Among them, w _ij represents the weight of i identification tag in scene relation j; tf _ij represents the number of occurrences of i identification tag in j scene relation; df _i represents the number of scene relations containing i identification tag; N represents the total number of scene relations.

It can be seen that in this embodiment, the weight assignment (Wn) of each identification tag is carried out according to its term frequency (TermFrequency) in the knowledge graph. If the knowledge map output label slot is empty, the weight is 0.

Word frequency is a statistical method used to evaluate how important a word (label) is to a set of documents (knowledge graph) or to one of the documents (scene) of a corpus. The importance of identifying a tag increases proportionally with its frequency of occurrence in the scene, but at the same time decreases inversely proportional to its frequency of occurrence in the entire knowledge graph. That is to say, when the identification tag appears in the knowledge graph, it has a weight value, and the higher the number of occurrences, the greater the weight may be, but at the same time it needs to be set in combination with the frequency of its appearance in the entire knowledge graph, In other words, the higher the frequency of appearance in the entire knowledge graph, the less likely it is to uniquely determine the scene relationship. In this embodiment, word frequency is used to quantify and measure the importance of a label in a scene, and the obtained result is more reasonable.

Then, the matching degree of the recognition label set and the scene relationship in the knowledge map can be obtained based on the following mathematical formula:

Among them, P _s represents the matching degree of the recognition tag in the scene relationship s; n represents the total number of label slots in the scene relation s; X _n represents the confidence of the label n in the knowledge map in the recognition label set .

Through the above method, the matching degree between the recognition label set and each scene relationship in the knowledge map can be calculated.

Generally speaking, at least 2 tags in the scene relationship need to be included in the recognition tag set, and the interaction will be performed. If there are less than 2 tags, the scene relationship cannot be matched, and no interaction will be output.

After the matching degree with each scene relationship is obtained, a scene relationship with the highest matching degree can be selected and determined as the scene relationship corresponding to the recognition tag set.

It should be noted that, in order to ensure the accuracy of the matching result, a threshold can be set to prevent false matching.

Specifically, after obtaining the matching degree of each scene relationship, first determine the scene relationship whose matching degree is greater than the preset threshold, and select the scene relationship with the highest matching degree, and determine it as the scene relationship corresponding to the recognition tag set. If all If the matching degrees are all less than the preset threshold, it can be determined that there is no matching result, and there is no scene relationship corresponding to the label set in the knowledge graph. In this case, the default interactive output content can be selected as the current interactive output content. Among them, the default interactive Output content can be preset and stored locally.

Step 104, deriving and obtaining the concept phenomenon corresponding to the scene relationship based on the knowledge graph.

Step 105, determine interactive output content based on the concept phenomenon.

The knowledge graph in this embodiment may include a general knowledge graph and a professional knowledge graph, where the professional knowledge graph may be determined based on the application scenario of the intelligent robot. In this embodiment, the prior knowledge and reasoning function of the knowledge graph can be used to perform knowledge reasoning to determine the conceptual phenomenon corresponding to the scene relationship. As shown in FIG. 2 , FIG. 2 is a schematic diagram of the structure of knowledge map fragments in the first embodiment of the present invention. After identifying the confirmed scene relationship between the label set and the entity label of the knowledge map [Crying in the Park], you can use the prior knowledge and reasoning function of the knowledge map to perform knowledge reasoning, and get the conceptual phenomenon behind it from the scene relationship reasoning [Separation Anxiety] , and further output the corresponding interactive output content [playing games] through the preset knowledge content on the knowledge graph.

Step 106, obtain corresponding computer interaction instructions according to the interactive output content, and execute them.

After the interactive output content is determined, the preset computer interactive instruction can be queried and executed according to the interactive output content, so as to realize the interactive output. For example, when the interactive output content is "playing a game", the interactive game program preset by the system can be invoked and run to realize the interactive output.

It can be seen that the above method collects and recognizes the data in the interactive scene in real time to obtain a multi-dimensional identification label set, and further performs matching calculations between the identification label set and the knowledge map to determine the mapping of the identification label set. The scene relationship can not only realize the automatic recognition of the scene relationship, but also the recognition result is more accurate; further, after determining the scene relationship corresponding to the recognition label set, it is necessary to further derive the implicit concept phenomenon in the scene relationship, so that Provide more accurate interactive output content, so that intelligent robots can automatically judge the psychological needs of characters according to the scene, so as to provide reasonable and intelligent services.

In one implementation, after confirming the scene relationship corresponding to the identification label set through the above step 103, the vacant label slots in the scene relationship can be determined based on the identification label set, and supplemented based on the identification label set. Describe the vacant label slot. In the process of robot interaction, the content of the knowledge map is continuously improved by matching scene relationships, and content is added to the vacant slots in the knowledge map through successful matching, so as to continuously improve the information in the knowledge map and improve the accuracy of matching.

Furthermore, after the intelligent robot outputs the interaction, the other label matching fed back by the robot's perceptron can be used as the result verification, and the knowledge map can be supplemented as a complete process matching (such as the original emotion label of separation anxiety is sadness, and the perception after the output interaction The emotion label is happy).

At the same time, based on the identification tag set and its mapping relationship in the knowledge map, a character model can be constructed and stored in the form of a graph database. If a character stores content:

Taking the successful recognition of the scene relationship as the generated event, record the time, place, action label, emotion label, intent label, scene label, event (knowledge graph entity).

The character model record can be used for: the construction of artificial intelligence Internet of Things in the cognitive field with the educational service robot as the terminal, and the collection of specific data records of professional knowledge triggered by the unit character with the educational service robot as the terminal. It can be used as first-hand research data for scholars and researchers, and collecting through robot terminals can reduce a lot of manpower and standardize data standards. The artificial intelligence Internet of Things (including smart terminals) can be used to conduct more valuable research topics such as children's behavior observation and analysis, and provide a data basis for robot developers and designers to optimize product content.

Based on the same inventive concept, an embodiment of the present invention also provides a customer service auxiliary device. For details, please refer to FIG. 3 , which is a schematic structural diagram of an intelligent robot device provided by the second embodiment of the present invention.

As shown in FIG. 3 , the intelligent robot device 300 includes a receiving data collection module 301, an interactive output module 303, and a data processing module 302 that is communicatively connected to the data collection module 301 and the interactive output module 303, respectively,

Wherein, the data collection module 301 is used to collect interaction data in real time, and send the interaction data to the data processing module 302;

The data processing module 302 receives the interaction data, and identifies the interaction data to obtain a recognition tag set; based on the recognition tag set, performs matching calculation with the knowledge map to obtain the scene relationship corresponding to the recognition tag set , and further obtain the conceptual phenomenon corresponding to the scene relationship based on the knowledge graph, determine the interactive output content, and send the interactive output content to the interactive output module 303 . For the identification of the interaction data by the data processing module 302 and the matching calculation method, please refer to the description in the embodiment of FIG. 1 , and details will not be repeated here.

The interactive output module 303 obtains and executes corresponding computer interactive instructions based on the interactive output content.

It should be noted that: the intelligent robot device provided by the above-mentioned embodiments can be implemented based on computer programs, and the division of the above-mentioned functional modules is only used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional modules according to needs. That is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above.

The intelligent robot device provided in this embodiment has basic cognitive thinking ability, can actively react to the scene based on multi-modal interactive data + knowledge graph inference, instead of commanding input and output, and can also modularize human Computer interactive content, reducing a lot of design work.

Yet another embodiment of the present invention relates to a computer-readable storage medium storing a computer program. When the computer program is executed by the processor, the above-mentioned embodiment of the interaction method based on the intelligent robot is realized.

Those skilled in the art can understand that all or part of the steps in the method of the above-mentioned embodiments can be completed by instructing related hardware through a program, the program is stored in a storage medium, and includes several instructions to make a device (which can be A single chip microcomputer, a chip, etc.) or a processor (processor) executes all or part of the steps of the methods in the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disk, and other media that can store program codes.

The above descriptions are only preferred embodiments of the present invention, and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention shall be included in the protection of the present invention. within range.

Claims (10)

1. A kind of interactive method based on intelligent robot, it is characterized in that, described method comprises steps: Collecting interactive data in real time, the interactive data includes character data and scene data; Identifying the character data and the scene data to obtain an identification tag set; Based on the identification tag set, perform matching calculation with the knowledge map to obtain the scene relationship corresponding to the identification tag set, and further derive and obtain the concept phenomenon corresponding to the scene relationship based on the knowledge map; determining interactive output content based on the conceptual phenomenon; Obtain corresponding computer interaction instructions according to the interactive output content, and execute them. 2. The method according to claim 1, wherein the interaction data includes multimodal data obtained by audio and video collection equipment and GPS. 3. The method according to claim 1, wherein said character data and scene data are identified to obtain an identification tag set, which specifically includes: Determine the identity label of the person based on face recognition and voiceprint recognition; According to the computer vision and the first deep learning model, the semantic recognition model outputs the emotional label of the character; Output character action labels according to computer vision and the second deep learning model; Deriving character dialogue content in text form according to speech recognition, and performing character intent recognition based on the character dialogue content to obtain intent labels; output object labels according to computer vision and the third deep learning model; Output event labels according to computer vision and the fourth deep learning model; Determining the time at which the event occurs based on the system time of the robot itself; According to GPS positioning and map information, infer the scene where the event happened; And at the same time determine the confidence level of each identification tag. 4. The method according to claim 3, wherein the matching calculation is performed with the knowledge map based on the identification tag set to obtain the scene relationship corresponding to the identification tag set, specifically comprising: Calculate the weight of each of the identification tags in the knowledge map respectively; Based on the weight and the confidence degree, calculate the matching degree between the recognition label set and the scene relationship in the knowledge map; Based on the matching degree, the scene relationship corresponding to the identification tag set is determined. 5. The method according to claim 4, wherein the method for calculating the weight of each of the identification tags in the knowledge graph specifically comprises: The weights of each of the identification tags in the knowledge map are obtained based on the following mathematical formula: Among them, w _ij represents the weight of i identification tag in scene relation j; tf _ij represents the number of occurrences of i identification tag in j scene relation; df _i represents the number of scene relations containing i identification tag; N represents the total number of scene relations. 6. The method according to claim 4 or 5, wherein, based on the weight and the degree of confidence, the method for calculating the degree of matching between the identification label set and the scene relationship in the knowledge graph specifically comprises: The matching degree of the recognition tag set and the scene relationship in the knowledge map is obtained based on the following mathematical formula: Among them, P _s represents the matching degree of the recognition tag in the scene relationship s; n represents the total number of label slots in the scene relation s; X _n represents the confidence of the label n in the knowledge map in the recognition label set . 7. The method according to claim 1, characterized in that, after determining the scene relationship corresponding to the identification tag set, further comprising the steps of: A vacant label slot in the scene relationship is determined based on the identification label set, and the vacant label slot is supplemented based on the identification label set. 8. The method according to claim 1, characterized in that, after determining the scene relationship corresponding to the identification label set, further comprising the steps of: Based on the identification tag set and its mapping relationship in the knowledge map, a character model is constructed and stored in a graph database. 9. An intelligent robot device, characterized in that, the device comprises a data acquisition module, an interactive output module, and a data processing module that is communicatively connected to the data acquisition module and the interactive output module respectively; Wherein, the data collection module is used to collect interactive data in real time, and send the interactive data to the data processing module; The data processing module receives the interaction data, and identifies the interaction data to obtain an identification tag set; based on the identification tag set, performs matching calculation with the knowledge map to obtain the scene relationship corresponding to the identification tag set, And further deriving and obtaining the concept phenomenon corresponding to the scene relationship based on the knowledge graph, determining the interactive output content, and sending the interactive output content to the interactive output module; The interactive output module acquires and executes corresponding computer interactive instructions based on the interactive output content. 10. A computer-readable storage medium storing a computer program, characterized in that, when the computer program is executed by a processor, the intelligent robot-based interaction method according to any one of claims 1 to 8 is implemented.