JP2025000882A - Information processing device, information processing method, and program - Google Patents

Abstract

To enable a user U to know that a plurality of devices in a home perform functions affecting each other only by performing a conversation with a watching robot 2 by the user when the function is executed.SOLUTION: A cloud server 1 comprises: an interface control part 31; a device operation setting information acquisition part 33; and an impact information notification part 34. The interface control part 31 performs a control of an interface containing a conversation between a user U and a watching robot 2. The device operation setting information acquisition part 33 acquires each device operation setting information of a plurality of devices which can be connected via a network N. The impact information notification part 34 notifies the user of a determination by the conversation on the basis of a control of the interface control part 31 in the case where it is determined that an execution of a predetermined function is impacted to an operation setting of the other device by referring the device operation setting information in accordance with the acquisition of information that the predetermined function is executed from the predetermined device among the plurality of devices.SELECTED DRAWING: Figure 4

Description

The present invention relates to an information processing device, an information processing method, and a program.

Conventionally, there have been dialogue systems that switch dialogue modes to avoid misrecognition due to ambient noise and smoothly complete current multi-round dialogue tasks (see, for example, Patent Document 1).

Special Publication No. 2022-554219

However, the above-mentioned Patent Document 1 does not propose a dialogue mechanism for communicating with multiple devices and informing them when executing functions that affect each other.

The present invention was made in consideration of such circumstances, and aims to enable a user to know when multiple devices are executing functions that affect each other simply by interacting with them.

In order to achieve the above object, an information processing device according to an embodiment of the present invention comprises:
an interface control means for controlling an interface including a dialogue with a predetermined user;
a device operation setting information acquiring means for acquiring device operation setting information of each of a plurality of devices connectable via a communication network;
an impact information notifying means for, in response to receiving information indicating that a predetermined function is to be executed from a predetermined device among the plurality of devices, referring to the device operation setting information and determining that the execution of the predetermined function will affect operation settings of other devices, notifying a user of that effect through the dialogue or by interrupting the dialogue under the control of the interface control means;
Equipped with.
An information processing method and a program corresponding to the information processing device according to one aspect of the present invention are also provided as an information processing method and a program according to one aspect of the present invention.

According to the present invention, a user can know when multiple devices are executing functions that affect each other simply by interacting with them.

1 shows a network configuration of an entire information processing system according to an embodiment of the present invention. 2 is a block diagram showing a hardware configuration of a cloud server in the information processing system of FIG. 1 . 2 is a block diagram showing a hardware configuration of a monitoring robot in the information processing system of FIG. 1 . 2 is a functional block diagram showing a functional configuration of a cloud server in the information processing system of FIG. 1 . 5 is a flowchart showing the operation of the cloud server of FIGS. 3 and 4 . FIG. 2 is a diagram showing a first embodiment of a living robot care service in the information processing system of FIG. 1 . FIG. 13 is a diagram showing a second embodiment of a living robot care service in the information processing system of FIG. 1 . FIG. 13 is a diagram showing a third embodiment of a living robot care service in the information processing system of FIG. 1 . FIG. 13 is a diagram showing a fourth embodiment of a living robot care service in the information processing system of FIG. 1 . FIG. 13 is a diagram showing a fifth embodiment of a living robot care service in the information processing system of FIG. 1 . FIG. 13 is a diagram showing a first example of a sixth embodiment of a living robot care service in the information processing system of FIG. 1 . FIG. 5 is a functional block diagram showing a second functional configuration of the cloud server of FIG. 4 . A figure showing a seventh embodiment of a living robot care service in an information processing system including a cloud server having the second functional configuration of Figure 12. FIG. 13 is a diagram showing an eighth embodiment of a living robot care service in an information processing system including a cloud server having the second functional configuration of FIG. 12 . A figure showing a ninth embodiment of a living robot care service in an information processing system including a cloud server having the second functional configuration of Figure 12. A figure showing a tenth embodiment of a living robot care service in an information processing system including a cloud server having the second functional configuration of Figure 12. A figure showing an eleventh embodiment of a living robot care service in an information processing system including a cloud server having the second functional configuration of Figure 12. A figure showing a twelfth embodiment of a living robot care service in an information processing system including a cloud server having the second functional configuration of Figure 12.

The following describes an embodiment of the present invention with reference to the drawings.

FIG. 1 shows the overall configuration of an information processing system according to an embodiment of the present invention.
The information processing system shown in Figure 1 is configured to be capable of communicating with each other by connecting a cloud server 1, a monitoring robot 2 (interface), a control target device 3, a user terminal 4, an emergency team terminal 5, and an external terminal 6 to each other via a predetermined network N such as the Internet.

The cloud server 1 is an information processing device that executes various processes to manage the operations of the interfaces of the monitoring robot 2 and the like, the controlled devices 3, the user terminal 4, the ambulance team terminal 5, and the external terminal 6.

Specifically, in this embodiment, the cloud server 1 manages each operation of the watching robot 2 and the user terminal 4 as follows.
That is, the cloud server 1 is managed by a service provider, such as a service operating company, that provides this service, and is operated by a person in charge of the service provider. The cloud server 1 manages various information (lifestyle information, conversation information, device operation information, specific report information, confidential information, abnormality determination method information, detection information from each sensor, etc.) obtained when the watching-over robot 2 operates the control target device 3 during a conversation between the user U and the watching-over robot 2, and executes processing based on each piece of information.
The process executed by the cloud server 1 will be described in detail later with reference to FIG.
The controlled devices 3 are some of the devices placed in the user U's home, such as a television, video, air conditioner, refrigerator, washing machine, bathroom dryer, electric kettle, etc., and are devices that are controlled by control signals from the monitoring robot 2 or the cloud server 1.
In addition, the controlled devices 3 also include devices that detect things themselves and transmit detection signals to the monitoring robot 2 or the cloud server 1, such as various sensors such as bathroom fall sensors, odor sensors, fire alarms, environmental monitoring sensors, excretion sensors, sleep vital signs/wake-up sensors, and surveillance cameras.
The user terminal 4 is an information processing device managed and operated by an elderly user U, such as an elderly person, and is managed for each user. The user terminal 4 is a mobile terminal that the user U can carry or wear to browse and input information, such as a smartphone (hereinafter referred to as "smartphone"), a tablet terminal, or a laptop computer. In addition, the user terminal 4 also includes a remote controller (hereinafter referred to as "remote control") capable of operating home appliances such as a television (hereinafter referred to as "television"), a video recorder, and lighting equipment.
The ambulance team terminal 5 is, for example, a personal computer (PC), tablet terminal, smartphone, etc., which is operated by an information processing device for processing emergency calls installed in, for example, a fire station, or an ambulance technician rushing to the scene from the fire station.
The external terminal 6 is an information processing device managed, for example, by the children or grandchildren of an elderly user U, a staff member of a service provider S (service operating company) that provides monitoring services, or a staff member in charge of small-scale delivery, and is, for example, a personal computer (PC), a tablet terminal, a smartphone, etc.

Figure 2 is a block diagram showing the hardware configuration of the cloud server in the information processing system of Figure 1.

As shown in FIG. 2, the cloud server 1 includes a CPU (Central Processing Unit) 11, a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, a bus 14, an input/output interface 15, an output unit 16, an input unit 17, a memory unit 18, a communication unit 19, and a drive 20.

The CPU 11 executes various processes according to a program recorded in the ROM 12 or a program loaded from the storage unit 18 to the RAM 13 .
The RAM 13 also stores data and the like necessary for the CPU 11 to execute various processes.

The CPU 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input/output interface 15 is also connected to this bus 14. An output unit 16, an input unit 17, a memory unit 18, a communication unit 19, and a drive 20 are connected to the input/output interface 15.

The output unit 16 is composed of a display such as a liquid crystal display, a speaker, etc., and outputs various information as images and sounds.
The input unit 17 is configured with, for example, a keyboard, and outputs various information.
The storage unit 18 is configured with a dynamic random access memory (DRAM) or the like, and stores various data.
The communication unit 19 communicates with other devices (such as the monitoring robot 2 and the external terminal 6 in FIG. 3) via a network N including the Internet.

Removable media 21, such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, is appropriately loaded into the drive 20. A program read from the removable media 21 by the drive 20 is installed in the storage unit 18 as necessary.
Further, the removable medium 21 can store various data stored in the storage unit 18 in the same manner as the storage unit 18 .

Although not shown, the user terminal 4, ambulance crew terminal 5, external terminal 6, etc. have basically the same hardware configuration as the cloud server 1 shown in FIG. 4. Therefore, explanations of the hardware configurations of the user terminal 4, ambulance crew terminal 5, external terminal 6 (e.g., a terminal of a family member of user U or a terminal of service provider S, etc.) will be omitted.

These types of hardware and software shown in Figure 2 work together to enable a series of processes related to the interaction between the user U and the monitoring robot 2.

FIG. 3 is a block diagram showing a hardware configuration of a monitoring robot in the information processing system of FIG.
As shown in FIG. 3 , the monitoring robot 2 includes a CPU (Central Processing Unit) 101, a ROM (Read Only Memory) 102, a RAM (Random Access Memory) 103, a bus 104, an input/output interface 105, an output unit 106, an input unit 107, a storage unit 108, a communication unit 109, and a drive 110.
The hardware configuration of the watching robot 2 is almost the same as the hardware configuration of the cloud server 1, but the configurations of the output unit 106 and the input unit 107 are different.
In the case of the monitoring robot 2, the body has an appearance resembling a human shape so as to facilitate natural conversation with the user U. Therefore, the microphone of the output unit 106 is located in the place corresponding to the ear, the camera in the place corresponding to the eye, the sensors in the hands and feet, the speaker and buzzer in the place corresponding to the mouth, and the display is located, for example, on the front part of the body.

Figure 4 is a functional block diagram showing the functional configuration of the cloud server in the information processing system of Figure 1.

A DB 140 including lifestyle information 141, dialogue information 142, equipment operation setting information 143, specific report information 144, abnormality determination method information 145, and secret information 146 is stored in one area of the memory unit 18 (Figure 2) of the cloud server 1.
Examples of lifestyle information 141 include planned wake-up/wake-up time, planned meal/meal time (meal start time/meal end time), meal content (menu, calories, etc.), planned activity/activity record, planned bedtime/bedtime, planned bath time (entry/exit)/bath time (entry/exit), etc.
The dialogue information 142 is voice data or text data of the contents of the conversation (both parties) in which the watching robot 2 responds to a question from the user U. If there is no limit to the storage capacity, the dialogue information 142 may include videos or images including the actions of the user U captured by the watching robot 2 during the dialogue with the user U.
The equipment operation setting information 143 includes, for example, detection settings and sensitivity settings of a bathroom fall sensor, sensitivity settings of an odor sensor, detection settings and sensitivity settings of a fire alarm, television operation settings, video recording settings, air conditioner operation settings and temperature settings, refrigerator operation settings and temperature settings, washing machine operation settings, bathroom dryer operation settings, electric kettle operation settings, etc.
Specific report information 144 is report information from a specific device, such as contact information and a message to notify the fire department when a bathroom fall sensor detects a person's fall in the bathroom.
The abnormality determination method information 145 is information on the determination process or procedure for a device such as a bathroom fall sensor to determine an abnormality and issue a specific report.
The secret information 146 is information stored based on a specific management attribute designated by the user U, and if the specific management attribute is, for example, a secret conversation, the information is not disclosed to others when the user U has a secret conversation with the watching robot 2. Specifically, for example, the user U's monologue, diary, feelings at the time, etc. are not output to anyone other than the user U, cannot be accessed, and are stored in the DB 140 as information that cannot be erased even by the user U.

In addition, in the CPU 11 (Figure 2) of the cloud server 1, as shown in Figure 4, when executing a series of processes related to the living robot care service for the user U, an interface control unit 31, a dialogue unit 32, an equipment operation setting information acquisition unit 33, an impact information notification unit 34, a specific notification unit 35, and an anomaly determination method update unit 36 function.

The interface control unit 31 controls the interface including the dialogue with the user U.
For example, the interface control unit 31 treats elderly people (old people) living at home, people who need care, people with disabilities, etc. as a user U (a specified user), and controls the interface including the dialogue between this user U and the monitoring robot 2.

Specifically, when a specific device among the multiple devices present in the home, such as a bathroom fall sensor, attempts to notify the fire department, the interface control unit 31 controls the output to confirm this to the user U.
Specifically, for example, when the bathroom fall sensor is about to notify the fire department of a bathroom fall, the interface control unit 31 can make the watching robot 2 speak to ask the user U whether it is OK to notify the fire department, even though the bathroom fall sensor has detected a fall in the bathroom, and there is a possibility that this is a malfunction due to the noise of the bathroom dryer.

The interface control unit 31 acquires lifestyle information of the user U based on a dialogue with the user U, and the specific reporting unit 35 performs an abnormality determination based on the lifestyle information.
This allows user U to have his/her own lifestyle information managed, for example, the time that user U usually takes a bath, simply by interacting with the device, thereby improving the accuracy of specific reports.
Managing lifestyle information includes, for example, managing the implementation record of "schedule management (calling),""musictherapy," and "brain games." By managing the lifestyle information of user U in this way, for example, if the bathroom fall sensor detects a fall when the user is not taking a bath and the bathroom dryer is operating, it can be determined that the false detection is due to the noise of the bathroom dryer, making it possible to prevent false reports in such cases and improve reporting accuracy.

Furthermore, when the user U converses with another user who has a specific relationship with the user U, such as the elderly user U's child or grandchild, a staff member of the service provider S (service operating company) that provides the monitoring service, or a staff member of small-lot delivery, via the monitoring robot 2, the interface control unit 31 performs predetermined input/output processing (for example, automatically starting the eye camera of the monitoring robot 2 to capture and image the elderly user U who is the subject of the conversation, and distributes the image to the external terminal 6 of the child or grandchild. In addition, the interface control unit 31 may also perform operations such as having the monitoring robot 2 vocally communicate to the user U what has happened since the previous monitoring visit.
This can improve the convenience of conversation between the elderly user U and his/her children or grandchildren, and provide topics of conversation, thereby making the user U feel more positive.

The dialogue unit 32 converses with the user U via the monitoring robot 2. The words and the like used for the dialogue may be realized by, for example, thought processing of AI or the like, or phrases such as answers to questions may be registered in advance in the DB 140 as dialogue information 142, and appropriate words may be read from the dialogue information 142 during the dialogue with the user U.

The equipment operation setting information acquisition unit 33 acquires equipment operation setting information 143 for each of a number of devices that can be connected via a communication network such as network N, such as a bathroom fall sensor, odor sensor, fire alarm, television, video, air conditioner, refrigerator, washing machine, bathroom dryer, electric kettle, etc.
Equipment operation setting information 143 includes, for example, detection settings and sensitivity settings of a bathroom fall sensor, sensitivity settings of an odor sensor, detection settings and sensitivity settings of a fire alarm, television operation settings, video recording settings, air conditioner operation settings and temperature settings, refrigerator operation settings and temperature settings, washing machine operation settings, bathroom dryer operation settings, and electric kettle operation settings.

When the impact information notification unit 34 receives information from a specific device, such as a bathroom fall sensor, among multiple devices that an emergency call will be made, and determines by referring to the device operation setting information 143 that the execution of a specific function will affect the operation settings of other devices, it notifies, through dialogue or by interrupting the dialogue, for example, the user U, other family members K, service provider S, etc., based on the control of the interface control unit 31.

As a specific example, if the impact information notification unit 34 determines that the operation of a bathroom dryer is causing a malfunction of the bathroom fall sensor, it will send a message to the monitoring robot 2 indicating that the operation of the bathroom dryer may be causing a malfunction of the bathroom fall sensor, and the monitoring robot 2 will issue a voice message to notify the user U.

As a method of notifying the message, other than vocalizing the message from the watching robot 2, the message may be displayed as a pop-up on the user terminal 4, for example.
As a result, the user U can learn, simply by interacting with the monitoring robot 2 or by operating and viewing the user terminal 4, that there is a possibility of malfunction when executing functions that affect each other, such as the bathroom dryer, bathroom fall sensor, and emergency reporting function.

The device operation setting information acquisition unit 33 acquires the device operation setting information 143 based on a dialogue with the user U under the control of the interface control unit 31 .
This allows the user U to configure the operation of home appliances simply by interacting with the monitoring robot 2.
For example, during a conversation between user U and the monitoring robot 2, if user U says, "It's a little hot, isn't it?", the equipment operation setting information acquisition unit 33 acquires the operation settings of the air conditioner and transmits them to the interface control unit 31, and the interface control unit 31 adjusts the settings, such as raising the set temperature included in the operation settings of the air conditioner by one degree.

The impact information notification unit 34 notifies the user of any impact (possibility of malfunction, etc.) between functions occurring between devices based on a dialogue with the user U under the control of the interface control unit 31 .
Specifically, if, during a conversation between user U and monitoring robot 2, user U says something like "the bathroom dryer is running," and the bathroom fall sensor detects a fall even though user U is not in the bathroom, it will determine that if the bathroom dryer is running, there is a possibility that the noise from the bathroom dryer will cause the bathroom fall sensor to malfunction, and will notify user U that the noise from the bathroom dryer will affect the fall detection function of the bathroom fall sensor (possibility of malfunction, etc.).
As a result, by simply interacting with the monitoring robot 2, the user U can learn, for example, that executing the drying function of the bathroom dryer will affect the operation of the bathroom fall sensor to detect falls, report emergency calls, etc.

When the specific notification unit 35 determines that an abnormality has occurred based on the device operation setting information 143 of a plurality of devices, the specific notification unit 35 issues a specific notification, such as an emergency notification to a fire department, under the control of the interface control unit 31 .
Here, abnormality determination means that when the bathroom fall sensor (a specific device) detects a fall, the device determines whether the detection information of the bathroom fall sensor is abnormal by referring to the operation information of devices other than the bathroom fall sensor, such as the bathroom dryer, and comments by user U ("I'm fine" or "Help me!").
This makes it possible to improve the accuracy of specific notifications (for example, emergency notifications triggered by a fall detection by a bathroom fall sensor) based on the device operation setting information 143 of multiple devices.

Based on the control of the interface control unit 31, the specific reporting unit 35 further outputs information regarding the abnormality determination (e.g., the location of the bathroom where user U has fallen) to a responder, such as a paramedic C from the fire department, who has responded to the emergency call (specific call).
Specifically, when paramedic C of the ambulance team arrives at the scene and says something like "This is the ambulance team," the interface control unit 31 identifies paramedic C and causes the monitoring robot 2 to say something like "He's collapsed in the bathroom at the end of the hallway on the right side!"
This makes it possible, for example, to inform a paramedic C of the ambulance who has arrived at the scene where the user U has collapsed.

The abnormality determination method update unit 36 updates the determination logic and determination method (abnormality determination method) of the abnormality determination by the specific reporting unit 35 based on the abnormality determination result, under the control of the interface control unit 31, and based on the acquired information on the need for a specific report.
This allows, for example, if the device determines that there is an abnormality and attempts to make a specific report, and dialogue with the user U reveals that the specific report is unnecessary, the method of determining the abnormality thereafter can be modified.

Based on the control of the interface control unit 31, the secret information management unit 37 manages the secret conversation based on the specific management attribute when the user U specifies, for example, a secret conversation between the user U and the monitoring robot 2 and the cloud server 1 acquires the information of the secret conversation via the monitoring robot 2.
As information of a secret conversation, for example, the user U's monologue, diary, feelings at the time, etc. are stored in a storage means such as a non-volatile memory that is not output to anyone other than the user U, cannot be accessed, and cannot be erased even by the user U.
This allows the user U to develop an attachment to the watching robot 2 through the exchange of secret information that he or she has confessed only to the watching robot 2.

Next, the operation of the cloud server in this information processing system will be described with reference to FIG.
FIG. 5 is a flowchart showing the operation of the cloud server of FIGS.
In the cloud server 1, in step S11, the interface control unit 31 controls the interface including the dialogue between the user U and the watching robot 2.
In step S12, the device operation setting information acquisition unit 33 acquires the device operation setting information 143 of each of a plurality of control target devices connectable via the network N.
In step S13, in response to acquiring information that a specified function will be executed from a specified device among the multiple controlled devices, the impact information notification unit 34 refers to the device operation setting information 143 of each device related to the specified device and determines that the execution of the specified function will affect the operation settings of the other devices, and then notifies the user U of this through dialogue or by interrupting the dialogue based on the control of the interface control unit 31 in step S11.

In this way, according to the operation of the cloud server 1, while the monitoring robot 2 is interacting with the user U, the cloud server acquires that information, and the interface control unit 31 takes into consideration the operating status of the devices operating in the user U's home and notifies the user U that the execution of a specified function of a specified device will affect the operating settings of other devices. Therefore, by simply interacting with the monitoring robot 2, the user can learn when multiple devices in the home are executing functions that affect each other (e.g., how the execution of a specified function of a specified device will affect and spread to other devices).

Various embodiments of the living robot care service in the information processing system of FIG. 1 will be described below.
First, a first embodiment of the living robot care service in the information processing system of FIG. 1 will be described with reference to FIG.
FIG. 6 is a diagram showing a first embodiment of a living robot care service in the information processing system of FIG.

The first embodiment of the living robot care service shown in Figure 6 shows an overall picture of the information processing system that realizes the living robot care service.

In the user U's home, various devices (a monitoring robot 2, a control target device 3) including, for example, sensors and services are arranged.
The control target devices 3 include household electrical appliances (hereinafter referred to as "home appliances") such as toilets, refrigerators, electric kettles, lighting fixtures, air conditioners, and automatic opening and closing curtains.
The sensors include, for example, sensors for monitoring the bathroom (falls, submersion in water), excretion sensors, cameras for monitoring presence in the room, sleep vital signs/wake-up sensors, environmental monitoring sensors (temperature, humidity, abnormal sounds, etc.), and each sensor transmits detected information to the cloud server 1 via the network N.
The sleep vitals/wake-up sensor is one of the devices in a sleep monitoring system that provides a sleep monitoring service.
The bathroom monitoring sensor has the function of alerting (contacting) emergency responders if it detects a fall or submersion in water.

Services include sleep monitoring services, lunch box delivery services, flat-rate taxi shuttle services, brain game services, and music therapy services.
In addition, in the user U's home, a wireless router serves as a base station, and the monitoring robot 2 and the controlled device 3 are connected to a cloud server 1 and the external terminals 6 of the service provider S and family member K via a network N such as the Internet.

When the watching robot 2 talks to the user U or when the user U operates the television remote control while interacting with the user U, the watching robot 2 receives notification of the operation and transmits the operation information to the cloud server 1.
The cloud server 1 can control each device via the network N and obtain information from each device.
The cloud server 1 acquires lifestyle information of the user U in advance and manages it using a dashboard D1 and a scheduler.

When the cloud server 1 acquires abnormality information from the monitoring robot 2 or the controlled device 3, it comprehensively assesses the information, taking into account the influence of other devices, etc., and if it determines that an abnormality has occurred, it sends an abnormality notification to the external terminal 6.
When the external terminal 6 receives the abnormality notification, it notifies the service provider S and the family K by issuing the received abnormality notification (display, sound, etc.). Also, not only when an abnormality is detected, but also periodically at a preset time or at regular intervals, the user U is checked for monitoring, and the user U's condition is notified. According to the first embodiment, a platform can be built in which various sensors, home appliances, and services are connected and the monitoring robot 2 serves as a UI (user interface).
As a result, a comprehensive living robot care service that provides total care for the user U can be provided.

Next, a second embodiment of the living robot care service in the information processing system of FIG. 1 will be described with reference to FIG.
FIG. 7 is a diagram showing a second embodiment of the living robot care service in the information processing system of FIG.
The second embodiment of the living robot care service shown in FIG. 7 is an example of an information processing system that realizes the living robot care service, in which the state of each sensor and home appliance is optimized by the action of a monitoring robot 2.

In this case, the cloud server 1 causes the watching robot 2 to speak based on the lifestyle information, thereby starting a dialogue with the user U.
Specifically, when it is time to go to the toilet based on the daily life information of the user U, the watching robot 2 will say, for example, "It's about time to go to the toilet." If the user U wakes up a little later than his usual wake-up time, the watching robot 2 will say, "Didn't you sleep well last night? Let's do some exercises together." When it is detected that the user U has gotten up from the bed, the watching robot 2 will say, "Good morning, I'll open the curtains." When the room temperature exceeds a certain temperature, the watching robot 2 will say, "It's hot, so I'll turn on the air conditioner, and make sure you drink plenty of water."

The detection information of various sensors acquired by the cloud server 1 is presented on the dashboard D1, so that users of the external terminal 6, such as the service provider S and family K, can constantly monitor the status of the user U and the user U's home.

According to the second embodiment, the watching robot 2 can interact with the user U to deepen the relationship with the user U, and can optimize the state of each sensor and home appliance based on the user U's lifestyle information in accordance with the current state of the user U. In addition, feedback can be provided to the user U himself/herself.
As a result, by communicating with the user U, the monitoring robot 2 can create an optimal environment for the user U's life.

Next, a third embodiment of the living robot care service in the information processing system of FIG. 1 will be described with reference to FIG.
FIG. 8 is a diagram showing a third embodiment of the living robot care service in the information processing system of FIG.
8 is an example of a processing operation executed by the watching robot 2 when an abnormality in daily life is detected by a sensor in the house in an information processing system that realizes the living robot care service. More specifically, it is an example in which the watching robot 2 detects an abnormality that occurs in the daily life of the user U and reports the abnormality to others on behalf of the user U.

In the third embodiment, when a bathroom monitoring sensor in the home detects, for example, a person's fall in the bathroom, an abnormality detection signal is sent from the bathroom monitoring sensor to the cloud server 1.
When the cloud server 1 receives the abnormality detection signal, it controls the monitoring robot 2 to call out to the person and give a notification (alert).
As for calling out to people, for example, the monitoring robot 2 may say to the user U who has collapsed, "Are you OK? Shall I call for help?". In addition, the monitoring robot 2 may say to the family member K, "Go and check on the bathtub" or may control home appliances to blink lights. Furthermore, if the family member K is not nearby, the cloud server 1 may send a message or notify the external terminal 6 (e.g., the family member's smartphone) of the abnormality, and the external terminal 6 may issue an alert (e.g., display the report content on a screen or output it as a voice), thereby notifying the family member K and the service provider S of the abnormality of the user U.

When the user U responds to the voice of the watching robot 2 by saying "Everything seems fine," the cloud server 1 does not issue a specific report from the specific reporting unit 35, and the interface control unit 31 controls the watching robot 2 to notify the external terminal 6 of family member K of the voice saying "Everything seems fine."
Furthermore, when the user U utters a command such as "Call for help!" or "Help me!", the cloud server 1 immediately makes an emergency call to the fire brigade under the control of the interface control unit 31.

In addition, if user U does not respond despite being called out to, the cloud server 1 determines whether or not the detection information of the bathroom fall sensor is abnormal based on the device operation setting information of multiple devices in the home, for example by referring to the operation information of the bathroom dryer and lifestyle information 141, and if it determines that there is an abnormality, it makes an emergency call to the fire brigade based on the control of the interface control unit 31.

When paramedic C is dispatched from the fire brigade in response to the emergency call and rushes to the user U's home, the cloud server 1 identifies paramedic C using the words of the paramedic C or the camera in the eye of the monitoring robot 2, and controls the monitoring robot 2 to say something like, "He's collapsed in the bathroom at the end of the hallway on the right side!", thereby communicating the location of user U to paramedic C.

In this way, according to the third embodiment, an alert is issued after the sensor detects an abnormality, and at the same time, the monitoring robot 2 at the scene works with the user U, family member K, and the emergency medical technician C who rushes to the scene, allowing smooth and rapid response according to the situation at the scene.

Next, a fourth embodiment of the living robot care service in the information processing system of FIG. 1 will be described with reference to FIG.
FIG. 9 is a diagram showing a fourth embodiment of the living robot care service in the information processing system of FIG.
The fourth embodiment of the living robot care service shown in FIG. 9 is an example of improving the life of a user U through schedule management in an information processing system that realizes the living robot care service.

In this fourth embodiment, family member K of user U operates the application screen of external terminal 6 or the browser of external terminal 6 to access cloud server 1, displaying user U's personal page (for family member K), and operating the "Schedule setting, confirmation" button in the personal page, causing the interface control unit 31 of cloud server 1 to activate the schedule management function (scheduler) of cloud server 1 and display the scheduler screen D2 (schedule management screen).

This scheduler screen D2 allows the user U to set and confirm items related to plans among the lifestyle information 141, and by operating the screen, family member K can register or change plans such as time to take medicine, time to exercise, and other plans such as time to watch television (rakugo) in the schedule.
When the schedule of the user U is updated on the scheduler screen D2, the interface control unit 31 of the cloud server 1 controls the watching robot 2 to notify the user U of the schedule update by speaking.

Also, the user U or the family K can talk to the watching robot 2 and register an event in the schedule by saying, for example, "I'll go out the day after tomorrow at 10 o'clock." Furthermore, the user U or the family K can operate a remote control in hand to display a schedule screen on the screen of a controlled device 3 such as a TV, and can operate the contents of the schedule to register or change an event in the schedule in the same manner as described above.
Then, when the schedule has been successfully registered or changed, the watching robot 2 will say, "It's been registered in your schedule! Would you like to check it on TV?" or "I've registered an appointment to go out the day after tomorrow at 10 o'clock!", etc., so that the user U can confirm that the appointment has been successfully registered or changed in the schedule. When the user U says "Yes" to the watching robot 2's question, "Would you like to check it on TV?", the interface control unit 31 of the cloud server 1 will display the schedule screen D2 on the TV screen, so that the user U can check the appointment registered in the schedule.

Then, when the actual scheduled time arrives, the interface control unit 31 of the cloud server 1 controls the watching robot 2 to make the watching robot 2 say something like, "It's time for medicine/exercises/rakugo...!" to notify the user U.

Thus, according to this fourth embodiment, the schedule management function (scheduler) of the cloud server 1 can register or change (including delete) the schedule of user U through dialogue between family member K or user U himself and the monitoring robot 2.
In addition, the user U's daily schedule can be managed on the cloud server 1, and appropriate information can be provided to the user U in a timely manner through dialogue from the monitoring robot 2.

Next, a fifth embodiment of the living robot care service in the information processing system of FIG. 1 will be described with reference to FIG.
FIG. 10 is a diagram showing a fifth embodiment of the living robot care service in the information processing system of FIG.
10 is an example of an information processing system for realizing the living robot care service, in which a schedule management function is linked with various services. More specifically, the living robot 2 is an example of a system in which a user is notified of service plans, such as home delivery and transportation, in cooperation with the service based on a previously set schedule.

In this fifth embodiment, by registering plans such as home delivery, flat-rate taxi service, music therapy, and brain games in the scheduler, when the user U and the monitoring robot 2 communicate with each other before the scheduled time, for example, when waking up, the monitoring robot 2 will say something like, "Let's go out by taxi starting at 11 o'clock today!"
In addition, based on past information left in the scheduler, the monitoring robot 2 can provide topics of conversation, such as "It's been a week since the last time the delivery guy came! This and that happened."

According to this fifth embodiment, by linking the schedule management function with various services, during the dialogue between the user U and the monitoring robot 2, the monitoring robot 2 vocalizes the schedule of upcoming services the user U will receive and the contents of services that the user has received in the past, providing topics of conversation, so that the user U can always experience new activities or reminisce about past topics.

Next, a sixth embodiment of the living robot care service in the information processing system of FIG. 1 will be described with reference to FIG.
FIG. 11 is a diagram showing a sixth embodiment of the living robot care service in the information processing system of FIG.
The sixth embodiment of the living robot care service shown in FIG. 11 is a basic example in which the watching robot 2 itself serves as a UI in an information processing system that realizes the living robot care service.
More specifically, this is an example in which the monitoring robot 2 utilizes controllable devices such as a TV connected to the network N to realize two-way communication.

In this sixth embodiment, when the user U and the watching robot 2 are conversing, for example, if the user U says, for example, "I want to call my grandchild," the watching robot 2 will say, "I'll make a call,""I'll turn on the TV," etc.
The interface control unit 31 of the cloud server 1 starts up the TV and attempts to make a call to an external terminal 6 of a distant family member K using the videophone function. When connected to the external terminal 6, an image of family member K captured by the camera of the external terminal 6 is displayed on the TV screen and a videophone call is initiated, so that the user U can make a call to family member K (grandchild) simply by interacting with the monitoring robot 2.

In this way, according to the sixth embodiment, a specific keyword that the user U speaks to the watching robot 2 is associated with an action corresponding to that keyword and registered in the cloud server 1. As the user U converses with the watching robot 2, the watching robot 2 becomes a UI when the user U utters just one word, and can cause devices in the home to execute a series of operations that the user wants to perform on behalf of the user U, allowing the user U to spend his or her time in comfort.

Next, a second functional configuration of the cloud server 1 of an embodiment of the information processing device according to the present invention will be described with reference to FIG. 12. In describing the second functional configuration, the same components as those in FIG. 4 will be denoted by the same reference numerals, and the description thereof will be omitted.

In the second functional configuration, in the CPU 11 of the cloud server 1 (see Figures 2 and 4), as shown in Figure 12, the interface control unit 31 and the push notification unit 38 function when executing a series of processes related to the living robot care service for the user U.

The interface control unit 31 controls the monitoring robot 2 so as to link the smartphone T operated by the user U to the dialogue with the user U.
Specifically, the interface control unit 31 links the monitoring robot 2, which is the human interface, and the smartphone T in a coexisting relationship (two UIs), so that, for example, the monitoring robot 2 is mainly responsible for calling out to people and giving simple voice instructions from people, while complex operations such as entering schedules and using services are mainly performed by the smartphone T or a remote control, etc.
By linking the smartphone T with the monitoring robot 2 in this manner, the user U can use the interface in a way that suits his or her convenience and situation, allowing the user U to receive care services more comfortably.

The interface control unit 31 starts controlling the watching robot 2, including dialogue with the user U, when a certain condition is met, such as activating the conversation mode of the watching robot 2 by using a preset specific word or physically touching the watching robot 2. This allows the user U to wake up the watching robot 2 and talk to it at a time of his or her choice.

The push notification unit 38 acquires the contents to be push-notified to a predetermined user U when at least some of the devices among the plurality of devices execute a predetermined function, and pushes the contents to the user U or an administrator through dialogue based on the control of the interface control unit 31. Administrators include people designated by the user U (e.g., family members, etc.).
This allows the monitoring robot 2 to call out to the user U through a push notification.

Hereinafter, various embodiments of a living robot care service in an information processing system including a cloud server having the second functional configuration of FIG. 12 will be described.
First, a seventh embodiment of the living robot care service in the information processing system of FIG. 1 will be described with reference to FIG.
FIG. 13 is a diagram showing a seventh embodiment of a living robot care service in an information processing system including a cloud server having the second functional configuration of FIG. 12 .

The seventh embodiment of the living robot care service shown in FIG. 13 shows a modification of the first embodiment of the living robot care service shown in FIG. 6.

In the seventh embodiment, as shown in FIG. 13, a smartphone T is placed in the user U's home as a mobile terminal operated by the user U, for example.
An application program for smartphones (hereinafter referred to as "smartphone app") provided by the service provider is installed on smartphone T.
The smartphone app works in conjunction with the watching robot 2 under the control of the cloud server 1 to provide information to the user U. The smartphone app also transmits to the cloud server 1 input information and instructions input by the user U through operations on the smartphone app screen.

By having the monitoring robot 2 coexist with the smartphone T and become the interface for the user U, the user U can be cared for through two user interfaces: the smartphone T and the monitoring robot 2.

In this information processing system, all operations can be performed by the user U using the smartphone T, but the monitoring robot 2 is mainly responsible for calling out to the user U and receiving simple voice instructions from the user U, and complex operations (such as entering a schedule or using a service) are mainly performed by operating the smartphone T or TV screen (for example, a graphic user interface: GUI or TV remote control operation).

The interaction between the user U and the monitoring robot 2 is reflected in real time on the status displayed on the smartphone screen of the smartphone T.
The voice exchange between the user U and the monitoring robot 2 is also reflected in chat format on the smartphone screen of the smartphone T, allowing the user U to monitor the process of the exchange in text form.

Even if the monitoring robot 2 misrecognizes the voice, the user U can check the text that the monitoring robot 2 recognizes on the smartphone screen, and can find out the reason for the success or failure of the voice recognition.

In addition, when the user U corrects mistakes in the exchange between the user U and the watching robot 2 using the smartphone T, correct/incorrect data is stored in the cloud server 1, and the correct/incorrect data is used to train the conversational AI of the watching robot 2 by machine learning under the control of the cloud server 1, thereby improving the voice recognition accuracy of the watching robot 2.

Next, an eighth embodiment of the living robot care service in the information processing system of FIG. 1 will be described with reference to FIG.
FIG. 14 is a diagram showing an eighth embodiment of a living robot care service in an information processing system including a cloud server having the second functional configuration of FIG. 12 .

The eighth embodiment of the living robot care service shown in FIG. 14 shows a modification of the second embodiment of the living robot care service shown in FIG. 7.

In the eighth embodiment, the watching robot 2 is provided with a "push call mode" and a "conversation mode."
In the "push call mode" of the monitoring robot 2, home appliances and sensors are controlled in conjunction with the monitoring robot 2 calling out to the user U by push notification. The control of home appliances and sensors is performed according to the time, environment, and various sensor information (sleep vital sensor, etc.).

In the case of the eighth embodiment, the control content is not just a reading out of data, such as the results of a comparison with past data, but is also digested. Specifically, as shown in the speech bubble in Figure 14, a message is spoken to the user U to provide guidance or advice on his or her next actions.
At that time, there is no interaction between the user U and the monitoring robot 2, and no voice is reflected, but the push call content of the monitoring robot 2 is transmitted from the cloud server 1 to the smartphone T and reflected on the smartphone screen, so that the user U can check the push call content by the text displayed on the smartphone screen.

The "conversation mode" is a function that enables the user U to wake up the watching robot 2 by performing a predetermined action and enjoy a conversation with the watching robot 2.
Specifically, when user U takes an action such as speaking a specific word (e.g., Mr. Robot) or physically touching the monitoring robot 2, the cloud server 1 detects the specified action and controls the monitoring robot 2 to activate the "conversation mode" of the monitoring robot 2.

After activating the "conversation mode," the user U's spoken instructions (e.g., saying "open the curtains") and the user U's voice instructions are recognized by the monitoring robot 2, and the curtains are opened under control of the cloud server 1.
At the same time, the watching robot 2 responds by saying, "Good morning, I'll open the curtains," etc., and if the "conversation mode" is activated later than usual, the watching robot 2 makes push-talk to the user U, saying, "Didn't you sleep better than usual last night?", "Let's do some exercises together," etc., and continues the conversation with the user U.

In addition, if the temperature detected by the temperature sensor installed in the room is higher than usual, the monitoring robot 2 will say to the user U, "It's hot, so I'll turn on the air conditioner. Make sure you drink plenty of water." The air conditioner is operated under the control of the cloud server 1.

In addition, when a predetermined time has passed since the user U last went to the toilet or when a certain time arrives, the watching robot 2 calls out to the user U saying, "It's about time to go to the toilet."
The above series of exchanges will be reflected on the chat screen displayed on the smartphone T when the "conversation mode" is activated.
As a result, the user U can say "Open the curtains" in the text displayed on the chat screen of the smartphone T, causing the monitoring robot 2 to activate "conversation mode", after which the user U can check the content of the conversation with the monitoring robot 2 and the operating status of the home appliances in the room. Also, on the dashboard D1, an administrator or the like can monitor the status of each sensor as well as the content of the conversation between the user U and the operating status of the home appliances in the room.

According to this eighth embodiment, the "push call mode" of the monitoring robot 2 is used to call out to the user U and check the user U's condition based on the user U's response, and the monitoring robot 2 can also optimize the state of each sensor and home appliance in the room to a state preferred by the user U.
Furthermore, according to the eighth embodiment, by providing a "conversation mode," the user U can wake up the watching robot 2 by uttering specific words or the like and have a conversation with it. When the user U activates the "conversation mode," the watching robot 2 determines that a conversation request has arisen, and speaks to the user U, for example, "Didn't you sleep better than usual yesterday?" or "Let's do some exercise together," and provides feedback to the user U himself, so that the user U can enjoy a conversation with the watching robot 2 whenever he or she wants to talk.

Next, a ninth embodiment of the living robot care service in the information processing system of FIG. 1 will be described with reference to FIG.
FIG. 15 is a diagram showing a ninth embodiment of a living robot care service in an information processing system including a cloud server having the second functional configuration of FIG. 12 .

The ninth embodiment of the living robot care service shown in FIG. 15 shows a modification of the third embodiment of the living robot care service shown in FIG. 8.

In this ninth embodiment, in addition to the monitoring robot 2 as an interface for the user U, a smartphone T is provided. When the monitoring robot 2 detects an abnormality in the user's daily life, the detected content is notified to the smartphone T and displayed on the application screen. Depending on the content of the abnormality, the smartphone T's own alarm function such as a warning sound, a warning light, and vibration is used to notify the user U of the abnormality.

The monitoring robot 2 also notifies the user of the abnormality by making a warning sound and moving. The monitoring robot 2 moves to the area where the abnormality is detected and communicates with the outside world by calling out to the user, taking pictures with a camera, and using communication means such as a TV phone. Specifically, the monitoring robot 2 heads to the location of the sensor that detected the abnormality or the source of the abnormal sound (scream, sound of something falling, sound of glass breaking) by automatic driving using a camera and LiDAR, and uploads on-site information such as voice and images acquired at the site to the cloud server 1. If it is confirmed that there is no malfunction of the sensor or that there is no problem with the user U himself, the operation of the abnormality detection action ends there.
On the other hand, if an abnormal state occurs, the watching robot 2 notifies at least one of the family, the service operating company, and the security company, and monitors the state until someone arrives.
The monitoring robot 2 transmits the robot's own location (location information) and behavioral history, etc. to the smartphone T via the cloud server 1, so that the user U can always check the location and behavior, etc. of the monitoring robot 2 on the app screen of the smartphone T.

Next, a tenth embodiment of the living robot care service in the information processing system of FIG. 1 will be described with reference to FIG.
FIG. 16 is a diagram showing a tenth embodiment of a living robot care service in an information processing system including a cloud server having the second functional configuration of FIG. 12 .

The tenth embodiment of the living robot care service shown in FIG. 16 shows a modification of the fourth embodiment of the living robot care service shown in FIG. 9.

In this tenth embodiment, in addition to the monitoring robot 2 as an interface for the user U, a smartphone T is provided, and the schedule management function of the cloud server 1 manages the schedule of the user U on the application screen of the smartphone T or on the screen of the family member's smartphone R.

With this schedule management function, the user U's daily schedule is managed on the application screen of the smartphone T, and the monitoring robot 2 and the smartphone T notify the user U at the start and end times of events set in the schedule, allowing the user U to live their daily life according to the set schedule.

Schedules and other information can be input manually into the smartphone T itself, or through a television, a family member's smartphone R, or conversation with the monitoring robot 2. Input methods include voice input and manual input using the touch or switch parts of the robot. If a user U has difficulty inputting information into a smartphone, the monitoring robot 2 will start up the television and guide the user through the television input.

In addition, since the schedule is uploaded to the cloud server 1, the user U can share (input and check) his/her schedule with his/her family by sending the schedule from the cloud server 1 to the family's smartphone R or the like.
Thus, according to the tenth embodiment, the user U's schedule can be managed by the monitoring robot 2 and the smartphone T, thereby improving the user U's life.

Next, an eleventh embodiment of the living robot care service in the information processing system of FIG. 1 will be described with reference to FIG.
FIG. 17 is a diagram showing an eleventh embodiment of a living robot care service in an information processing system including a cloud server having the second functional configuration of FIG. 12 .

The eleventh embodiment of the living robot care service shown in FIG. 17 shows a modification of the fifth embodiment of the living robot care service shown in FIG. 10.

In addition to the schedule management function, the eleventh embodiment also has a linking function for linking the schedule with various services.
With this collaboration function, calendar appointments, push reminder notifications from the monitoring robot 2, etc. are notified to the smartphone T via the cloud server 1, and the status of each is displayed on the app screen of the smartphone T, so that the user U can check his or her own schedule from the status displayed on the app screen of the smartphone T.
In addition to smartphone T, by notifying user U's schedule to, for example, a television or a family member's smartphone R, user U can check his or her schedule on the television as well.
Furthermore, family members can check User U's schedule for the day on the screens of their own smartphones.

In this way, according to the 11th embodiment, by providing a linking function that links the schedule with various services, the user U's schedule can be shared between the user and his/her family.

Next, a twelfth embodiment of the living robot care service in the information processing system of FIG. 1 will be described with reference to FIG.
FIG. 18 is a diagram showing a twelfth embodiment of a living robot care service in an information processing system including a cloud server having the second functional configuration of FIG. 12 .

The twelfth embodiment of the living robot care service shown in FIG. 18 shows a modification of the sixth embodiment of the living robot care service shown in FIG. 11.

This twelfth embodiment is a basic configuration example in which the monitoring robot 2 and the smartphone T each serve as the UI for the user U.
In this case, when the user U speaks to the monitoring robot 2, for example as shown in Figure 18, saying "I want to call my grandchild," the monitoring robot 2 responds to the user U by saying "I'm going to call you, so please have your smartphone ready," and an instruction to make a call is transmitted to the cloud server 1, and the cloud server 1 controls the outgoing call to the family member's smartphone R that has been registered in advance on the smartphone T, and a phone call or video call is made from the user U's smartphone T to the family member's smartphone R.
This allows the user U's current appearance, state, surrounding situation, etc. to be displayed on the family member's smartphone R. In addition to telephone calls between the user U's smartphone T and the family member's smartphone R, it is also possible to hold web meetings using Internet telephone/SNS on tablets, PCs, etc.

In this way, according to the 12th embodiment, the monitoring robot 2 and the smartphone T each become the UI of the user U, and when the user U speaks to the monitoring robot 2 saying, for example, "I want to call my family," the user U can make a call from the smartphone T to the family's smartphone R without directly operating the smartphone T. This means that even a user U who is unsure about operating the smartphone T can easily call their family.

Although one embodiment of the present invention has been described above, the present invention is not limited to the above-mentioned embodiment, and any modifications, improvements, etc. that can achieve the object of the present invention are included in the present invention.

For example, the hardware configuration shown in FIG. 2 is merely an example for achieving the objectives of the present invention and is not particularly limited.

The functional block diagram shown in FIG. 4 is merely an example and is not particularly limited. In other words, it is sufficient that the information processing device is provided with a function capable of executing the above-mentioned series of processes as a whole, and the functional blocks and databases used to realize this function are not particularly limited to the example in FIG. 4.

Furthermore, the locations of the functional blocks and databases are not limited to those shown in FIG. 4 and may be arbitrary.
Furthermore, each functional block and database may be configured as a single piece of hardware, a single piece of software, or a combination of these.

When the processing of each functional block and database is executed by software, the program constituting the software is installed into a computer or the like from a network or a recording medium.
The computer may be a computer built into dedicated hardware, or may be a computer capable of executing various functions by installing various programs, such as a cloud server, a general-purpose smartphone, a personal computer, or the like.

Recording media containing such programs are not only configured as removable media that are distributed separately from the device body in order to provide each user with the program, but also configured as recording media etc. that are provided to each user in a state where they are pre-installed in the device body.

In this specification, the steps of describing a program to be recorded on a recording medium include not only processes that are performed in chronological order, but also processes that are not necessarily performed in chronological order but are executed in parallel or individually.

In addition, in this specification, the term "system" refers to an overall device that is composed of a plurality of devices, a plurality of means, etc.
In the above-described embodiment, examples of other users (related parties) to user U (a specific user) include family members (children, grandchildren, etc.) and service providers S (a service management company and its personnel, etc.), but the present invention may be other than these.

In summary, the information processing apparatus to which the present invention is applied is sufficient if it has the following configuration, and can take various different embodiments.
That is, an information processing device to which the present invention is applied (for example, the cloud server 1 in FIG. 4 )
An interface control unit (e.g., the interface control unit 31 in FIG. 4 ) that controls an interface including a dialogue with a predetermined user (e.g., the user U in FIG. 4 );
a device operation setting information acquisition means (such as the device operation setting information acquisition unit 33 in FIG. 4 ) for acquiring device operation setting information (such as the detection setting and sensitivity setting of the bathroom fall sensor, the sensitivity setting of the odor sensor, the detection setting and sensitivity setting of the fire alarm, the operation setting of the television, the recording setting of the video, the operation setting and temperature setting of the air conditioner, the operation setting and temperature setting of the refrigerator, the operation setting of the washing machine, the operation setting of the bathroom dryer, the operation setting of the electric kettle, etc.) of a plurality of devices (such as a bathroom fall sensor, an odor sensor, a fire alarm, a television, a video, an air conditioner, a refrigerator, a washing machine, a bathroom dryer, an electric kettle, etc.) that can be connected via a communication network (such as the network N in FIG. 1 );
an impact information notification means (e.g., impact information notification unit 34 of FIG. 4 ) for notifying a user (e.g., user U, other family member K, service operator S of FIG. 4 ) of this fact (e.g., that the operation of the bathroom dryer may be causing a malfunction of the bathroom fall sensor) through the dialogue or by interrupting the dialogue under control of the interface control means (e.g., interface control unit 31 of FIG. 4 ) in response to obtaining information that a predetermined function (e.g., an emergency call) will be executed from a predetermined device (e.g., a bathroom fall sensor) among the plurality of devices;
Equipped with.
This allows the user to know when multiple devices (e.g. bathroom dryer, bathroom fall sensor, emergency call function, etc.) are executing functions that affect each other simply by interacting with them.

In the information processing device (for example, the cloud server 1 in FIG. 4 ),
The device operation setting information acquisition means (e.g., the device operation setting information acquisition unit 33 in Figure 4) acquires operation setting information of the device based on a dialogue with the specified user (e.g., user U in Figure 4) under the control of the interface control means (e.g., the interface control unit 31 in Figure 4).
This allows the user to configure the operation of the device simply by interacting with the device.

In the information processing device (for example, the cloud server 1 in FIG. 4 ),
The impact information notification means (e.g., the impact information notification unit 34 in Figure 4) issues a notification based on a dialogue with the specified user (e.g., user U in Figure 4) under the control of the interface control means (e.g., the interface control unit 31 in Figure 4).
As a result, a user (such as user U in Figure 4) can learn that, simply by interacting with the monitoring robot 2, the execution of a specified function (such as a drying function) of a specified device (such as a bathroom dryer) will affect the operation (fall detection, emergency reporting, etc.) of other devices (such as a bathroom fall sensor).

In the information processing device (for example, the cloud server 1 in FIG. 4 ),
The interface control means (e.g., interface control unit 31 in Figure 4) controls the output to confirm to a specified user (e.g., user U in Figure 4) that a specific device among the multiple devices (e.g., a bathroom fall sensor) is about to make a specific report (e.g., a report to the fire department).
Specifically, for example, when reporting a fall in the bathroom to the fire department, the user U can be asked to confirm this.

In the information processing device (for example, the cloud server 1 in FIG. 4 ),
a specific notification means (e.g., specific notification unit 35 in FIG. 4 ) that makes the specific notification (e.g., an emergency notification to a fire department, etc.) under the control of the interface control means (e.g., interface control unit 31 in FIG. 4 ) when an abnormality determination is made based on the device operation setting information of the plurality of devices (e.g., determining whether or not to treat the detection information of a bathroom fall sensor as abnormal by also referring to the operation information of a bathroom dryer),
Equipped with.
This makes it possible to improve the accuracy of specific reports based on the device operation setting information of multiple devices.

In the information processing device (for example, the cloud server 1 in FIG. 4 ),
The specific reporting means (e.g., the specific reporting unit 35 in Figure 4) further outputs information regarding the abnormality determination to a responder (e.g., a paramedic from the fire department) who has responded to the specific report (e.g., an emergency report) based on the control of an interface control means (e.g., the interface control unit 31 in Figure 4).
This makes it possible to provide an arriving ambulance with information regarding where a specific user (such as user U in FIG. 4) has collapsed, for example.

In the information processing device (for example, the cloud server 1 in FIG. 4 ),
an abnormality determination method updating means (e.g., the abnormality determination method updating unit 36 in FIG. 4 ) for updating a method of the abnormality determination by the specific notification means (e.g., the specific notification unit 35 in FIG. 4 ) based on the information on necessity of the specific notification acquired under the control of the interface control means (e.g., the interface control unit 31 in FIG. 4 ) for the determination result of the abnormality determination,
It further comprises:
This allows, for example, if the device determines that there is an abnormality and attempts to make a specific report, and then, through dialogue with the user, it becomes clear that the specific report is unnecessary, the method of determining the abnormality thereafter can be modified.

In the information processing device (for example, the cloud server 1 in FIG. 4 ),
The interface control means (e.g., the interface control unit 31 in FIG. 4, etc.) acquires lifestyle information of the predetermined user based on a dialogue with the predetermined user,
The specific notification means (for example, the specific notification unit 35 in FIG. 4) performs the abnormality determination based on the life information.
This allows the user to have their own lifestyle information managed (for example, the time when user U usually takes a bath) simply by interacting with the device, improving the accuracy of specific reports. For example, if the bathroom fall sensor detects a fall when the user is not taking a bath, if the bathroom dryer is operating, it can be determined that this is a false detection due to the noise of the bathroom dryer, and such a report can be prevented.

In the information processing device (for example, the cloud server 1 in FIG. 4 ),
The interface control means (e.g., the interface control unit 31 in FIG. 4, etc.) performs predetermined input/output processing (e.g., automatically starting the eye camera of the monitoring robot 2 to capture and photograph the elderly user U, and distributing the photograph to the other user's terminal (e.g., the external terminal 6 in FIG. 4, etc.), speaking to user U about events that have occurred since the last monitoring visit, etc.) when the specified user (e.g., user U in FIG. 4, etc.) has a dialogue with another user who has a specific relationship with the specified user (e.g., the child or grandchild of elderly user U, a staff member of the service management company that provides the monitoring service, a staff member of small-lot delivery, etc.).
This can improve the convenience of conversations with other users and provide topics of conversation, making the user feel more positive.

In the information processing device (for example, the cloud server 1 in FIG. 4 ),
The system further includes a specific information management means (e.g., secret information management unit 37 in FIG. 4 ) for managing the specified information (e.g., secret conversation, etc.) based on a specific management attribute (e.g., a secret conversation between user U and the monitoring robot 2 only) specified from the specified user (e.g., user U in FIG. 4 ) based on the control of the interface control means (e.g., interface control unit 31 in FIG. 4 ), in response to acquiring the specified information (e.g., information about a secret conversation, etc.) based on the specific management attribute.
As information of a secret conversation, for example, the user U's monologue, diary, feelings at the time, etc. are not output (cannot be accessed) by anyone other than the user U, and are stored in a storage means such as a non-volatile memory that cannot be erased even by the user U.
This allows the user U to develop an attachment to the watching robot 2 through the exchange of secret information that he or she has confessed only to the watching robot 2.

The interface control means (e.g., the interface control unit 31 in FIG. 12 ) controls the interface (e.g., the robot 2 in FIG. 12 ) so as to link a mobile terminal (e.g., the smartphone T in FIG. 12 ) operated by the predetermined user to a dialogue with the predetermined user (e.g., the user U in FIG. 12 ).
It is possible.

a push notification means (e.g., the push notification unit 38 in FIG. 12 ) that acquires content to be push-notified to the predetermined user when at least a part of the plurality of devices executes a predetermined function, and push-notifies the predetermined user or an administrator of the content through the dialogue based on the control of the interface control means;
Further comprising:
It is possible.

The interface control means (e.g., the interface control unit 31 in FIG. 12) starts the control of the interface (e.g., the robot 2 in FIG. 12), including the dialogue with the predetermined user (e.g., the user U in FIG. 12), when a preset condition (a specific word or a physical touch on the robot 2 activates the conversation mode of the robot 2) is satisfied as a trigger.
It is possible.

In the information processing method executed by the information processing device (e.g., the cloud server 1 in FIG. 4 ),
An interface control step (e.g., step S11 in FIG. 5 ) for controlling an interface including a dialogue with a predetermined user (e.g., user U in FIG. 4 );
A device operation setting information acquisition step (e.g., step S12 in FIG. 5 ) of acquiring device operation setting information of each of a plurality of devices connectable via a communication network;
an impact information notifying step (e.g., step S13 in FIG. 5 ) of notifying a user of the effect through the dialogue or by interrupting the dialogue based on control in the interface control step when it is determined that the execution of the predetermined function will affect the operation settings of other devices by referring to the device operation setting information in response to obtaining information indicating that a predetermined function will be executed from a predetermined device among the plurality of devices;
Includes.
This allows the user to know when multiple devices (e.g. bathroom dryer, bathroom fall sensor, emergency call function, etc.) are executing functions that affect each other simply by interacting with them.

In a computer (e.g., the CPU 11 of the cloud server 1 in FIG. 4 ),
An interface control step (e.g., step S11 in FIG. 5 ) for controlling an interface including a dialogue with a predetermined user (e.g., user U in FIG. 4 );
A device operation setting information acquisition step (e.g., step S12 in FIG. 5 ) of acquiring device operation setting information of each of a plurality of devices connectable via a communication network;
an impact information notifying step (e.g., step S13 in FIG. 5 ) of notifying a user of the effect through the dialogue or by interrupting the dialogue based on control in the interface control step when it is determined that the execution of the predetermined function will affect the operation settings of other devices by referring to the device operation setting information in response to obtaining information indicating that a predetermined function will be executed from a predetermined device among the plurality of devices;
The control process includes:
This allows the user to know when multiple devices (e.g. bathroom dryer, bathroom fall sensor, emergency call function, etc.) are executing functions that affect each other simply by interacting with them.

1: Cloud server, 2: Monitoring robot, 3: Control target device, 4: User terminal (e.g. smartphone or remote control), 5: Emergency team terminal, 6: External terminal (e.g. smartphone), 31: Interface control unit, 32: , 33: Device operation setting information acquisition unit, 34: Impact information notification unit, 35: Specific notification unit, 36: Anomaly determination method update unit, 37: Confidential information management unit, 38: Push notification unit, K: Family (children, grandchildren, etc.), N: Network, S: Service provider (service operating company, etc.), U: User

Claims (15)

an interface control means for controlling an interface including a dialogue with a predetermined user;
a device operation setting information acquiring means for acquiring device operation setting information of each of a plurality of devices connectable via a communication network;
an impact information notifying means for, in response to receiving information indicating that a predetermined function is to be executed from a predetermined device among the plurality of devices, referring to the device operation setting information and determining that the execution of the predetermined function will affect operation settings of other devices, notifying a user of that effect through the dialogue or by interrupting the dialogue under the control of the interface control means;
An information processing device comprising: the device operation setting information acquisition means acquires operation setting information of the device based on a dialogue with the predetermined user under the control of the interface control means;
The information processing device according to claim 1 . the impact information notifying means notifies based on a dialogue with the predetermined user under the control of the interface control means;
The information processing device according to claim 1 . the interface control means, when a specific device among the plurality of devices is about to make a specific report, executes control regarding an output for confirming that fact to a predetermined user.
The information processing device according to claim 1 . a specific notification means for issuing the specific notification under the control of the interface control means when an abnormality is determined based on the device operation setting information of the plurality of devices;
The information processing device according to claim 4 , further comprising: The specific notification means further outputs information regarding the abnormality determination to a responder who responds to the specific notification under the control of the interface control means.
The information processing device according to claim 5 . an abnormality determination method update means for updating a method of the abnormality determination by the specific notification means based on information on necessity of the specific notification acquired under control of the interface control means with respect to a result of the abnormality determination;
The information processing device according to claim 5 , further comprising: The interface control means acquires lifestyle information of the predetermined user based on a dialogue with the predetermined user,
The specific notification means performs the abnormality determination based on the life information.
The information processing device according to claim 1 . the interface control means performs a predetermined input/output process when the predetermined user interacts with another user who has a specific relationship with the predetermined user.
The information processing device according to claim 1 . a specific information management means for managing the specific information based on a specific management attribute in response to acquisition of the specific information by the specific user under control of the interface control means,
The information processing device according to claim 1 . the interface control means controls the interface so as to link a mobile terminal operated by the predetermined user to a dialogue with the predetermined user;
The information processing device according to claim 1 . a push notification means for acquiring content to be push-notified to the predetermined user when at least a part of the plurality of devices executes a predetermined function, and for pushing-notifying the predetermined user or an administrator of the predetermined user through the interaction based on the control of the interface control means;
The information processing device according to claim 1 , further comprising: the interface control means starts the control of the interface, including the dialogue with the predetermined user, when a preset condition is satisfied as a trigger.
The information processing device according to claim 1 . An information processing method executed by an information processing device,
an interface control step for controlling an interface including a dialogue with a predetermined user;
a device operation setting information acquisition step of acquiring device operation setting information of each of a plurality of devices connectable via a communication network;
an impact information notifying step of notifying a user of the effect by the dialogue or by interrupting the dialogue based on control in the interface control step when it is determined by referring to the device operation setting information that the execution of the predetermined function will affect operation settings of other devices in response to obtaining information indicating that a predetermined function will be executed from a predetermined device among the plurality of devices;
An information processing method comprising: On the computer,
an interface control step for controlling an interface including a dialogue with a predetermined user;
a device operation setting information acquisition step of acquiring device operation setting information of each of a plurality of devices connectable via a communication network;
an impact information notifying step of notifying a user of the effect by the dialogue or by interrupting the dialogue based on control in the interface control step when it is determined by referring to the device operation setting information that the execution of the predetermined function will affect operation settings of other devices in response to obtaining information indicating that a predetermined function will be executed from a predetermined device among the plurality of devices;
A program that executes control processing including:

Images