JP2013258555A - Head-mounted display, biological information management apparatus, and biological information display method - Google Patents

Abstract

It is difficult to play a game with a vital sensor attached to the body.
A head-mounted display includes a forehead contact portion 120 that contacts the wearer's forehead, a temporal contact portion 130 that contacts the wearer's temporal region, a forehead contact portion 120, and A biological information acquisition sensor that is mounted on at least one of the temporal contact portions 130 and detects biological information of the wearer, and a display that displays an image to the wearer are provided. It further includes a communication control unit that transmits the biological information detected by the biological information acquisition sensor or an index related to the biological information and receives an index related to the biological information of another person, and the display displays the index related to the biological information of the other person together with the image of the other person. To do.
[Selection] Figure 1

Description

  The present invention relates to a head mounted display and a biological information display method in the head mounted display.

  Some game machines in the home are equipped with a camera, which captures a user playing a game and displays the user's image on the game screen, or detects a user's action from the captured image of the user to play a game. It can be reflected. It is also possible to detect the user's facial expression from the user's image captured by the camera, grasp the user's mental state during game play, and reflect it in the development of the game. In addition, a plurality of players each have a controller and can operate a game machine, and a game is often played by a plurality of people including family members and friends.

  In order to detect the degree of tension of the user during game play, the user wears a sensor that detects biological information such as a pulse, and causes the game machine to input a measurement result of the biological information such as a pulse output by the sensor to the game machine. Reflecting is also done.

  Conventional vital sensors are often worn in a way that makes the user feel uncomfortable, such as sticking to the chest with an adhesive, fixing with a belt, or pinching with a clip, and sometimes the sensor is detached during measurement. In particular, during game play, the user often moves his / her body with a controller, so it is difficult to play the game with the vital sensor attached to the body.

  The present invention has been made in view of such problems, and the purpose thereof is a head-mounted display that can easily acquire biological information, a biological information management device that manages biological information acquired by the head-mounted display, and The object is to provide a biological information display method using a head-mounted display.

  In order to solve the above-described problems, a head-mounted display according to an aspect of the present invention includes a forehead contact portion that contacts the forehead of the wearer, and a temporal contact portion that contacts the side of the wearer. And a biological information acquisition sensor that is mounted on at least one of the forehead contact portion and the temporal contact portion and detects biological information of the wearer, and a display that displays an image to the wearer.

  Another aspect of the present invention is a biological information management device. This apparatus includes a biometric information acquisition unit that receives a user's biometric information from a head-mounted display that is worn on the user's head, and converts the received user's biometric information into an index that indicates the user's psychological state. And a communication unit that searches the user database for an index indicating the psychological state of another user in response to a request from the head mounted display and transmits the index to the head mounted display.

  Yet another embodiment of the present invention is a biological information display method. This method includes a step of acquiring a user's biological information detected by a biological information acquisition sensor mounted on a head-mounted display worn by the user, and converting the acquired biological information into an index indicating a user's psychological state. In the user database, in response to a request from the head mounted display, searching for an index indicating the psychological state of another user from the user database and transmitting it to the head mounted display; Displaying an index indicating the psychological state of another user.

  It should be noted that any combination of the above-described constituent elements and the expression of the present invention converted between a method, an apparatus, a system, a computer program, a data structure, a recording medium, and the like are also effective as an aspect of the present invention.

  ADVANTAGE OF THE INVENTION According to this invention, biometric information can be acquired easily and the parameter | index regarding biometric information can be displayed.

It is an external view of a head mounted display. It is a functional lineblock diagram of a head mount display. It is a figure explaining the camera and infrared LED (Light Emitting Diode) which are incorporated in a head mounted display. It is a figure explaining the system which shares the parameter | index which shows biometric information or a psychological state between the head mounted displays of a some user. It is a figure explaining the system which shares the parameter | index which shows biometric information or a psychological state between the head mounted displays of a some user by peer to peer. It is a functional block diagram of a server. It is a flowchart explaining the procedure which shares the parameter | index which shows a user's psychological state between several user's head mounted displays. It is an example of the parameter | index which shows the psychological state displayed on the display of a head mounted display. It is another example of the parameter | index which shows the psychological state displayed on the display of a head mounted display.

  FIG. 1 is an external view of the head mounted display 100. The head mounted display 100 includes a main body portion 110, a forehead contact portion 120, and a temporal contact portion 130.

  The head mounted display 100 is a display device that is worn on the user's head and enjoys still images and moving images displayed on the display, and listens to sound and music output from the headphones.

  The position information of the user can be measured by a position sensor such as GPS (Global Positioning System) incorporated in or externally attached to the head mounted display 100. Further, posture information such as the orientation and inclination of the head of the user wearing the head mounted display 100 can be measured by a posture sensor built in or externally attached to the head mounted display 100.

  The main body 110 includes a display, a position information acquisition sensor, a posture sensor, a communication device, and the like. The forehead contact unit 120 and the temporal contact unit 130 include a biological information acquisition sensor.

  The biological information acquisition sensor is a sensor that can measure biological information such as body temperature, pulse, blood component, sweating, brain waves, and cerebral blood flow of the user wearing the head mounted display 100. The acquired biometric information is converted into an index such as a numerical value, an icon, and a color, and is displayed on the display of the head mounted display 100. In addition, a user's biometric information or an index related to biometric information acquired on the head-mounted display 100 of a certain user is shared by a server by wire or wireless, and transmitted to and displayed on the display of the head-mounted display 100 of another user. .

  The forehead contact unit 120 is a part that contacts the user's forehead, and is located close to the brain, so it is equipped with an electroencephalogram sensor that detects information about the electroencephalogram and a blood sensor that detects information about the blood component. Is suitable. The temporal contact part 130 is a part that comes into contact with the user's ear and its surroundings, and is located at a position where body temperature and pulse can be easily measured. Therefore, a temporal sensor for detecting body temperature and a pulse sensor for detecting pulse are mounted. Suitable for

  The head mounted display 100 may further be provided with a camera that captures the eyes of the user. The camera mounted on the head mounted display 100 can detect the user's line of sight, pupil movement, blinking, and the like.

  Here, a configuration in which a biological information acquisition sensor is mounted on the head mounted display 100 will be described. However, the configuration is not limited to the head mounted display 100 in a narrow sense, but glasses, glasses type displays, glasses type cameras, headphones, headsets (headphones with microphones). The biological information acquisition sensor may be mounted on earphones, earrings, ear-mounted cameras, hats, hats with cameras, hair bands, and the like.

  FIG. 2 is a functional configuration diagram of the head mounted display 100.

  The control unit 10 is a main processor that processes and outputs signals such as image signals and sensor signals, commands and data. The input interface 20 receives operation signals and setting signals from the touch panel and the touch panel controller, and supplies them to the control unit 10. The output interface 30 receives the image signal from the control unit 10 and displays it on the display. The backlight 32 supplies a backlight to the liquid crystal display.

  The communication control unit 40 transmits data input from the control unit 10 to the outside through wired or wireless communication via the network adapter 42 or the antenna 44. The communication control unit 40 also receives data from the outside via wired or wireless communication via the network adapter 42 or the antenna 44 and outputs the data to the control unit 10.

  The storage unit 50 temporarily stores data, parameters, operation signals, and the like processed by the control unit 10.

  The GPS unit 60 receives position information from a GPS satellite and supplies it to the control unit 10 in accordance with an operation signal from the control unit 10. The radio unit 62 receives position information from the radio base station and supplies it to the control unit 10 in accordance with an operation signal from the control unit 10.

  The attitude sensor 64 detects attitude information such as the orientation and inclination of the main body 110 of the head mounted display 100. The posture sensor 64 is realized by appropriately combining a gyro sensor, an acceleration sensor, an angular acceleration sensor, and the like.

  The external input / output terminal interface 70 is an interface for connecting peripheral devices such as a USB (Universal Serial Bus) controller. The external memory 72 is an external memory such as a flash memory.

  The clock unit 80 sets time information according to a setting signal from the control unit 10 and supplies time data to the control unit 10.

  The biological information acquisition sensor 90 detects various biological information in accordance with an operation signal from the control unit 10 and supplies the detected sensor signal to the control unit 10. The biological information acquisition sensor 90 includes a CCD image sensor that detects the movement of the eyeball in addition to the vital sensors that detect vital signs such as the above-described brain wave sensor, blood sensor, body temperature sensor, and pulse sensor.

  The control unit 10 processes the sensor signal output from the biological information acquisition sensor 90 and converts it into an index indicating the user's psychological state (hereinafter referred to as “psychological state index”). Human blood flow and body temperature are adjusted by the autonomic nerve, and the degree of autonomic nerve tension can be estimated by analyzing heart rate variability. For example, the heart rate variability of the user is an index indicating the tension of the autonomic nerve of the user's heart. The degree of autonomic nervous tension can be estimated by analyzing the spectrum of the frequency component of the heartbeat period fluctuation.

  Autonomic nerves are composed of sympathetic nerves and parasympathetic nerves, and sympathetic nerves increase the heart rate and raise blood pressure to support human activities. On the other hand, parasympathetic nerves support human rest by lowering heart rate and lowering blood pressure. It is possible to detect whether the body is moving toward the active or resting state by the balance between the functions of the two opposing nerves. In particular, it has been found that sympathetic nerves increase activity when mental stress is applied.

  When the heartbeat variability is subjected to frequency analysis, an LF (Low Frequency) component and an HF (High Frequency) component are detected. The LF component indicates the degree of activity of both the sympathetic nerve and the parasympathetic nerve, and the HF component indicates the degree of activity of the parasympathetic nerve. Therefore, by measuring the LF / HF ratio, an index indicating the degree of sympathetic nerve activity, in other words, an index indicating the tension of the autonomic nerve (corresponding to the degree of stress) can be obtained.

  The control unit 10 can supply the user's psychological state index data to the output interface 30 and display the data on the display, or supply the data to the communication control unit 40 to be transmitted to the outside. When the processing performance of the control unit 10 is limited, the biological information detected by the biological information acquisition sensor 90 is supplied to the communication control unit 40 and transmitted to the server 200 described later without being converted into a psychological state index. Then, the server 200 side may convert the biological information into a psychological state index, and the communication control unit 40 may be configured to receive the converted psychological state index from the server 200.

  FIG. 3 is a diagram for explaining a camera 150 and an infrared LED (Light Emitting Diode) 160 built in the head mounted display 100. The user looks at the display 140 installed in front of the eyes, but the user's eyeball moves or the number of blinks changes depending on the user's psychological state. For example, when the user is staring at the video displayed on the display 140, the movement of the eyeball stops and the number of blinks is reduced, but the user is distracted or interested in the video displayed on the display 140. When there is no eyeball, the eyeball moves from side to side and the number of blinks increases.

  The infrared LED 160 irradiates infrared rays to the user's eyeball, and the camera 150 captures infrared rays reflected from the eyeball. Since the direction and amount of the reflected infrared rays vary depending on the movement of the eyeball and the frequency of blinking, the control unit 10 causes the user's line-of-sight direction, pupil movement, and blinking according to the direction and amount of infrared rays captured by the camera 150. And the like can be detected.

A method for detecting the direction of the line of sight by photographing the reflected light of the light applied to the eyeball in this way is described in, for example, the following document.
Saito, Takahashi, Iwamura, Oki, Hashiguchi “Detection of Gaze Direction: Effect of Using Scleral Imaging” (Yamanashi University Faculty of Engineering Research Report No. 42, 13-15, December 1991)

  The control unit 10 converts the detected user's line-of-sight direction, pupil movement, blink frequency, and the like into a user's psychological state index using a predetermined conversion formula. For example, when the user's pupil movement is intense, it is determined that there is no calmness, and when the gaze direction does not move, it is determined that the user is interested in the displayed content.

  FIG. 4 is a diagram for explaining a system for sharing biological information or a psychological state index among head mounted displays of a plurality of users.

  Each of the head mounted displays 100a to 100d worn by a plurality of users transmits the biometric information of each user or the user's psychological state index converted from the biometric information to the server 200 via the network. The server 200 stores the biometric information or psychological state index of each user in association with the user identification information. In response to requests from the head mounted displays 100a to 100d worn by each user, the server 200 transmits biological information or psychological state indicators of other users to the requesting head mounted displays 100a to 100d. Other users' biometric information or psychological state indicators are displayed on the displays of the head mounted displays 100a to 100d of each user in association with images of other users such as icons, face images, and avatars.

  FIG. 5 is a diagram illustrating a system for sharing biometric information or psychological state indicators among head mounted displays of a plurality of users on a peer-to-peer basis.

  Each of the head mounted displays 100a and 100b worn by a plurality of users is biological information acquired by the biological information acquisition sensors of the respective head mounted displays 100a and 100b via a network such as a wireless ad hoc network, or biological information. The psychological state index of the user converted from is transmitted. Each user's head-mounted display 100a, 100b receives other users' biometric information or psychological state indicators and displays them on the display in association with images of other users such as icons, face images, or avatars.

  A system for sharing biometric information or a psychological state index among a plurality of users at a server as shown in FIG. 4, and a system for sharing biometric information or a psychological state index among a plurality of users through peer-to-peer communication as shown in FIG. In either case, the biological information or the psychological state index can be exchanged with each other, and the psychological state index of another user can be displayed on the display 140 of each head mounted display 100.

  For example, the other person's psychological state can be grasped by displaying the other party's heart rate and sweating numerical values or the converted values of graphics data such as icons on the other party's face. Thus, the psychological state of the other party can be inferred by looking at a numerical value such as a heartbeat without reading the facial expression, and can be used for bargaining in a card game or the like. A poker face opponent can produce a situation where it is difficult to deceive by giving a numerical value.

  FIG. 6 is a functional configuration diagram of the server 200. The server 200 is an example of a biometric information management device, and includes a biometric information receiving unit 210, a psychological state index converting unit 220, a psychological state index transmitting unit 270, a content providing unit 240, a tag adding unit 250, a user database 230, and a content database. 260 is included.

  The biological information receiving unit 210 receives the user's biological information transmitted from the head mounted display 100.

  The psychological state index conversion unit 220 converts the user's biological information into the user's psychological state index according to a predetermined conversion method, and registers the psychological state index in the user database 230 in association with the user's identification information.

  For example, when the heart rate is rising, the psychological state index conversion unit 220 determines that the psychological state is an excited state, and sets an index indicating the degree of the excited state according to the heart rate. If the heart rate rises when watching a horror movie, it is judged that the person is excited with a fear.

  The psychological state index conversion unit 220 estimates the degree of concentration based on the increase or decrease of the user's blink, and sets an index indicating the degree of concentration according to the number of blinks. For example, if the number of blinks decreases during a race game, it is determined that the person is concentrated.

  In addition, the psychological state index conversion unit 220 determines that the state of excitement is present when the number of breaths increases, and sets an index indicating the degree of the state of excitement according to the number of breaths. For example, if the breathing increases during the game, it is determined that the player is excited.

  When the server 200 receives a request for the mental state index of another user from the head mounted display 100, the psychological state index transmission unit 270 searches the user database 230 for the psychological state index of the other user, and the head mounted display 100. Send to.

  The content providing unit 240 reads content data displayed on the display of the head mounted display 100 from the content database 260 and transmits the content data to the head mounted display 100.

  The tag providing unit 250 tags the user's psychological state index to content such as video provided to the user by the content providing unit 240. The tag assignment unit 250 acquires content identification information currently provided to the user from the content database 260, acquires the user's psychological state index from the psychological state index conversion unit 220, and identifies the content identification information and the user's psychological state. Information associated with the index is registered in the user database 230 as tag information.

  When providing content to the user, the content providing unit 240 acquires tag information from the user database 230, analyzes the tag information, reads content that the user is interested in from the content database 260, and content that matches the user's preference Is transmitted to the head mounted display 100 of the user.

  The tag addition unit 250 can perform tagging together with the content being played in the game and the time stamp of the shot video, which can be used for analyzing the user's interests and preferences. Furthermore, the content providing unit 240 can recommend recommended information and content to the user according to the user's interests and preferences. Thus, entertainment property can be improved by acquiring information indicating vital signs in real time and displaying the data in combination with other information such as images.

  For example, the tag assigning unit 250 profiles the excitement level when watching a video for each user, estimates the user's preference, and assigns tag information to the content. The content providing unit 240 provides, for example, recommended information of comedy videos to those who are excited to see a comedy video, and informs the purchase ranking of content of users having similar preferences.

  In addition, the psychological state index conversion unit 220 analyzes the time interval (RR interval) of adjacent R waves of the heartbeat acquired from the biological information reception unit 210, and analyzes the activity of the autonomic nerve to analyze the user's activity. Estimate the degree of stress. The content providing unit 240 provides the same type of music to the user's head-mounted display 100, for example, for those who have a low degree of stress after listening to classical music.

  Further, for example, in a shooting game, when the user's heart rate is high, the content providing unit 240 takes a deep breath by transmitting to the head mounted display 100 an image to which an effect of blurring the aim or an effect of blurring the image is transmitted. It is possible to produce an effect that it is difficult to hit the target without lowering the heart rate.

  If at least a part of the functional configuration of the server 200 described with reference to FIG. 6 has sufficient resources on the head mounted display 100 side, it can be mounted on the head mounted display 100. For example, the psychological state index conversion unit 220 may be mounted on the head mounted display 100 and configured to convert biological information into a psychological state index on the head mounted display 100 side.

  FIG. 7 is a flowchart illustrating a procedure for sharing a user's psychological state index among a plurality of user's head mounted displays.

  The biometric information acquisition sensor 90 mounted on the user's head mounted display 100 acquires the biometric information of the user (S10). The communication control unit 40 of the head mounted display 100 transmits the acquired biological information to the server 200 (S12).

  The psychological state index conversion unit 220 of the server 200 converts the biological information received from the head mounted display 100 into a user's psychological state index according to a predetermined conversion formula, and stores it in the user database 230 in association with the user identification information. (S14).

  Here, the server 200 converts the received biological information into the user's psychological state index, but the control unit 10 of the head mounted display 100 converts the biological information received from the biological information acquisition sensor 90 into the user according to a predetermined conversion formula. It may be converted into a psychological state index. In this case, the communication control unit 40 transmits the user's psychological state index to the server 200, and the server 200 stores the psychological state index received from the head mounted display 100 in the user database 230 in association with the user identification information. .

  The communication control unit 40 of the head mounted display 100 requests the server 200 for a psychological state index of another user (S16). The psychological state index transmission unit 270 of the server 200 searches the user database 230 for the requested psychological state index of the other user and transmits it to the head mounted display 100 (S18).

  The communication control unit 40 of the head mounted display 100 receives another user's psychological state index from the server 200, and the output interface 30 includes other users 'photos and videos, images of other users' avatars, and the like. The psychological state index of the user is displayed on the display 140 (S20).

  In the above procedure, the server 200 registers the psychological state indicators of a plurality of users. However, when the head-mounted display 100 of a plurality of users performs peer-to-peer communication, the plurality of heads are not connected via the server 200. The psychological state indicators are exchanged between the mount displays 100.

  8A and 8B are examples of psychological state indicators displayed on the display of the head mounted display 100. FIG. In FIG. 8A and FIG. 8B, indicators indicating emotions such as heart rate, sweating, anger and joy of other users are displayed along with other users' face images.

  In FIG. 8A, the values of the indexes indicating the heart rate and sweating of other users are both small, and the feeling of joy is stronger than anger. The face image of another user may be a real image or a virtual image. Since the psychological state index indicates a calm state and a pleasant emotion, the user's face shows a smile as shown in FIG. 8A. If the face image is a virtual image such as an avatar, a smiling face image may be synthesized and displayed based on the psychological state index.

  In FIG. 8B, the index values indicating the heart rate and sweating of other users are both large, and the feeling of anger is stronger than pleasure and anger. Since the psychological state index indicates the user's restless state and annoying emotion, the user's face is impatient as shown in FIG. 8B. If the face image is a virtual image such as an avatar, an impatient face image may be synthesized and displayed based on the psychological state index.

  In the above description, the user's biological information is acquired by a vital sensor or a CCD image sensor mounted on the head mounted display 100, and the biological information is converted into the user's psychological state index. However, the user's biological information is connected to a device such as a game machine. The user's face image may be captured by a camera, and the user's emotion may be estimated from the user's face image. In the above description, the orientation sensor mounted on the head mounted display 100 detects the orientation and inclination of the head of the user wearing the head mounted display 100. However, the user is detected by a camera connected to a device such as a game machine. The orientation and inclination of the user's face may be acquired by photographing the face image and analyzing the face image.

  As described above, according to the head-mounted display 100 and the biological information display method using the head-mounted display 100 according to the present embodiment, the user acquires information indicating the vital sign of the user while wearing the head-mounted display 100. The psychological state of the user can be estimated and the psychological state index can be displayed on the display.

  Apart from the head-mounted display 100, there is no need to attach a vital sensor to the user's body, so there is no burden on the user, and there is no fear that the vital sensor will come off even if the body is moved during game play.

  In addition to simply displaying the vital signal detected by the biological information acquisition sensor mounted on the head mounted display 100, a game in which the psychological state of the user estimated from the vital signal is displayed on the display of the head mounted display 100. Can be reflected in the content. Thereby, entertainment property can be improved.

  The present invention has been described based on the embodiments. The embodiments are exemplifications, and it will be understood by those skilled in the art that various modifications can be made to combinations of the respective constituent elements and processing processes, and such modifications are within the scope of the present invention. . Such a modification will be described.

  10 control unit, 20 input interface, 30 output interface, 32 backlight, 40 communication control unit, 42 network adapter, 44 antenna, 50 storage unit, 60 GPS unit, 62 wireless unit, 64 attitude sensor, 70 external input / output terminal interface 72 external memory, 80 clock part, 90 biological information acquisition sensor, 100 head mounted display, 110 main body part, 120 frontal head contact part, 130 side head contact part, 140 display, 150 camera, 160 infrared LED, 200 server 210 biological information receiving unit, 220 psychological state index converting unit, 230 user database, 240 content providing unit, 250 tag adding unit, 260 content database, 70 psychological state index transmission unit.

Claims (10)

A forehead contact portion that contacts the wearer's forehead;
A temporal contact portion that contacts the temporal side of the wearer;
A biological information acquisition sensor that is mounted on at least one of the forehead contact portion and the temporal contact portion and detects the wearer's biological information;
A head-mounted display comprising a display for displaying an image to a wearer.   The biological information acquisition sensor includes at least one of an electroencephalogram sensor that acquires information about a wearer's brain wave and a blood sensor that acquires information about a blood component of the wearer, and the electroencephalogram sensor and the blood sensor are in contact with the forehead. The head mounted display according to claim 1, wherein the head mounted display is mounted on a portion.   The biological information acquisition sensor includes at least one of a pulse sensor that acquires information related to a wearer's pulse and a body temperature sensor that acquires information related to the body temperature of the wearer, and the pulse sensor and the body temperature sensor include the temporal contact portion. The head mounted display according to claim 1, wherein the head mounted display is mounted on the head mounted display.   The head-mounted display according to any one of claims 1 to 3, wherein the biological information acquisition sensor includes an image sensor that detects at least one of a wearer's line of sight, pupil movement, and blinking.   The head mounted display according to any one of claims 1 to 4, further comprising a communication control unit that transmits biological information detected by the biological information acquisition sensor or an index related to biological information. The communication control unit receives an index related to another person's biological information,
The head mounted display according to claim 5, wherein the display displays an index related to other person's biological information together with an image of the other person. A biometric information acquisition unit that receives the biometric information of the user from a head mounted display mounted on the user's head;
A psychological state index converting unit that converts the received biometric information of the user into an index indicating the psychological state of the user and registers the index in the user database;
A biological information management apparatus comprising: a communication unit that searches an index indicating a psychological state of another user in response to a request from the head mounted display from the user database and transmits the index to the head mounted display.   The biometric information management apparatus according to claim 7, further comprising a tag adding unit that tags a content that is viewed on the head-mounted display with an index indicating a user's psychological state and registers the tag in the user database. Acquiring biometric information of the user detected by a biometric information acquisition sensor mounted on a head mounted display worn by the user;
Converting the acquired biological information into an index indicating the psychological state of the user and registering it in the user database;
In response to a request from the head-mounted display, searching for an index indicating the psychological state of another user from the user database and transmitting the index to the head-mounted display;
And displaying an index indicating a psychological state of another user on the head mounted display. Acquiring biometric information of the user detected by a biometric information acquisition sensor mounted on a head mounted display worn by the user;
Transmitting biological information detected by the biological information acquisition sensor or an index related to biological information;
Receiving an index relating to biometric information of another user;
A program for causing a computer to execute a step of displaying an index related to biological information of another user together with an image of the other user.

Images