JP2020013179A - Gaze target detection device, gaze target detection method, and program - Google Patents

Abstract

A gaze target detection device capable of detecting a target gaze at a driver even if detection accuracy of a gaze direction is not sufficient.
A gaze target detecting device (1) obtains a target based on a surrounding monitoring sensor (10) for detecting an object in the vicinity of a vehicle, an in-vehicle camera (20) for photographing a driver, and an image taken by the in-vehicle camera (20). A processing unit 21 for obtaining an object that the driver is gazing based on the driver's line of sight and the distance or relative speed between each object and the host vehicle detected by the surrounding monitoring sensor 10. Prepare.
[Selection diagram] Fig. 1

Description

  The present invention relates to a gazing target detecting device, a gazing target detecting method, and a program for detecting a target gazed by a driver.

  2. Description of the Related Art Conventionally, there has been known a driving support device that detects an object that a driver who is driving a vehicle is gazing at and assists driving.

  For example, the driving assistance device described in Patent Literature 1 stores in advance a driving state of a vehicle, a driving operation of a driver, and a pattern of a combination of a driver's line of sight, and detects a detected driving state of the vehicle. If the driver's driving operation and the driver's line of sight are different from the pattern stored in advance, a warning signal is output. This makes it possible to detect occurrence of a dangerous situation such as inattentive driving or erroneous operation, and warn the driver.

JP 2018-67198 A

  Various methods have been studied as a method for detecting the line of sight of a human, but a camera provided on a dashboard is often used as a device for detecting the direction of the driver's line of sight. Since there is a certain distance from the camera to the driver, there is an error of about several degrees to about 10 degrees in the driver's gaze direction that can be detected based on the image captured by the camera.

  Therefore, for example, in a situation where a preceding vehicle travels 40 m ahead and there is a pedestrian trying to cross a pedestrian crossing on the sidewalk also 40 m ahead, the angle difference between the preceding vehicle and the pedestrian as seen from the driver is about 4 °. It is. As described above, when the angle difference between the objects viewed from the driver is small, even if it is known that the driver's line of sight is facing forward, the driver is watching the preceding vehicle or watching the pedestrian. In some cases, it was difficult to distinguish the watch target.

  Therefore, the present invention provides a gaze target detection device, a gaze target detection method, and a program that can detect a target gaze at a driver even if detection accuracy of a gaze direction is not sufficient. The purpose is to:

  A gaze target detecting device of the present invention includes a surroundings monitoring sensor for detecting an object in the vicinity of a vehicle, a camera for photographing the driver, and a gaze direction of the driver obtained based on an image photographed by the camera. And an arithmetic processing unit that obtains an object that the driver is gazing on, based on the distance or relative speed between each object detected by the surrounding monitoring sensor and the host vehicle.

  It is considered that the driver gazes at a high-risk object while driving. The risk of the target has a relationship with the distance or relative speed between the host vehicle and each target. According to the configuration of the present invention, it is possible to obtain an object that the driver is gazing at based on the distance or relative speed between each object and the host vehicle in addition to the driver's line of sight.

  In the gaze target detection device of the present invention, when only a single target is present in the driver's line of sight, the arithmetic processing unit uses data on the distance or relative speed between each target and the host vehicle. Alternatively, an object in the driver's line of sight may be determined as the object to be watched.

  As described above, when only a single object exists in the line of sight, the gaze target can be easily obtained without using the distance or relative speed between the host vehicle and each object.

  A gaze target detection device according to another aspect of the present invention includes a surroundings monitoring camera that captures an object in the vicinity of a vehicle, a camera that captures a driver, and the type of the object and the magnitude of the risk for the type. Based on the stored risk information storage unit and the type of the object in the vicinity based on the image of the surrounding monitoring camera, based on the information stored in the risk information storage unit, the magnitude of the risk of each object And a calculation processing unit for obtaining an object that the driver is gazing on, based on the magnitude of risk of each object and the driver's gaze direction obtained based on the image captured by the camera. .

  With this configuration, it is possible to appropriately obtain the target object that the driver is gazing at, based on the magnitude of the risk set in advance according to the type of the target object.

  The gaze target detecting device of the present invention includes an in-vehicle sensor for acquiring operation information for the own vehicle, and the arithmetic processing unit is configured to determine an object on which the driver is gazing based on the operation information acquired by the on-vehicle sensor. You may ask for things.

  Since the driving of the vehicle is performed according to the situation of an external object, etc., it is determined whether or not the driver is performing an operation corresponding to the external object based on the operation information. Can be determined. According to the configuration of the present invention, it is possible to appropriately obtain an object that the driver is watching.

  In the gazing object detection device of the present invention, the arithmetic processing unit obtains a driver's face orientation based on an image captured by the camera, and when the driver's face is at a predetermined angle or more, performs inattentive driving. It may be determined that there is.

  As a human characteristic, it is known that when viewing an object within a predetermined angle range from the front, only the line of sight moves, and when the object exceeds the predetermined angle range, the face direction changes. By utilizing this characteristic, when the driver's face exceeds a predetermined angle range from the front, it is determined that the driver is looking aside, so that unnecessary calculation processing can be omitted.

  In the gaze target detecting device of the present invention, the arithmetic processing unit may set a front direction of the face direction based on the face direction when the driver sitting in the driver's seat faces forward.

  Although the driver's face direction varies from person to person due to the seat position, sitting height, etc., the front direction of the face direction is corrected based on the face direction when the driver sitting in the driver's seat is facing forward. This makes it possible to appropriately determine the face orientation.

  In the gaze target detection device of the present invention, the arithmetic processing unit, when the stop time of the driver's line of sight is equal to or less than a predetermined threshold, determines that the driver is not gazing at the object in the line of sight, The gaze target may be obtained.

  The movement of the driver's line of sight during driving includes stationary and high-speed movements. According to the present invention, when the stationary time of the line of sight is equal to or less than the predetermined threshold value, it is determined that the object is not gazing at the target, so that the object in the direction in which the line of sight is directed by high-speed motion is erroneously detected. It can be determined that the object is not a gaze target.

  The gaze target detection method of the present invention is a method for detecting an object that the driver is gazing at, the step of detecting an object in the vicinity of the vehicle with a surrounding monitoring sensor, A step of photographing the driver, a gaze direction of the driver obtained based on an image photographed by the camera, and a distance or a relative speed between each object and the own vehicle detected by the surrounding monitoring sensor. Obtaining the object that the driver is watching.

  The program of the present invention is a program for detecting an object that the driver is gazing at, the computer acquires the data of the object around the vehicle detected by the surrounding monitoring sensor, A step of acquiring the image of the driver taken by the camera, the direction of the driver's line of sight determined based on the image taken by the camera, and the relationship between each object detected by the surrounding monitoring sensor and the host vehicle. A step of obtaining an object that the driver is gazing at based on the distance or the relative speed.

  According to the configuration of the present invention, the gaze target can be obtained from the information of the target even in a situation where it is difficult to identify the gaze target only from the gaze direction.

FIG. 2 is a diagram illustrating a configuration of a gaze target detecting device according to the first embodiment. FIG. 4 is a diagram illustrating an example of a model for obtaining a gaze target based on gaze and environment information. It is a figure showing operation of a gaze target object detecting device of a 1st embodiment. It is a figure showing composition of a gazing object detecting device of a 2nd embodiment. FIG. 4 is a diagram illustrating an example of a model for obtaining a gaze target based on gaze, environment, and operation information. It is a figure showing operation of a gazing object detecting device of a 2nd embodiment. It is a figure showing composition of a gazing object detecting device of a 3rd embodiment. FIG. 6 is a diagram illustrating an example of information stored in a risk information storage unit. It is a figure showing operation of a gazing object detecting device of a 3rd embodiment.

  Hereinafter, a gaze target detecting device according to an embodiment of the present invention will be described with reference to the drawings. The gaze target detecting device according to the present embodiment is mounted on a vehicle and used, and has a function of detecting a target gazed by a driver (this is referred to as a “gaze target”). The state in which the user is gazing at a certain object is a state in which attention is distributed to the object. By detecting the gaze target, it is determined whether the driver has correctly recognized the risky situation, and an alarm is issued if appropriate attention is not allocated to objects around the vehicle. Thus, safe driving can be supported. Further, by detecting the gaze target, it is possible to avoid outputting an unnecessary alarm when the driver has already paid attention.

(First Embodiment)
FIG. 1 is a diagram illustrating a configuration of a gaze target detecting device 1 according to the first embodiment. The gazing target detecting device 1 is a gazing target detecting device 1 which detects an object around the vehicle, an in-vehicle camera 20 for photographing the driver, a gaze direction of the driver and each object. The vehicle includes an arithmetic processing unit 21 that obtains an object to be watched based on environmental information of an object, and an alarm output unit 22 that issues an alarm to a driver.

  The periphery monitoring sensor 10 of the present embodiment includes a camera 11 outside the vehicle, a millimeter wave radar 12, and a LIDAR 13. The outside camera 11 is a camera that captures a situation around the vehicle. As the camera 11 outside the vehicle, two cameras may be used as stereo cameras. The millimeter-wave radar 12 irradiates a radio wave having a very short wavelength called a millimeter wave and measures a reflected wave to measure a distance to an object. The LIDAR 13 is a sensor that measures a distance from a reflection time by irradiating infrared laser light in a pulse shape. In the present embodiment, three types of sensors are given as examples of the surroundings monitoring sensor 10, but the surroundings monitoring sensor 10 is not limited to these sensors.

  The in-vehicle camera 20 is a camera that is provided on a dashboard and captures an image of a driver. An infrared camera is used as the in-vehicle camera 20 so that shooting can be performed even at night. The arithmetic processing unit 21 performs a process of detecting a gaze target that the driver is gazing from, based on the driver's line of sight and information on the surrounding situation obtained by the surrounding monitoring sensor 10. Specifically, the arithmetic processing unit 21 detects the direction of the driver's line of sight based on the movement of the driver's eyeball photographed by the in-vehicle camera 20. A near-infrared point light source may be provided adjacent to the in-vehicle camera 20 in order to accurately determine the movement of the driver's eyeball. The light from the point light source is reflected on the surface of the driver's eyeball, but since the eyeball is spherical, the position of the reflection point does not change depending on the direction of the eyeball. The direction of the eyeball can be determined with high accuracy depending on where the reflection point is located in the pupil.

  There is an error of several degrees to about 10 degrees in the line-of-sight direction obtained by the in-vehicle camera 20. This is because an image of the in-vehicle camera 20 taken from a place away from the driver is used. When there are a plurality of objects within the range of the error, it is difficult for the driver to specify which object.

  Therefore, the arithmetic processing unit 21 of the present embodiment uses the information on the environment in order to obtain the gaze target in the line of sight. In the present embodiment, the distance to the target and the relative speed are used as environment information. Environmental information is obtained using sensor information obtained from the surrounding monitoring sensor 10. That is, the arithmetic processing unit 21 calculates the distance to the target in the line of sight and the relative speed between the target and the host vehicle based on the sensor information of the LIDAR 13 and the millimeter wave radar 12. The arithmetic processing unit 21 calculates the risk of each object based on the distance to the object and the relative speed.

  Here, as an example, it is assumed that there are two objects, a pedestrian and a preceding vehicle, in the line of sight, and it is difficult to determine which one is gazing from only the line of sight. For example, when the distance between the pedestrian and the own vehicle is 40 m and the distance between the preceding vehicle and the own vehicle is 30 m, it is determined that the preceding vehicle closer to the own vehicle is more dangerous. For example, when the preceding vehicle is traveling at a higher speed than the own vehicle, the preceding vehicle is away, while when the pedestrian is stopped, the pedestrian is relatively driven at the own vehicle speed. It is approaching. In this case, it is determined that the pedestrian is more dangerous. The arithmetic processing unit 21 performs such processing to set a risk for the own vehicle for each object. Here, the distance and the relative speed to the target are listed as factors that determine the risk, but both the distance and the relative speed may be used, or only one of them may be used.

  Then, the arithmetic processing unit 21 detects the gaze target based on the gaze direction information and the risk information of the target. As an example, the arithmetic processing unit 21 determines the possibility of a pedestrian or a preceding vehicle obtained from the line of sight for a model as shown in FIG. 2 in which the node of the line of sight and the environment and the node of the gaze target are connected with a predetermined weight. Then, the pedestrian and the possibility of the preceding vehicle obtained according to the environment are input, and the gaze target is obtained.

  In the example illustrated in FIG. 2, in the determination based on the line of sight, there is a possibility that the pedestrian is “0.5” and the preceding vehicle is “0.5”, and the gaze target cannot be specified by the line of sight alone. In the judgment based on the environment, the pedestrian is “0.75” and the preceding vehicle is “0.25”. Then, by using the information of both, for example, the pedestrian is “0.75”, the preceding vehicle is “0.25”, and the pedestrian is obtained as the object to be watched. In this example, since there is no difference between the probability distribution of the pedestrian and the preceding vehicle based on the line of sight, the probability distribution of the pedestrian and the preceding vehicle based on the environment is reflected in the result.

  The alarm output unit 22 outputs an alarm when the operation processing unit 21 detects that the watch target is detected, for example, when it is determined that the driver is not paying attention to the target to be watched. Call attention. The mode of the warning may be a sound or a display indicating a direction to be watched.

  FIG. 3 is a flowchart illustrating an operation of the gaze target detecting device 1 according to the first embodiment. The gaze target detection device 1 photographs the driver with the in-vehicle camera 20 (S10), and detects the driver's line of sight based on the captured video (S11). Further, the gaze target detecting device 1 monitors the periphery of the vehicle with the periphery monitoring sensor 10 (S12), and detects an object in the periphery (S13).

  Subsequently, the gazing object detection device 1 detects an object in the direction of the detected line of sight and determines whether or not the number of objects in the line of sight is one (S14). When there is only one target object in the line of sight, the target object is determined to be a target to be watched (S15). When the number of objects in the line of sight is two or more, the gaze target object detection device 1 calculates the distance to each object and the relative speed (S16). Then, the gaze target detection device 1 detects the gaze target based on the gaze direction and the information on the environment (S17).

  Although the configuration and operation of the gazing object detection device 1 according to the present embodiment have been described, an example of hardware of the gazing object detection device 1 is an ECU including a CPU, a RAM, a ROM, a communication interface, and the like. . A program having a module for realizing each of the above-described functions is stored in a RAM or a ROM, and the program is executed by the CPU, whereby the above-described gaze-target detecting device 1 is realized. Such a program is also included in the scope of the present invention.

  Next, effects of the gaze target detecting device 1 of the present embodiment will be described. In the invention described in Japanese Patent Application Laid-Open No. 2018-67198 exemplarily described as the background art, an alarm is output when the vehicle does not match the previously stored combination of the driving state, the driving operation, and the line-of-sight pattern. It does not try to detect what the driver is watching. Even in this background art, unless the accuracy of the line-of-sight detection is sufficient, it is not possible to detect the object that the driver is gazing at.

  The gaze target detecting device 1 according to the present embodiment can be used when the gaze target cannot be specified only from the driver's gaze due to measurement errors in the gaze direction and the positional relationship of the target only from the information of the in-vehicle camera 20. Also, by using information on the environment (distance or relative speed between each object and the host vehicle), the object to be watched can be appropriately detected.

  The gaze target detecting device 1 of the present embodiment obtains the target as the gaze target when there is only one target in the gaze direction detected from the image of the driver by the in-vehicle camera 20. Calculation processing can be reduced without performing processing based on environmental information. Note that, in the present embodiment, an example in which environmental information is used when the number of objects is two or more has been described, but a gaze target may be obtained using environment information regardless of the number of objects in the line of sight. .

(Second embodiment)
FIG. 4 is a diagram illustrating a configuration of the gaze target detecting device 2 according to the second embodiment. The gaze target detecting device 2 according to the second embodiment is connected to the on-vehicle sensor 30, and detects the gaze target based on operation information obtained from the on-vehicle sensor 30.

  FIG. 5 is a diagram illustrating an example of a model for obtaining a gazing target used in the gazing target detecting device 2 according to the second embodiment. In the second embodiment, in addition to the nodes of the line of sight and the environment, a node of operation information is provided. The possibility of left and right pedestrians and the preceding vehicle determined by the line of sight and the possibility of left and right pedestrians and the preceding vehicle determined by the environment are input to the nodes of the line of sight and the environment. In this embodiment, accelerator / brake ON / OFF information is input to the operation information node. In this example, in the determination based on the line of sight, the left pedestrian is “0.7”, the preceding vehicle is “0.2”, the right pedestrian is “0.1”, and the operation information is both accelerator and brake. In the determination based on the environment, the left pedestrian and the right pedestrian are both "0.5", and the preceding vehicle and the others are "0" in the determination based on the environment. In the probability distribution based on the environment, there is no difference between the left and right pedestrians, and since the preceding vehicle and the others are “0”, the target gazing object is “0.875” for the left pedestrian and “0” for the right pedestrian. .125 ", which indicates that the user is gazing at the left pedestrian. As another example, if the accelerator is “ON” in the operation information, the attention distribution to the vicinity of the vehicle is considered to be half, and the gazing target is “0.4375” for the left pedestrian and the right pedestrian for the pedestrian. Can be obtained as "0.0625", and the others as "0.5". The calculation shown here is an example, and this model may be represented by a machine learning model such as a Bayesian network or SVM. The arithmetic processing unit 21 obtains a gaze target based on the input information.

  FIG. 6 is a flowchart illustrating the operation of the gaze target detecting device 2 according to the second embodiment. The gaze target detection device photographs the driver with the in-vehicle camera 20 (S20), and detects the driver's line of sight based on the captured video (S21). Further, the gaze target detection device 2 monitors the periphery of the vehicle with the periphery monitoring sensor 10 (S22), and detects a target nearby (S23).

  Subsequently, the gaze target detecting device 2 detects the target in the direction of the detected line of sight, and determines whether or not the number of objects in the line of sight is one (S24). If there is only one object in the line of sight, the object is determined to be a gazing object (S25). When the number of objects in the line of sight is two or more, the watch target object detection device 2 calculates the distance to each object and the relative speed (S26). The target object detection device 2 acquires operation information from the vehicle-mounted sensor 30 (S27). Then, the gaze target detection device 2 detects the gaze target based on the gaze direction, the environment, and the operation information (S28).

  As described above, the gaze target detecting device 2 of the second embodiment can detect the gaze target with higher accuracy by using the operation information in addition to the information on the line of sight and the environment. For example, as in the example shown in FIG. 5, if operation information indicating that the accelerator is turned off without inputting the brake is input, the driver recognizes the pedestrian, determines that there is a danger, and presses the accelerator. It is probable that the player has released his foot from, and is now in an operating state where he can step on the brakes at any time. That is, it is possible to infer the recognition and judgment of the driver from the operation state. By using a model including operation information as a node, the detection accuracy of the gaze target increases.

(Third embodiment)
FIG. 7 is a diagram illustrating the configuration of the gaze target detecting device 3 according to the third embodiment. The basic configuration of the gaze target detecting device 3 according to the third embodiment is the same as that of the first embodiment, but includes a risk information storage unit 23 that can be accessed by the arithmetic processing unit 21. Are different.

  FIG. 8 is a diagram illustrating an example of data stored in the risk information storage unit 23. As shown in FIG. 8, the risk information storage unit 23 stores the type of the target object and the magnitude of the risk corresponding thereto. The type of the object is a classification of the object that is considered to be on the road. There are pedestrians, bicycles, preceding vehicles, oncoming vehicles, and the like. The risk information storage unit 23 has data of the magnitude of risk for each type. For example, pedestrians and bicycles are “8”, preceding vehicles are “6”, and oncoming vehicles are “4”.

  When there are a plurality of targets on the driver's line of sight, the arithmetic processing unit 21 reads out the magnitude of the risk corresponding to the type of the targets from the risk information storage unit 23. In the gaze target detecting device 1 according to the first embodiment, the risk of each target is calculated based on the distance to the target and the relative speed. However, instead of this, the risk information storage is performed in the present embodiment. The gaze target is obtained using the magnitude of the risk read from the unit 23.

  FIG. 9 is a flowchart illustrating an operation of the gazing object detection device 3 according to the third embodiment. The gaze target detecting device 3 photographs the driver with the in-vehicle camera 20 (S30), and detects the driver's line of sight based on the captured video (S31). The target object detection device 3 monitors the periphery of the vehicle with the periphery monitoring sensor 10 (S32), and detects an object in the vicinity (S33).

  Subsequently, the gaze target detecting device 3 detects the target in the direction of the detected line of sight, and determines whether or not the number of objects in the line of sight is one (S34). When there is only one target object in the line of sight, the target object is determined to be a target to be watched (S35). When the number of objects in the line of sight is two or more, the gaze target object detection device 3 specifies the type of each object (S36), and outputs information on the magnitude of the risk corresponding to the type to the risk information. It is read from the storage unit 23 (S37). The type of the target object may be specified by pattern matching of the target object reflected on the camera 11 outside the vehicle, or may be estimated using a learned model that classifies the type of the target object. Then, the gaze target detecting device 3 detects the gaze target based on the gaze direction and the information on the magnitude of the risk (S38).

  The gaze target detection device 3 of the present embodiment appropriately determines the gaze target that the driver is gazes based on the gaze direction detected from the driver's image by the in-vehicle camera 20 and the type of the target. Can be detected.

As described above, the gaze target detecting device according to the embodiment of the present invention has been described in detail with reference to the embodiment, but the present invention is not limited to the above embodiment.
The arithmetic processing units 21 of the gaze target detecting devices 1 to 3 according to the present embodiment can determine not only the driver's line of sight but also the driver's face orientation based on the video of the in-vehicle camera 20. According to human visual characteristics, it has been found that when a human looks outward, the line of sight is more outward than the face. According to this, it can be said that, when the face orientation is looking outward beyond a certain level, the line of sight is looking further outward. Using this characteristic, the arithmetic processing unit 21 detects the driver's face direction from the image of the in-vehicle camera 20, and when the face direction is outside a predetermined angle, it is needless to detect the line of sight. , May be determined to be inattentive.

  Note that the face direction differs from the line-of-sight direction, and there are individual differences due to the seat position where the driver sits and the sitting height of the driver. When performing the process of determining the face direction, a process of correcting the front direction of the face direction may be performed. As a specific method of performing the correction, for example, it is conceivable that the face direction when traveling on a straight road (that is, the steering angle is 0 °) is set to the front direction. Further, in the future, when a vehicle that performs iris authentication at the time of starting the engine of the car appears, it is conceivable that the face orientation when performing iris authentication is set to the front direction.

  According to human visual characteristics, it has been found that it takes a certain time (about 100 ms to 300 ms) for a human to gaze at a specific target. By utilizing this characteristic, when the time during which the line of sight stops and the object is visually recognized is equal to or shorter than a predetermined time (for example, 100 ms), it may be determined that the object is not being watched. Further, a condition that there is no high-speed or large gaze movement (saccade) may be used for the gaze determination of the target object.

  INDUSTRIAL APPLICABILITY The present invention is useful as a gaze target detection device for detecting a target gaze at a driver.

1-3 Target object detection unit 10 Surrounding monitoring sensor 11 Outside camera 12 Millimeter wave radar 13 LIDAR
Reference Signs List 20 in-car camera 21 arithmetic processing unit 22 alarm output unit 23 risk information storage unit 30 in-vehicle sensor

Claims (9)

A surroundings monitoring sensor for detecting an object around the vehicle,
A camera that captures the driver,
The driver's gaze direction based on the driver's gaze direction obtained based on the image captured by the camera and the distance or relative speed between each object and the host vehicle detected by the surrounding monitoring sensor. An arithmetic processing unit for finding an object
A gaze target detection device comprising:   When there is only a single object in the driver's line of sight, the arithmetic processing unit is in the driver's line of sight without using data on the distance or relative speed between each object and the host vehicle. The gaze target detecting device according to claim 1, wherein the gaze target is obtained as the gaze target. A peripheral surveillance camera that shoots objects around the vehicle,
A camera that captures the driver,
A risk information storage unit that stores the type of the object and the magnitude of the risk for the type,
The type of the object in the vicinity is specified based on the image of the surrounding monitoring camera, and the magnitude of the risk of each object is determined based on the information stored in the risk information storage unit. Based on the size of and the driver's line of sight determined based on the image captured by the camera, an arithmetic processing unit that determines an object that the driver is gazing at,
A gaze target detection device comprising: Equipped with an in-vehicle sensor that acquires operation information for the vehicle,
The arithmetic processing unit,
The gaze target detecting device according to any one of claims 1 to 3, wherein a gaze target that is gazed at by a driver is obtained based on the operation information acquired by the on-vehicle sensor.   5. The arithmetic processing unit according to claim 1, wherein the direction of the driver's face is determined based on an image captured by the camera, and when the driver's face is at a predetermined angle or more, it is determined that the driver is looking aside. The gaze target detecting device according to any one of the above.   The gazing object detection device according to claim 4, wherein the arithmetic processing unit determines the front direction of the face direction based on the face direction when the driver sitting in the driver's seat faces forward.   When the dwell time of the driver's line of sight is equal to or less than a predetermined threshold, the arithmetic processing unit determines that the object in the line of sight is not gazing, and obtains the gazing object. The gaze target detecting device according to any one of the above. A method for detecting an object that a driver is gazing at,
Detecting an object in the vicinity of the vehicle with the surrounding monitoring sensor;
Photographing the driver with a camera;
The driver's gaze direction is determined based on the driver's line of sight direction determined based on the image captured by the camera, and the distance or relative speed between each object and the host vehicle detected by the surrounding monitoring sensor. Seeking an object that is
A gaze target detection method comprising: A program for detecting an object that the driver is gazing at.
Obtaining data of an object around the vehicle detected by the surrounding monitoring sensor;
Obtaining an image of the driver taken by the camera;
The driver's gaze direction based on the driver's gaze direction obtained based on the image captured by the camera and the distance or relative speed between each object and the host vehicle detected by the surrounding monitoring sensor. Seeking an object that is
A program that executes

Images