CN110458104A - The human eye sight direction of human eye sight detection system determines method and system - Google Patents
The human eye sight direction of human eye sight detection system determines method and system Download PDFInfo
- Publication number
- CN110458104A CN110458104A CN201910741288.0A CN201910741288A CN110458104A CN 110458104 A CN110458104 A CN 110458104A CN 201910741288 A CN201910741288 A CN 201910741288A CN 110458104 A CN110458104 A CN 110458104A
- Authority
- CN
- China
- Prior art keywords
- coordinate
- human eye
- coordinate system
- steering engine
- dimensional world
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/59—Context or environment of the image inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
- G06V20/597—Recognising the driver's state or behaviour, e.g. attention or drowsiness
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/18—Eye characteristics, e.g. of the iris
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/18—Eye characteristics, e.g. of the iris
- G06V40/193—Preprocessing; Feature extraction
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Ophthalmology & Optometry (AREA)
- Human Computer Interaction (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Length Measuring Devices By Optical Means (AREA)
- Image Analysis (AREA)
Abstract
The embodiment of the invention provides a kind of human eye sight directions of human eye sight detection system to determine method and system, the human eye sight detection system includes: multiple video cameras, steering engine holder, with the laser ranging system that the steering engine holder is arranged in, the described method includes: determining coordinate of the hot spot of laser ranging system transmitting in the three-dimensional world coordinate system using the rotation center of steering engine holder as coordinate origin, obtain multiple user images of multiple video camera acquisitions, to determine coordinate of the human eye in three-dimensional world coordinate system, based on hot spot coordinate and human eye coordinates, determine direction vector of the human eye sight in three-dimensional world coordinate system.By the way that steering engine holder is arranged, allow to collect the human eye sight image data of continuous orientation angle, but also can be suitably used for various scenes, such as narrow automobile cab etc..
Description
Technical field
The present invention relates to technical field of computer vision, more particularly to a kind of human eye sight of human eye sight detection system
Direction determines method and a kind of human eye sight detection system.
Background technique
Currently, human eye sight angle detecting has become a heat subject of computer vision field, with deep learning
The development of technology, the human eye sight angle detecting scheme based on appearance are increasingly becoming the mainstream scheme of the project.But human eye regards
The collection work of line bearing data is extremely difficult.
Existing human eye sight bearing data collection scheme has: 1, being collected object and head position is kept to fix, go to see several
The camera of a fixed position, while image data under cameras record.2, it is collected object to go to see the several points of fixed position, use
More mesh cameras record data obtain human eye sight direction by calculating the spatial position of human eye and fixed point.3, with a large-size screen monitors
Random display two differentiable points of curtain, collected subject face face one of point, and eyes go to see another point, use phase
Data under machine An arrayed recording, the relative spatial co-ordinates by calculating human eye and random point obtain direction of visual lines.1st, 2 kind of scheme can
With different scenes (such as in automobile cab small space) acquire sight data, but direction of visual lines angle be it is discrete,
The natural law of sight is not met.Although the 3rd kind of scheme can collect the sight data of continuous orientation angle, scene is being collected
There are considerable restraints for aspect.
Summary of the invention
In view of the above problems, it proposes the embodiment of the present invention and overcomes the above problem or at least partly in order to provide one kind
A kind of human eye sight direction of the human eye sight detection system to solve the above problems determines method and a kind of corresponding human eye sight
Detection system.
To solve the above-mentioned problems, the embodiment of the invention discloses a kind of human eye sight directions of human eye sight detection system
Determine method, the human eye sight detection system includes: multiple video cameras, steering engine holder and is arranged in swashing for the steering engine holder
Optical range finding apparatus, which comprises
Determine the hot spot of the laser ranging system transmitting using the rotation center of the steering engine holder as coordinate origin
Coordinate in three-dimensional world coordinate system；
Multiple user images of the multiple video camera acquisition are obtained, to determine human eye in the three-dimensional world coordinate system
Coordinate；
Based on the hot spot coordinate and human eye coordinates, determine direction of the human eye sight in the three-dimensional world coordinate system to
Amount.
Optionally, the steering engine holder is made of orthogonal two steering engines in direction of rotation；The determination laser
Seat of the hot spot of range unit transmitting in the three-dimensional world coordinate system using the rotation center of the steering engine holder as coordinate origin
Mark, comprising:
Obtain the rotation angle of the steering engine holder when laser ranging system launch spot；
Determine the hot spot to the steering engine holder rotation center distance；
Using the distance and rotation angle, determine the hot spot using the rotation center of the steering engine holder as coordinate original
Coordinate in the three-dimensional world coordinate system of point.
Optionally, the multiple user images for obtaining the multiple video camera acquisition, to determine human eye in the three-dimensional
Coordinate in world coordinate system, comprising:
Determine coordinate of multiple eye images in pixel coordinate system in the multiple user images；
By coordinate of the multiple eye image in pixel coordinate system, multiple eye images are converted into the three-dimensional generation
Coordinate in boundary's coordinate system；
By the origin of the camera coordinates system of the multiple video camera, it is converted into multiple in three-dimensional world coordinate system
Camera origin；
Respectively according to the multiple camera origin and corresponding eye image in the three-dimensional world coordinate system
Coordinate determines coordinate of the human eye in the three-dimensional world coordinate system.
Optionally, the coordinate by the multiple eye image in camera coordinates system, is converted into multiple eye images
Coordinate in the three-dimensional world coordinate system, comprising:
Using the transformational relation between preset pixel coordinate system and image coordinate system, by the multiple eye image in picture
Coordinate in plain coordinate system converts coordinate of the multiple eye image in image coordinate system；
Using the transformational relation between preset image coordinate system and camera coordinates system, the multiple eye image is being schemed
As the coordinate in coordinate system, it is converted into coordinate of the multiple eye image in camera coordinates system；
Using the transformational relation between preset camera coordinates system and three-dimensional world coordinate system, by multiple eye images in phase
Coordinate in machine coordinate system is converted into coordinate of the multiple eye image in the three-dimensional world coordinate system.
Optionally, described respectively according to the multiple camera origin and corresponding eye image in the three-dimensional world
Coordinate in coordinate system determines that coordinate of the human eye in the three-dimensional world coordinate system further comprises:
Respectively according to the multiple camera origin and corresponding eye image in the three-dimensional world coordinate system
Coordinate determines a plurality of straight line；
Calculate the coordinate in the closest approach of a plurality of straight line；The closest approach be with the straight line of a plurality of straight line away from
From shortest point；
The coordinate for determining the closest approach of a plurality of straight line is coordinate of the human eye in the three-dimensional world coordinate system.
Optionally, further includes:
Using direction vector of the human eye sight in the three-dimensional world coordinate system and the user images, people is generated
An eye line image.
Optionally, the human eye sight detection system further includes host computer and steering engine control panel, the host computer with it is described
Steering engine control panel and laser ranging system connection；The steering engine holder is connect with the steering engine control panel；
The human eye sight direction of the human eye sight detection system determines that method is applied to the host computer, the steering engine cloud
The rotation angle of platform is controlled by the steering engine control panel.
Optionally, further includes:
Obtain the positional relationship information between the multiple video camera；
Using the positional relationship information between the multiple video camera, determine the camera coordinates system of the multiple video camera with
The transformational relation of the three-dimensional world coordinate system.
The embodiment of the invention also discloses a kind of human eye sight detection system, on the human eye sight detection system includes:
Position machine, multiple video cameras, steering engine holder and the laser ranging system that the steering engine holder is arranged in, the host computer include:
Hot spot coordinate determining module, for determining the hot spot of the laser ranging system transmitting with the steering engine holder
Rotation center is the coordinate in the three-dimensional world coordinate system of coordinate origin；
Human eye coordinates determining module, for obtaining multiple user images of the multiple video camera acquisition, to determine human eye
Coordinate in the three-dimensional world coordinate system；
Direction vector determining module determines human eye sight described three for being based on the hot spot coordinate and human eye coordinates
Tie up the direction vector in world coordinate system.
Optionally, the steering engine holder is made of orthogonal two steering engines in direction of rotation；The hot spot coordinate determines
Module, comprising:
Rotate angle acquisition submodule, the rotation of steering engine holder when for obtaining the laser ranging system launch spot
Gyration；
Distance determine submodule, for determine the hot spot to the steering engine holder rotation center distance；
Hot spot coordinate determines submodule, for determining the hot spot with the rudder using the distance and rotation angle
The rotation center of machine head is the coordinate in the three-dimensional world coordinate system of coordinate origin.
Optionally, the human eye coordinates determining module, comprising:
First human eye coordinates determine submodule, for determining multiple eye images in the multiple user images in pixel
Coordinate in coordinate system；
Human eye coordinates convert submodule, for the coordinate by the multiple eye image in pixel coordinate system, convert institute
State the coordinate for multiple eye images in the three-dimensional world coordinate system；
Origin converts submodule, for being converted into the origin of the camera coordinates system of the multiple video camera
Multiple camera origins in three-dimensional world coordinate system；
Second human eye coordinates determine submodule, for respectively according to the multiple camera origin and corresponding human eye figure
As the coordinate in the three-dimensional world coordinate system, coordinate of the human eye in the three-dimensional world coordinate system is determined.
Optionally, the human eye coordinates convert submodule, comprising:
First coordinate transformation unit, for using the transformational relation between preset pixel coordinate system and image coordinate system,
By coordinate of the multiple eye image in pixel coordinate system, seat of the multiple eye image in image coordinate system is converted
Mark；
Second coordinate transformation unit, for using the transformational relation between preset image coordinate system and camera coordinates system,
By coordinate of the multiple eye image in image coordinate system, the multiple eye image is converted into camera coordinates system
Coordinate；
Third coordinate transformation unit, for being closed using the conversion between preset camera coordinates system and three-dimensional world coordinate system
System, by coordinate of multiple eye images in camera coordinates system, is converted into the multiple eye image and sits in the three-dimensional world
Coordinate in mark system.
Optionally, second human eye coordinates determine submodule, comprising:
Straight line determination unit, for respectively according to the multiple camera origin and corresponding eye image described three
The coordinate in world coordinate system is tieed up, determines a plurality of straight line；
Coordinate calculating unit, the coordinate in the closest approach for calculating a plurality of straight line；The closest approach for institute
State the shortest point of linear distance of a plurality of straight line；
Coordinate determination unit, for determining that the coordinate in closest approach of a plurality of straight line is human eye in the three-dimensional world
Coordinate in coordinate system.
Optionally, further includes:
Human eye sight image generation module, for the direction using the human eye sight in the three-dimensional world coordinate system
User images described in vector sum generate human eye sight image.
Optionally, the human eye sight detection system further includes steering engine control panel, and the host computer and the steering engine control
Plate and laser ranging system connection；The steering engine holder is connect with the steering engine control panel；
The human eye sight direction of the human eye sight detection system determines that method is applied to the host computer, the steering engine cloud
The rotation angle of platform is controlled by the steering engine control panel.
Optionally, further includes:
Relation acquisition module, for obtaining the positional relationship information between the multiple video camera；
Transformational relation determining module, for determining described more using the positional relationship information between the multiple video camera
The transformational relation of the camera coordinates system of a video camera and the three-dimensional world coordinate system.
The embodiment of the invention also discloses a kind of devices, comprising: processor, memory and is stored on the memory simultaneously
The computer program that can be run on the processor realizes such as this hair when the computer program is executed by the processor
The step of human eye sight direction of human eye sight detection system described in bright embodiment determines method.
The embodiment of the invention also discloses a kind of computer readable storage medium, deposited on the computer readable storage medium
Computer program is stored up, the computer program realizes human eye sight detection as described in the embodiments of the present invention when being executed by processor
The step of human eye sight direction of system determines method.
The embodiment of the present invention includes following advantages:
In embodiments of the present invention, by determining the hot spot of laser ranging system transmitting with the rotation center of steering engine holder
For the coordinate in the three-dimensional world coordinate system of coordinate origin, multiple user images of multiple video camera acquisitions are obtained, to determine people
Coordinate of the eye in three-dimensional world coordinate system, is based on hot spot coordinate and human eye coordinates, determines human eye sight in three-dimensional world coordinate
Direction vector in system.By setting steering engine holder, allow to collect the human eye sight image data of continuous orientation angle, and
And it also can be suitably used for various scenes, such as narrow automobile cab etc..
Detailed description of the invention
Fig. 1 is the step of a kind of human eye sight direction of human eye sight detection system of the invention determines embodiment of the method stream
Cheng Tu；
Fig. 2 is a kind of structural schematic diagram of human eye sight detection system of the invention；
Fig. 3 is a kind of human eye sight detection system material object schematic diagram simply installed under laboratory scene of the invention；
Fig. 4 is a kind of human eye sight image of acquisition of the invention；
Fig. 5 is a kind of structural block diagram of human eye sight detection system embodiment of the invention.
Specific embodiment
In order to make the foregoing objectives, features and advantages of the present invention clearer and more comprehensible, with reference to the accompanying drawing and specific real
Applying mode, the present invention is described in further detail.
Referring to Fig.1, the human eye sight direction for showing a kind of human eye sight detection system of the invention determines that method is implemented
The step flow chart of example, the human eye sight detection system include: multiple video cameras, steering engine holder and are arranged in the steering engine cloud
The laser ranging system of platform.
The structural schematic diagram of a kind of human eye sight detection system of the invention as shown in Figure 2.In Fig. 2, human eye sight
Detection system includes: host computer (1), steering engine control device (2), laser ranging system (3), and two steering engines (4,5) and three take the photograph
Camera (6,7,8).
Wherein, host computer can be the control centre of human eye sight detection system, can with execution order and carry out a system
The calculating of column.Host computer (1) can be connected by TTL serial ports with steering engine control device (2), have fixed communication therebetween
Agreement, host computer can carry out any rotation to steering engine holder according to the communication protocol, be calibrated and compensated rotation error, true
Determine the controls operations such as slewing area and rotating manner, control rotation speed, wherein host computer is to the control of steering engine by steering engine
Control device sends commands to realize.
It should be noted that steering engine control device (2) can control more steering engines simultaneously, for example, 6 steering engines of control, steering engine
There is the connection pin of 6 group # on control device (2), the steering engine of corresponding control, the embodiment of the present invention can be found by the number
This is not intended to limit.In embodiments of the present invention, two steering engines are controlled with steering engine control device (2) to be illustrated.Steering engine control
Device (2) is connect with two steering engines (4,5) by two supply lines and a signal wire.It mutually hangs down the direction of rotation of two steering engines
Directly.
Laser ranging system (3) is mounted on steering engine holder, and when installation is necessary to ensure that laser beam just by two rudders
The shaft of machine (4,5), and direction when direction with two steering engines (4,5) is 0 degree is consistent.Laser ranging system (3)
It is connected directly with host computer (1) by TTL serial ports, equally has fixed communication protocol therebetween, host computer can be to laser
Range unit (3) carries out opening and closing light beam, measurement apart from these three simple control operations.Three video cameras (6,7,8)
It is fixed on suitable position, is directly controlled by host computer (1).
When carrying out the acquisition of human eye sight direction, collected object surface is sitting in behind steering engine holder against video camera.Such as
Fig. 3 shows a kind of human eye sight detection system material object schematic diagram simply installed under laboratory scene of the invention.
The human eye sight direction of the human eye sight detection system determines that method can specifically include following steps:
Step 101, the hot spot for determining the laser ranging system transmitting is being to sit with the rotation center of the steering engine holder
Mark the coordinate in the three-dimensional world coordinate system of origin.
Steering engine holder can be pivoted about with a point, which is the rotation center of steering engine holder.It is arranged and arrives
The laser ranging system of steering engine holder follows steering engine holder to rotate together with, so as to emit laser in multiple angles.In this hair
In bright embodiment, a three-dimensional world coordinate system can be established using the rotation center of steering engine holder as coordinate origin.
Laser ranging system can be with launch spot, and measures the distance between the hot spot and laser ranging system.
Hot spot can be calculated by the distance between the hot spot of acquisition laser ranging system measurement and laser ranging system
Coordinate in three-dimensional world coordinate system.It should be noted that calculating for convenience, laser ranging system can be set and be deployed in
The position of the rotation center of steering engine holder, at this point it is possible to which think laser ranging system measurement is the rotation of hot spot Yu steering engine holder
Turn the distance between center, directly acquire laser ranging system measurement hot spot and steering engine holder rotation center between away from
From coordinate of the calculating hot spot in three-dimensional world coordinate system.
In addition, can also be filled according to laser ranging even if laser ranging system is not placed into the rotation center of steering engine holder
The distance of the rotation center of steering engine holder is set, coordinate of the hot spot in three-dimensional world coordinate system is corrected.
In one preferred embodiment of the invention, the steering engine holder may include by Plane of rotation orthogonal two
Platform steering engine；E.g., including a steering engine horizontally rotated and a steering engine vertically rotated.
The step 101 may include following sub-step:
Obtain the rotation angle of the steering engine holder when laser ranging system launch spot；Determine the hot spot to institute
State the distance of the rotation center of steering engine holder；Using the distance and rotation angle, determine the hot spot with the steering engine cloud
The rotation center of platform is the coordinate in the three-dimensional world coordinate system of coordinate origin.
In embodiments of the present invention, laser ranging system is arranged on two steering engines, can be with when two steering engines rotate
Laser ranging system movement is driven, to change the projected position of the hot spot of laser ranging system transmitting.
Specifically, two rotation angle, θs can be generated at random,If rotation angle is all 90 degree, corresponding steering engine step
A length of Na、Nb, then rotating angle is θ,When steering engine step value V1, V2Meet following equation respectively:
V190 × N of=θ ÷a；
It, can be by steering engine step value V when by needing steering engine control device to control two steering engines1, V2It encodes into steering engine
In command statement, rotation order is generated.It is emitted to steering engine control device by serial ports, two of control flaps machine head can be completed
Steering engine carries out rotation process, meanwhile, rotation angle, θ is recorded,
After two steering engines complete the rotation of random angles, ranging order, laser ranging can be sent to laser ranging system
After device receives the ranging order, emit laser (alternatively launch spot), laser, can be in obstacle after encountering barrier
Hot spot is formed on object.Laser ranging system can further measure the distance between the rotation center of hot spot and steering engine holder L.
It in embodiments of the present invention, can be using the distance between the rotation center of hot spot and steering engine holder L and rotation angle
θ is spent,To calculate hot spot coordinate in three-dimensional world coordinate system.
Specifically, coordinate of the hot spot in three-dimensional world coordinate system meets following equation:
Wherein, θ,Indicate the rotation angle of steering engine holder, L indicate between hot spot and the rotation center of steering engine holder away from
From P is coordinate of the hot spot in three-dimensional world coordinate system.
By the distance L of the measurement and θ of record,Value substitute into coordinate of the above-mentioned hot spot in three-dimensional world coordinate system and meet
Formula, it can obtain coordinate of the hot spot in three-dimensional world coordinate system.
In one preferred embodiment of the invention, the human eye sight detection system further includes host computer and steering engine control
Plate, the host computer and the steering engine control panel and the laser ranging system connect；The steering engine holder and the steering engine
Control panel connection；The human eye sight direction of the human eye sight detection system determines that method is applied to the host computer, the rudder
The rotation angle of machine head is controlled by the steering engine control panel.
Step 102, multiple user images of the multiple video camera acquisition are obtained, to determine human eye in the three-dimensional world
Coordinate in coordinate system.
In laser ranging system launch spot, multiple video cameras of deployment can acquire user images respectively.It is acquiring
When user images, collected object watches the hot spot left in front obstacle at this time, while each video camera acquisition user's figure attentively
Picture completes shooting task.
In order to ensure when acquiring user images, the eyes (human eye) of collected object watch hot spot attentively, can be set
Each hot spot can flash at random it is several under, the person of watching attentively needs to say hot spot and flashes number, watches behavior attentively with ensure collected object
Occur.
It in embodiments of the present invention, can be using the eye image in multiple user images, to determine human eye in three-dimensional generation
Coordinate in boundary's coordinate system.Wherein, eye image can be the image in the user images of acquisition comprising human eye position.
In a preferred embodiment of an embodiment of the present invention, the step 102 may include following sub-step:
Sub-step S11 determines multiple eye images in the multiple user images in this coordinate of pixel coordinate system；
In embodiments of the present invention, it to determine coordinate of the human eye in three-dimensional world coordinate system, can determine first multiple
Coordinate of multiple eye images in pixel coordinate system in user images.
Wherein, the coordinate origin of pixel coordinate system is the upper left corner of user images, is to the right X-axis positive direction from the upper left corner,
It is downwards Y-axis positive direction from the upper left corner.Position P of the eyes in eye image can be marked manually, sat to obtain P in pixel
Coordinate in mark system.It should be noted that in the specific implementation, can also according to certain rules automatic marking eyes in human eye
Position P in image, for example, can be using position P of the automatic marking eyes in eye image as the center of eye image, this
Inventive embodiments are not intended to limit this.
Sub-step S12, by coordinate of the multiple eye image in pixel coordinate system, the described conversion is multiple human eye figures
As the coordinate in the three-dimensional world coordinate system；
It, can be further after determining coordinate of multiple eye images in pixel coordinate system in multiple user images
According to preset transformational relation, by coordinate of multiple eye images in pixel coordinate system, the described conversion is multiple eye images
Coordinate in the three-dimensional world coordinate system.
In a preferred embodiment of an embodiment of the present invention, the sub-step S12 may include steps of:
Using the transformational relation between preset pixel coordinate system and image coordinate system, by the multiple eye image in picture
Coordinate in plain coordinate system converts coordinate of the multiple eye image in image coordinate system；Using preset image coordinate
Coordinate of the multiple eye image in image coordinate system is converted institute by transformational relation between system and camera coordinates system
State coordinate of multiple eye images in camera coordinates system；Using between preset camera coordinates system and three-dimensional world coordinate system
Transformational relation converts the multiple eye image in the three-dimensional for coordinate of multiple eye images in camera coordinates system
Coordinate in world coordinate system.
Wherein, the origin of camera coordinates system is the pin hole of video camera, and the direction for being parallel to the X-axis of pixel coordinate system is camera
The direction of the X-axis of coordinate system is parallel to the direction of the Y-axis of pixel coordinate system as the direction of the Y-axis of camera coordinates system.
In the concrete realization, pixel coordinate system and image coordinate system be all on imaging plane, only respective origin and degree
It is different to measure unit.The origin of image coordinate system is the midpoint of imaging plane under normal conditions, and unit is a millimeter mm, belongs to physics
Unit.And the unit of pixel coordinate system is pixel p ixel, it is all several column of several rows that we describe a pixel usually.So this
Conversion between the two is as follows: wherein dx and dy indicates each column and every a line respectively represent how many mm, i.e. 1pixel=dx
mm。
Transformational relation between preset pixel coordinate system and image coordinate system, preset image coordinate system and camera coordinates
Transformational relation between system, the transformational relation between preset camera coordinates system and three-dimensional world coordinate system are to be directed to each to take the photograph
Transformational relation between the pre-set coordinate system of camera.Specifically, can determine each video camera by camera calibration
Parameter, and determine using the parameter of each video camera the transformational relation between coordinate system.
In embodiments of the present invention, the transformational relation between preset pixel coordinate system and image coordinate system can be used,
By coordinate of multiple eye images in pixel coordinate system, coordinate of multiple eye images in image coordinate system is converted；Using
Transformational relation between preset image coordinate system and camera coordinates system, by seat of multiple eye images in image coordinate system
Mark, is converted into coordinate of multiple eye images in camera coordinates system；Using preset camera coordinates system and three-dimensional world coordinate
Transformational relation between system converts multiple eye images in three-dimensional for coordinate of multiple eye images in camera coordinates system
Coordinate in world coordinate system.
In a kind of embodiment of the embodiment of the present invention, it can also include the following steps:
Obtain the positional relationship information between the multiple video camera；Using the positional relationship between the multiple video camera
Information determines the transformational relation between the camera coordinates system of the multiple video camera and the three-dimensional world coordinate system respectively.
Video camera has the camera coordinates system of itself, and the camera coordinates system of each video camera can be different.Each
Transformational relation between the camera coordinates system of video camera and the three-dimensional world coordinate system needs to determine respectively.
The camera coordinates system of video camera and the transformational relation of three-dimensional world coordinate system, can be by determining video camera in three-dimensional
The pose of world coordinate system determines.Pose includes translation information and rotation information, to be determined in human eye sight detection system
Each video camera at least needs just to can determine that using three video cameras in the pose of three-dimensional world coordinate system.
Specifically, the distance between the multiple video camera that the available laser ranging system measures respectively；
Obtain rotation angle when the distance between the steering engine holder measurement and video camera；Using the multiple distance, the multiple
Rotation angle solves the camera coordinates system of the multiple video camera and the translation information of the three-dimensional world coordinate system and rotation is believed
Breath, so that it is determined that camera coordinates tie up to the pose of three-dimensional world coordinate system.
In embodiments of the present invention, laser ranging system can measure between (laser ranging system) and each video camera away from
From.
In the specific implementation, hot spot can be emitted on video camera, thus measurement hot spot and laser ranging system it
Between distance, as the distance between video camera and laser ranging system.To laser ranging system launch spot to video camera
On, need the steering engine of steering engine holder to carry out the rotation of certain angle, to drive the rotation of laser ranging system progress certain angle
Turn, therefore, available is the rotation angle of steering engine at this time, and the position of video camera is determined using the distance and rotation angle of measurement
Appearance.By the method for semi-automatic calibrating camera pose, the calibration process of video camera is enormously simplified, human eye sight can be improved
The efficiency in the determination human eye sight direction of detection system.
Sub-step S13 converts the origin of the camera coordinates system of the multiple video camera in three-dimensional world coordinate
Multiple camera origins in system；
In embodiments of the present invention, the pin hole (i.e. the origin of camera coordinates system) of each video camera can be determined in three-dimensional generation
Coordinate in boundary's coordinate.
Specifically, can be using the transformational relation between preset camera coordinates system and three-dimensional world coordinate system, it will be more
The origin of the camera coordinates system of a video camera is converted into multiple camera origins in three-dimensional world coordinate system.
Sub-step S14, respectively according to the multiple camera origin and corresponding eye image in the three-dimensional world
Coordinate in coordinate system determines coordinate of the human eye in the three-dimensional world coordinate system.
It in embodiments of the present invention, can be respectively according to each camera origin and corresponding eye image described three
Tie up the coordinate in world coordinate system, coordinate of the human eye in the three-dimensional world coordinate system.
Specifically, can be determined according to the coordinate of each camera origin and eye image in three-dimensional world coordinate system
One straight line, and then coordinate of the human eye in three-dimensional world coordinate system is determined according to straight line determining respectively.Wherein, in every straight line
Coordinate in three-dimensional world coordinate system of camera origin and eye image, corresponding is the use of the same video camera shooting
The origin that coordinate and the user images of the eye image of family image in three-dimensional world coordinate system correspond to camera coordinates system is sat
Mark.
In a preferred embodiment of an embodiment of the present invention, the sub-step S14 may include steps of:
Respectively according to the multiple camera origin and corresponding eye image in the three-dimensional world coordinate system
Coordinate determines a plurality of straight line；Calculate the coordinate in the closest approach of a plurality of straight line；The closest approach be with it is described a plurality of straight
The shortest point of the linear distance of line；The coordinate for determining the closest approach of a plurality of straight line is human eye in the three-dimensional world coordinate
Coordinate in system.
By pinhole camera image-forming principle it is found that as (eye image), pin hole and object (human eye) are same under camera coordinates system
On one space line.
In embodiments of the present invention, coordinate and pin hole (camera of the eye image in three-dimensional world coordinate system are being determined
The origin of coordinate system) after the camera origin in three-dimensional world coordinate system, one can be determined using the two coordinates
Directly.
Specifically, setting human eye in the corresponding camera coordinates system of First video camera as point Pc, camera coordinates system origin is
Point Oc.Then point P of the human eye under three-dimensional world coordinate systemwAnd point O of the camera origin under three-dimensional world coordinate systemw, full respectively
Sufficient following equation:
Pw=(A*R) -1* (Pc- T)=(xw1, yw1, zw1)；
Ow=(A*R) -1* (Oc- T)=(xw2, yw2, zw2)。
Wherein, A is the internal reference matrix of video camera, and R is the spin matrix of video camera, and T is the translation vector of video camera, is passed through
Camera calibration can determine the corresponding A of each video camera, R, T.
By two point form it is found that the straight line between human eye and pin hole meets following equation:
F1:(x-xw1)/(x-xw2)=(y-yw1)/(y-yw2)=(z-zw1)/(z-zw2)。
When there is multiple cameras, it can determine what the corresponding straight line of each video camera met respectively using same method
Formula f2, f3... etc..
In embodiments of the present invention, the space line theoretically established under multiple cameras is intersection, because these are empty
Between straight line all pass through the point of position of human eye under three-dimensional world coordinate system.But due to calculating error, to may cause straight line non-intersecting,
Position of the position of these the shortest point of linear distance as human eye can be calculated using least square method.When these straight lines most
When short distance is 0, then these straight line intersections.
In the concrete realization, the equation relationship that above-mentioned every straight line can be met, generates linear equation group, by straight
Line equation group, which solve, can be obtained coordinate of the human eye in three-dimensional world coordinate system.
Step 103, the hot spot coordinate and human eye coordinates are based on, determine human eye sight in the three-dimensional world coordinate system
Direction vector.
It in embodiments of the present invention, can be using coordinate and human eye of the hot spot in three-dimensional world coordinate system in three-dimensional world
Coordinate in coordinate system determines direction vector of the human eye sight in world coordinate system.
Specifically, the seat of coordinate and human eye in world coordinate system by hot spot in three-dimensional world coordinate system can be passed through
Mark subtracts each other, and obtains direction vector of the human eye sight in three-dimensional world coordinate system.
In a preferred embodiment of an embodiment of the present invention, the method can also include the following steps:
Using direction vector of the human eye sight in the three-dimensional world coordinate system and the user images, people is generated
An eye line image.
Specifically, can be preset by the transformational relation between preset camera coordinates system and three-dimensional world coordinate system
Turn between transformational relation and preset pixel coordinate system and image coordinate system between image coordinate system and camera coordinates system
Relationship is changed, converts human eye sight direction in pixel coordinate system for direction vector of the human eye sight in three-dimensional world coordinate system
Vector, to generate human eye view using the user images of human eye sight direction vector and video camera acquisition in pixel coordinate system
Line image.
Further, it is also possible to by the transformational relation between camera coordinates system and three-dimensional world coordinate system, by hot spot in three-dimensional
Coordinate transformation in world coordinate system is coordinate of the hot spot in camera coordinates system, and by human eye in three-dimensional world coordinate system
Coordinate transformation be coordinate of the human eye in camera coordinates system；Using coordinate of the hot spot in camera coordinates system and human eye in camera
Coordinate in coordinate system determines direction vector of the human eye in camera coordinates system.In turn, according to preset image coordinate system and phase
The transformational relation between transformational relation and preset pixel coordinate system and image coordinate system between machine coordinate system, by human eye
Direction vector of the sight in camera coordinates system is converted into human eye sight direction vector in pixel coordinate system, thus using human eye
The user images of sight direction vector and video camera acquisition in pixel coordinate system, generate human eye sight image.
Specifically, setting coordinate of the human eye in the corresponding camera coordinates system of First video camera as Ec, sat in three-dimensional world
Coordinate in mark system is Ew；Coordinate of the hot spot in camera coordinates system is Qc, the coordinate in world coordinate system is Qw.Then, EcWith
EwAnd QcWith QwMeet following equation respectively:
Wherein, R is the spin matrix of video camera, and T is the translation vector of video camera.
Pass through above-mentioned EcWith EwAnd QcWith QwThe formula of satisfaction can determine seat of the hot spot in camera coordinates system respectively
Mark QcAnd coordinate of the human eye in camera coordinates system is Ec.By QcWith EcSubtract each other and can be obtained human eye sight under camera coordinates system
Direction vector.
If Fig. 4 is a kind of human eye sight image of acquisition of the present invention, in Fig. 4, in the human eye sight image of acquisition respectively
It is labelled with the human eye sight of different directions.The human eye sight direction of human eye sight detection system through the invention determines method,
The human eye sight in various directions can be acquired.
In embodiments of the present invention, by determining the hot spot of laser ranging system transmitting with the rotation center of steering engine holder
For the coordinate in the three-dimensional world coordinate system of coordinate origin, multiple user images of multiple video camera acquisitions are obtained, to determine people
Coordinate of the eye in three-dimensional world coordinate system, is based on hot spot coordinate and human eye coordinates, determines human eye sight in three-dimensional world coordinate
Direction vector in system.By setting steering engine holder, allow to collect the human eye sight image data of continuous orientation angle, and
And it also can be suitably used for various scenes, such as narrow automobile cab etc..
It should be noted that for simple description, therefore, it is stated as a series of action groups for embodiment of the method
It closes, but those skilled in the art should understand that, embodiment of that present invention are not limited by the describe sequence of actions, because according to
According to the embodiment of the present invention, some steps may be performed in other sequences or simultaneously.Secondly, those skilled in the art also should
Know, the embodiments described in the specification are all preferred embodiments, and the related movement not necessarily present invention is implemented
Necessary to example.
Referring to Fig. 5, a kind of structural block diagram of human eye sight detection system embodiment of the invention is shown, is applied to upper
Machine, the human eye sight detection system include: host computer, multiple video cameras, steering engine holder and the steering engine holder are arranged in
Laser ranging system, the host computer can specifically include following module:
Hot spot coordinate determining module 501, for determining the hot spot of the laser ranging system transmitting with the steering engine cloud
The rotation center of platform is the coordinate in the three-dimensional world coordinate system of coordinate origin；
Human eye coordinates determining module 502, for obtaining multiple user images of the multiple video camera acquisition, to determine people
Coordinate of the eye in the three-dimensional world coordinate system；
Direction vector determining module 503 determines human eye sight described for being based on the hot spot coordinate and human eye coordinates
Direction vector in three-dimensional world coordinate system.
In a preferred embodiment of an embodiment of the present invention, the steering engine holder is orthogonal two by direction of rotation
Steering engine composition；The hot spot coordinate determining module 501, comprising:
Rotate angle acquisition submodule, the rotation of steering engine holder when for obtaining the laser ranging system launch spot
Gyration；
Distance determine submodule, for determine the hot spot to the steering engine holder rotation center distance；
Hot spot coordinate determines submodule, for determining the hot spot with the rudder using the distance and rotation angle
The rotation center of machine head is the coordinate in the three-dimensional world coordinate system of coordinate origin.
In a preferred embodiment of an embodiment of the present invention, the human eye coordinates determining module 502, comprising:
First human eye coordinates determine submodule, for determining multiple eye images in the multiple user images in pixel
Coordinate in coordinate system；
Human eye coordinates convert submodule, for the coordinate by the multiple eye image in pixel coordinate system, convert institute
State the coordinate for multiple eye images in the three-dimensional world coordinate system；
Origin converts submodule, for being converted into the origin of the camera coordinates system of the multiple video camera
Multiple camera origins in three-dimensional world coordinate system；
Second human eye coordinates determine submodule, for respectively according to the multiple camera origin and corresponding human eye figure
As the coordinate in the three-dimensional world coordinate system, coordinate of the human eye in the three-dimensional world coordinate system is determined.
In a preferred embodiment of an embodiment of the present invention, the human eye coordinates convert submodule, comprising:
First coordinate transformation unit, for using the transformational relation between preset pixel coordinate system and image coordinate system,
By coordinate of the multiple eye image in pixel coordinate system, seat of the multiple eye image in image coordinate system is converted
Mark；
Second coordinate transformation unit, for using the transformational relation between preset image coordinate system and camera coordinates system,
By coordinate of the multiple eye image in image coordinate system, the multiple eye image is converted into camera coordinates system
Coordinate；
Third coordinate transformation unit, for being closed using the conversion between preset camera coordinates system and three-dimensional world coordinate system
System, by coordinate of multiple eye images in camera coordinates system, is converted into the multiple eye image and sits in the three-dimensional world
Coordinate in mark system.
In a preferred embodiment of an embodiment of the present invention, second human eye coordinates determine submodule, comprising:
Straight line determination unit, for respectively according to the multiple camera origin and corresponding eye image described three
The coordinate in world coordinate system is tieed up, determines a plurality of straight line；
Coordinate calculating unit, the coordinate in the closest approach for calculating a plurality of straight line；The closest approach for institute
State the shortest point of linear distance of a plurality of straight line；
Coordinate determination unit, for determining that the coordinate in closest approach of a plurality of straight line is human eye in the three-dimensional world
Coordinate in coordinate system.
In a preferred embodiment of an embodiment of the present invention, further includes:
Human eye sight image generation module, for the direction using the human eye sight in the three-dimensional world coordinate system
User images described in vector sum generate human eye sight image.
In a preferred embodiment of an embodiment of the present invention, the human eye sight detection system further includes steering engine control
Plate, the host computer and the steering engine control panel and the laser ranging system connect；The steering engine holder and the steering engine
Control panel connection；
The human eye sight direction of the human eye sight detection system determines that method is applied to the host computer, the steering engine cloud
The rotation angle of platform is controlled by the steering engine control panel.
In a preferred embodiment of an embodiment of the present invention, further includes:
Relation acquisition module, for obtaining the positional relationship information between the multiple video camera；
Transformational relation determining module, for determining described more using the positional relationship information between the multiple video camera
The transformational relation of the camera coordinates system of a video camera and the three-dimensional world coordinate system.
For device embodiment, since it is basically similar to the method embodiment, related so being described relatively simple
Place illustrates referring to the part of embodiment of the method.
The embodiment of the invention also provides a kind of devices, comprising:
Including processor, memory and it is stored in the computer that can be run on the memory and on the processor
Program, the computer program realize that the human eye sight direction of above-mentioned human eye sight detection system determines method when being executed by processor
Each process of embodiment, and identical technical effect can be reached, to avoid repeating, which is not described herein again.
The embodiment of the invention also provides a kind of computer readable storage medium, meter is stored on computer readable storage medium
Calculation machine program realizes the human eye sight direction determination side of above-mentioned human eye sight detection system when computer program is executed by processor
Each process of method embodiment, and identical technical effect can be reached, to avoid repeating, which is not described herein again.
The embodiments are all described in a progressive manner for each of this specification, the highlights of each of the examples are with
The difference of other embodiments, same and similar part may refer to each other between each embodiment.
It should be understood by those skilled in the art that, the embodiment of the embodiment of the present invention can provide as method, apparatus or calculate
Machine program product.Therefore, the embodiment of the present invention can be used complete hardware embodiment, complete software embodiment or combine software and
The form of the embodiment of hardware aspect.Moreover, the embodiment of the present invention can be used one or more wherein include computer can
With in the computer-usable storage medium (including but not limited to magnetic disk storage, CD-ROM, optical memory etc.) of program code
The form of the computer program product of implementation.
The embodiment of the present invention be referring to according to the method for the embodiment of the present invention, terminal device (system) and computer program
The flowchart and/or the block diagram of product describes.It should be understood that flowchart and/or the block diagram can be realized by computer program instructions
In each flow and/or block and flowchart and/or the block diagram in process and/or box combination.It can provide these
Computer program instructions are set to general purpose computer, special purpose computer, Embedded Processor or other programmable data processing terminals
Standby processor is to generate a machine, so that being held by the processor of computer or other programmable data processing terminal devices
Capable instruction generates for realizing in one or more flows of the flowchart and/or one or more blocks of the block diagram
The device of specified function.
These computer program instructions, which may also be stored in, is able to guide computer or other programmable data processing terminal devices
In computer-readable memory operate in a specific manner, so that instruction stored in the computer readable memory generates packet
The manufacture of command device is included, which realizes in one side of one or more flows of the flowchart and/or block diagram
The function of being specified in frame or multiple boxes.
These computer program instructions can also be loaded into computer or other programmable data processing terminal devices, so that
Series of operation steps are executed on computer or other programmable terminal equipments to generate computer implemented processing, thus
The instruction executed on computer or other programmable terminal equipments is provided for realizing in one or more flows of the flowchart
And/or in one or more blocks of the block diagram specify function the step of.
Although the preferred embodiment of the embodiment of the present invention has been described, once a person skilled in the art knows bases
This creative concept, then additional changes and modifications can be made to these embodiments.So the following claims are intended to be interpreted as
Including preferred embodiment and fall into all change and modification of range of embodiment of the invention.
Finally, it is to be noted that, herein, relational terms such as first and second and the like be used merely to by
One entity or operation are distinguished with another entity or operation, without necessarily requiring or implying these entities or operation
Between there are any actual relationship or orders.Moreover, the terms "include", "comprise" or its any other variant meaning
Covering non-exclusive inclusion, so that process, method, article or terminal device including a series of elements not only wrap
Those elements are included, but also including other elements that are not explicitly listed, or further includes for this process, method, article
Or the element that terminal device is intrinsic.In the absence of more restrictions, being wanted by what sentence "including a ..." limited
Element, it is not excluded that there is also other identical elements in process, method, article or the terminal device for including the element.
Method and one kind are determined to a kind of human eye sight direction of human eye sight detection system provided by the present invention above
Human eye sight detection system, is described in detail, and specific case used herein is to the principle of the present invention and embodiment
It is expounded, the above description of the embodiment is only used to help understand the method for the present invention and its core ideas；Meanwhile for
Those of ordinary skill in the art have change according to the thought of the present invention in specific embodiments and applications
Place, in conclusion the contents of this specification are not to be construed as limiting the invention.
Claims (11)
1. a kind of human eye sight direction of human eye sight detection system determines method, which is characterized in that the human eye sight detection
System includes: multiple video cameras, steering engine holder and the laser ranging system that the steering engine holder is arranged in, which comprises
Determine the hot spot of the laser ranging system transmitting using the rotation center of the steering engine holder as the three-dimensional of coordinate origin
Coordinate in world coordinate system；
Multiple user images of the multiple video camera acquisition are obtained, to determine seat of the human eye in the three-dimensional world coordinate system
Mark；
Based on the hot spot coordinate and human eye coordinates, direction vector of the human eye sight in the three-dimensional world coordinate system is determined.
2. the method according to claim 1, wherein the steering engine holder is orthogonal two by direction of rotation
Steering engine composition；The hot spot of the determination laser ranging system transmitting is using the rotation center of the steering engine holder as coordinate original
The coordinate of the three-dimensional world coordinate system of point, comprising:
Obtain the rotation angle of the steering engine holder when laser ranging system launch spot；
Determine the hot spot to the steering engine holder rotation center distance；
Using the distance and rotation angle, determine the hot spot using the rotation center of the steering engine holder as coordinate origin
Coordinate in three-dimensional world coordinate system.
3. the method according to claim 1, wherein the multiple users for obtaining the multiple video camera acquisition
Image, to determine coordinate of the human eye in the three-dimensional world coordinate system, comprising:
Determine coordinate of multiple eye images in pixel coordinate system in the multiple user images；
By coordinate of the multiple eye image in pixel coordinate system, the described conversion is multiple eye images in the three-dimensional generation
Coordinate in boundary's coordinate system；
By the origin of the camera coordinates system of the multiple video camera, it is converted into multiple cameras in three-dimensional world coordinate system
Origin；
Coordinate according to the multiple camera origin and corresponding eye image in the three-dimensional world coordinate system respectively,
Determine coordinate of the human eye in the three-dimensional world coordinate system.
4. according to the method described in claim 3, it is characterized in that, it is described by the multiple eye image in pixel coordinate system
Coordinate, be converted into coordinate of multiple eye images in the three-dimensional world coordinate system, comprising:
Using the transformational relation between preset pixel coordinate system and image coordinate system, the multiple eye image is sat in pixel
Coordinate in mark system, converts coordinate of the multiple eye image in image coordinate system；
Using the transformational relation between preset image coordinate system and camera coordinates system, the multiple eye image is sat in image
Coordinate in mark system, is converted into coordinate of the multiple eye image in camera coordinates system；
Using the transformational relation between preset camera coordinates system and three-dimensional world coordinate system, multiple eye images are sat in camera
Coordinate in mark system, is converted into coordinate of the multiple eye image in the three-dimensional world coordinate system.
5. according to the method described in claim 3, it is characterized in that, described respectively according to the multiple camera origin and phase
Coordinate of the eye image answered in the three-dimensional world coordinate system determines coordinate of the human eye in the three-dimensional world coordinate system
Further comprise:
Coordinate according to the multiple camera origin and corresponding eye image in the three-dimensional world coordinate system respectively,
Determine a plurality of straight line；
Calculate the coordinate in the closest approach of a plurality of straight line；The closest approach be with the linear distance of a plurality of straight line most
Short point；
The coordinate for determining the closest approach of a plurality of straight line is coordinate of the human eye in the three-dimensional world coordinate system.
6. according to the method described in claim 3, it is characterized by further comprising:
Using direction vector of the human eye sight in the three-dimensional world coordinate system and the user images, human eye view is generated
Line image.
7. the method according to claim 1, wherein the human eye sight detection system further includes host computer and rudder
Machine control panel, the host computer and the steering engine control panel and the laser ranging system connect；The steering engine holder and institute
State the connection of steering engine control panel；
The human eye sight direction of the human eye sight detection system determines that method is applied to the host computer, the steering engine holder
Rotation angle is controlled by the steering engine control panel.
8. the method according to claim 1, wherein further include:
Obtain the positional relationship information between the multiple video camera；
Using the positional relationship information between the multiple video camera, determine the camera coordinates system of the multiple video camera with it is described
The transformational relation of three-dimensional world coordinate system.
9. a kind of human eye sight detection system, which is characterized in that the human eye sight detection system includes: host computer, multiple takes the photograph
Camera, steering engine holder and the laser ranging system that the steering engine holder is arranged in, the host computer include:
Hot spot coordinate determining module, for determining the hot spot of the laser ranging system transmitting with the rotation of the steering engine holder
Center is the coordinate in the three-dimensional world coordinate system of coordinate origin；
Human eye coordinates determining module, for obtaining multiple user images of the multiple video camera acquisition, to determine human eye in institute
State the coordinate in three-dimensional world coordinate system；
Direction vector determining module determines human eye sight in the three-dimensional generation for being based on the hot spot coordinate and human eye coordinates
Direction vector in boundary's coordinate system.
10. a kind of device characterized by comprising processor, memory and be stored on the memory and can be described
The computer program run on processor is realized when the computer program is executed by the processor as in claim 1-8
The step of human eye sight direction of described in any item human eye sight detection systems determines method.
11. a kind of computer readable storage medium, which is characterized in that store computer journey on the computer readable storage medium
Sequence realizes such as human eye sight detection system described in any item of the claim 1 to 8 when the computer program is executed by processor
The step of human eye sight direction of system determines method.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910741288.0A CN110458104B (en) | 2019-08-12 | 2019-08-12 | Human eye sight direction determining method and system of human eye sight detection system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910741288.0A CN110458104B (en) | 2019-08-12 | 2019-08-12 | Human eye sight direction determining method and system of human eye sight detection system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110458104A true CN110458104A (en) | 2019-11-15 |
| CN110458104B CN110458104B (en) | 2021-12-07 |
Family
ID=68486028
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910741288.0A Active CN110458104B (en) | 2019-08-12 | 2019-08-12 | Human eye sight direction determining method and system of human eye sight detection system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110458104B (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111723716A (en) * | 2020-06-11 | 2020-09-29 | 深圳地平线机器人科技有限公司 | Method, apparatus, system, medium and electronic device for determining the orientation of a target object |
| CN113701710A (en) * | 2021-08-31 | 2021-11-26 | 高新兴科技集团股份有限公司 | Laser spot positioning method, distance measuring method, medium and equipment applied to security monitoring |
| CN114449319A (en) * | 2020-11-04 | 2022-05-06 | 深圳Tcl新技术有限公司 | Video picture dynamic adjustment method and device, intelligent terminal and storage medium |
| CN114913155A (en) * | 2022-05-11 | 2022-08-16 | 北京宾理信息科技有限公司 | Method, apparatus, computer equipment, vehicle, and medium for determining the direction of sight of a vehicle user |
| CN115588052A (en) * | 2022-10-24 | 2023-01-10 | 浙江极氪智能科技有限公司 | Sight direction data acquisition method, device, equipment and storage medium |
| CN117928383A (en) * | 2024-03-09 | 2024-04-26 | 广州泰宣科技有限公司 | Image pickup measurement method and system |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP4829141B2 (en) * | 2007-02-09 | 2011-12-07 | 株式会社東芝 | Gaze detection apparatus and method |
| CN102749991A (en) * | 2012-04-12 | 2012-10-24 | 广东百泰科技有限公司 | Non-contact free space eye-gaze tracking method suitable for man-machine interaction |
| WO2016142489A1 (en) * | 2015-03-11 | 2016-09-15 | SensoMotoric Instruments Gesellschaft für innovative Sensorik mbH | Eye tracking using a depth sensor |
| CN106662917A (en) * | 2014-04-11 | 2017-05-10 | 眼球控制技术有限公司 | Systems and methods of eye tracking calibration |
| CN206161864U (en) * | 2016-10-27 | 2017-05-10 | 武汉理工大学 | 3D laser scanner device |
| CN107562184A (en) * | 2016-07-01 | 2018-01-09 | Fove股份有限公司 | Image display system, image display method, image show program |
| CN108229284A (en) * | 2017-05-26 | 2018-06-29 | 北京市商汤科技开发有限公司 | Eye-controlling focus and training method and device, system, electronic equipment and storage medium |
| CN108700934A (en) * | 2015-09-24 | 2018-10-23 | 托比股份公司 | Wearables capable of eye tracking |
| CN108985172A (en) * | 2018-06-15 | 2018-12-11 | 北京七鑫易维信息技术有限公司 | A kind of Eye-controlling focus method, apparatus, equipment and storage medium based on structure light |
| JP6479272B1 (en) * | 2018-01-05 | 2019-03-06 | 三菱電機株式会社 | Gaze direction calibration apparatus, gaze direction calibration method, and gaze direction calibration program |
-
2019
- 2019-08-12 CN CN201910741288.0A patent/CN110458104B/en active Active
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP4829141B2 (en) * | 2007-02-09 | 2011-12-07 | 株式会社東芝 | Gaze detection apparatus and method |
| CN102749991A (en) * | 2012-04-12 | 2012-10-24 | 广东百泰科技有限公司 | Non-contact free space eye-gaze tracking method suitable for man-machine interaction |
| CN106662917A (en) * | 2014-04-11 | 2017-05-10 | 眼球控制技术有限公司 | Systems and methods of eye tracking calibration |
| WO2016142489A1 (en) * | 2015-03-11 | 2016-09-15 | SensoMotoric Instruments Gesellschaft für innovative Sensorik mbH | Eye tracking using a depth sensor |
| CN108700934A (en) * | 2015-09-24 | 2018-10-23 | 托比股份公司 | Wearables capable of eye tracking |
| CN107562184A (en) * | 2016-07-01 | 2018-01-09 | Fove股份有限公司 | Image display system, image display method, image show program |
| CN206161864U (en) * | 2016-10-27 | 2017-05-10 | 武汉理工大学 | 3D laser scanner device |
| CN108229284A (en) * | 2017-05-26 | 2018-06-29 | 北京市商汤科技开发有限公司 | Eye-controlling focus and training method and device, system, electronic equipment and storage medium |
| JP6479272B1 (en) * | 2018-01-05 | 2019-03-06 | 三菱電機株式会社 | Gaze direction calibration apparatus, gaze direction calibration method, and gaze direction calibration program |
| CN108985172A (en) * | 2018-06-15 | 2018-12-11 | 北京七鑫易维信息技术有限公司 | A kind of Eye-controlling focus method, apparatus, equipment and storage medium based on structure light |
Non-Patent Citations (3)
| Title |
|---|
| BEYMER DAVID ET AL.: "Eye gaze tracking using an active stereo head", 《2003 IEEE COMPUTER SOCIETY CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION》 * |
| STANISLAV PANEV ET AL.: "Human gaze tracking in 3D Space with an active multi-camera system", 《THE 7TH IEEE INTERNATIONAL CONFERENCE ON INTELLIGENT DATA ACQUISITION AND ADVANCED COMPUTING SYSTEMS: TECHNOLOGY AND APPLICATIONS》 * |
| 周泓 等: "面向驾驶员状态监测的人眼凝视方向判别", 《电路与系统学报》 * |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111723716A (en) * | 2020-06-11 | 2020-09-29 | 深圳地平线机器人科技有限公司 | Method, apparatus, system, medium and electronic device for determining the orientation of a target object |
| CN111723716B (en) * | 2020-06-11 | 2024-03-08 | 深圳地平线机器人科技有限公司 | Methods, devices, systems, media and electronic equipment for determining the orientation of target objects |
| CN114449319A (en) * | 2020-11-04 | 2022-05-06 | 深圳Tcl新技术有限公司 | Video picture dynamic adjustment method and device, intelligent terminal and storage medium |
| CN114449319B (en) * | 2020-11-04 | 2024-06-04 | 深圳Tcl新技术有限公司 | Video picture dynamic adjustment method and device, intelligent terminal and storage medium |
| CN113701710A (en) * | 2021-08-31 | 2021-11-26 | 高新兴科技集团股份有限公司 | Laser spot positioning method, distance measuring method, medium and equipment applied to security monitoring |
| CN113701710B (en) * | 2021-08-31 | 2024-05-17 | 高新兴科技集团股份有限公司 | Laser spot positioning method, ranging method, medium and equipment applied to security monitoring |
| CN114913155A (en) * | 2022-05-11 | 2022-08-16 | 北京宾理信息科技有限公司 | Method, apparatus, computer equipment, vehicle, and medium for determining the direction of sight of a vehicle user |
| CN115588052A (en) * | 2022-10-24 | 2023-01-10 | 浙江极氪智能科技有限公司 | Sight direction data acquisition method, device, equipment and storage medium |
| CN117928383A (en) * | 2024-03-09 | 2024-04-26 | 广州泰宣科技有限公司 | Image pickup measurement method and system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110458104B (en) | 2021-12-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110458104A (en) | The human eye sight direction of human eye sight detection system determines method and system | |
| US10275898B1 (en) | Wedge-based light-field video capture | |
| CN110191326A (en) | A kind of optical projection system resolution extension method, apparatus and optical projection system | |
| KR20160034847A (en) | System and method for calibrating a display system using a short throw camera | |
| US10432927B2 (en) | 3D test chart, adjusting arrangement, forming method and adjusting method thereof | |
| US9881377B2 (en) | Apparatus and method for determining the distinct location of an image-recording camera | |
| US9648271B2 (en) | System for filming a video movie | |
| US12395751B2 (en) | Auxiliary focusing method, apparatus, and system for generating auxiliary focus image based on depth information | |
| JP6174968B2 (en) | Imaging simulation device | |
| CN116524022B (en) | Offset data calculation method, image fusion device and electronic equipment | |
| CN102300043A (en) | Method for adjusting meeting place camera of remote presentation meeting system | |
| US11943539B2 (en) | Systems and methods for capturing and generating panoramic three-dimensional models and images | |
| US20210012515A1 (en) | Apparatus and a method for obtaining a registration error map representing a level of sharpness of an image | |
| EP3882846B1 (en) | Method and device for collecting images of a scene for generating virtual reality data | |
| JP5279078B2 (en) | Image shooting / display method, image shooting / display device, and program | |
| TW201447468A (en) | Panorama photographing method | |
| CN107592464A (en) | Zoom follows the switching method and device of curve during a kind of zoom | |
| KR100780701B1 (en) | Automatic 3D image generating device and method | |
| CN113850920A (en) | Virtual-real fusion method, system, device and storage medium based on spatial positioning | |
| US12271993B2 (en) | Method for forming an image of an object, computer program product and image forming system for carrying out the method | |
| CN113191032B (en) | Visual angle simulation method, device, system and storage medium | |
| JP2011166719A (en) | Image evaluation method and image evaluation system | |
| Calore et al. | Accelerometer based horizon and keystone perspective correction | |
| KR101990252B1 (en) | Method for producing virtual reality image, portable device in which VR photographing program for performing the same is installed, and server supplying the VR photographing program to the portable device | |
| CN120044707A (en) | Optical machine assembly method, device, equipment and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |