CN107153369B - It is a kind of to position the localization method of object, system and wear display equipment - Google Patents
It is a kind of to position the localization method of object, system and wear display equipment Download PDFInfo
- Publication number
- CN107153369B CN107153369B CN201710370301.7A CN201710370301A CN107153369B CN 107153369 B CN107153369 B CN 107153369B CN 201710370301 A CN201710370301 A CN 201710370301A CN 107153369 B CN107153369 B CN 107153369B
- Authority
- CN
- China
- Prior art keywords
- illuminator
- display equipment
- dimensional coordinate
- handle
- imu
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
- G05B19/00—Program-control systems
- G05B19/02—Program-control systems electric
- G05B19/04—Program control other than numerical control, i.e. in sequence controllers or logic controllers
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
- G01C21/10—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration
- G01C21/12—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning
- G01C21/16—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning by integrating acceleration or speed, i.e. inertial navigation
- G01C21/165—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning by integrating acceleration or speed, i.e. inertial navigation combined with non-inertial navigation instruments
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
- G01C21/20—Instruments for performing navigational calculations
- G01C21/206—Instruments for performing navigational calculations specially adapted for indoor navigation
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
- G01C21/26—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 specially adapted for navigation in a road network
- G01C21/28—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 specially adapted for navigation in a road network with correlation of data from several navigational instruments
- G01C21/30—Map- or contour-matching
- G01C21/32—Structuring or formatting of map data
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/017—Head mounted
Landscapes
- Engineering & Computer Science (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Automation & Control Theory (AREA)
- Optics & Photonics (AREA)
- Length Measuring Devices By Optical Means (AREA)
Abstract
The invention discloses a kind of localization method for positioning object, system and wear display equipment.This method comprises: wearing the spatial image in space locating for display equipment by wearing the camera acquisition being arranged in display equipment；The first posture information that display equipment is worn in the first IMU acquisition by wearing display equipment obtains the six-freedom degree information for wearing display equipment according to spatial image and the first posture information；Wear the historical track of the illuminator on display equipment acquisition handle and the historical track of the 2nd IMU；Wear the posture information that display equipment receives the handle of the 2nd IMU acquisition；According to the location information of luminous point in historical track and spatial image, the three-dimensional coordinate of illuminator is determined；According to the posture information of three-dimensional coordinate and handle, the six-freedom degree information of handle is determined.As it can be seen that the positioning device in the present invention is setting on positioning object, do not need to preset external positioning device, using convenience, orientation range is wide, improves user experience.
Description
Technical field
The present invention relates to field of locating technology, in particular to a kind of localization method for positioning object, system and display is worn
Equipment.
Background technique
Location technology is widely used in the fields such as virtual reality, augmented reality, motion capture, is the important of human-computer interaction
Component part.Wherein, optical alignment the advantages that its high-precision, low cost, high refresh rate to become in indoor positioning field by most
Widely applied technology.
Optical alignment in the prior art carries out the positioning of object generally by the positioning device for being placed outside object.But
It is that the visual angle of the orientation range dependence positioning device of this scheme and recognizable distance, orientation range is limited, is especially wearing
In the positioning application for showing equipment and handle, once wearing display equipment or handle not in orientation range, it cannot achieve and wear
Show the accurate positionin of equipment or handle；And needing to preset external positioning device, application is not convenient, influences user experience.
Summary of the invention
Limited in view of the orientation range of optical alignment scheme in the prior art, application is not convenient, influences user experience
Problem proposes the localization method and system of a kind of positioning object of the invention, above-mentioned to solve or at least to be partially solved
Problem.
According to an aspect of the invention, there is provided a kind of localization method for positioning object, which comprises
The spatial image in space locating for display equipment is worn described in the camera acquisition being arranged in display equipment by wearing；
Pass through the first Inertial Measurement Unit (Inertial measurement unit, the abbreviation for wearing display equipment
IMU the first posture information that display equipment) is worn described in acquisition is obtained according to the spatial image and first posture information
Take the six-freedom degree information for wearing display equipment；
The historical track worn display equipment and obtain the historical track and the 2nd IMU of illuminator；Wherein, described to shine
Body and the 2nd IMU are arranged to be worn on the corresponding handle of display equipment with described；
The posture information worn display equipment and receive the handle of the 2nd IMU acquisition；
According to the location information of luminous point in the historical track and the spatial image, the three-dimensional of the illuminator is determined
Coordinate, wherein the luminous point is corresponding with the illuminator；
According to the posture information of the three-dimensional coordinate and the handle, the six-freedom degree information of the handle is determined.
According to another aspect of the present invention, it provides one kind and wears display equipment, the display equipment of wearing includes taking the photograph
As head, the first Inertial Measurement Unit IMU；It is described to wear display equipment further include:
Image acquisition units, for the space diagram by wearing space locating for display equipment described in camera acquisition
Picture；
Positioning unit, the first posture by wearing display equipment described in the first Inertial Measurement Unit IMU acquisition are believed
Breath wears the six-freedom degree information of display equipment according to the spatial image and first posture information described in acquisition；
Acquiring unit, for obtaining the historical track of illuminator and the historical track of the 2nd IMU；Wherein, the illuminator
It is arranged at the 2nd IMU and is worn on the corresponding handle of display equipment with described；
Receiving unit, the posture information of the handle for receiving the 2nd IMU acquisition；
Three-dimensional coordinate determination unit, for the position letter according to luminous point in the historical track and the spatial image
Breath, determines the three-dimensional coordinate of the illuminator, wherein the luminous point is corresponding with the illuminator；
The positioning unit is also used to the posture information according to the three-dimensional coordinate and the handle, determines the handle
Six-freedom degree information.
According to a further aspect of the invention, a kind of positioning system for positioning object is provided, the system comprises: as before
It is described to wear display equipment and wear the corresponding handle of display equipment with described；
Display equipment is worn described in the wired or wireless connection of handle.
In conclusion the present invention, which passes through the camera acquisition worn and be arranged in display equipment, wears space locating for display equipment
Spatial image；The first posture letter of display equipment is worn in the first Inertial Measurement Unit IMU acquisition by wearing display equipment
Breath obtains the six-freedom degree information for wearing display equipment according to spatial image and the first posture information, and realization is worn display and set
Standby positioning；Also, it wears display equipment and obtains the historical track of illuminator and the historical track of the 2nd IMU；Wherein, illuminator
It is arranged on handle corresponding with display equipment is worn with the 2nd IMU；Receive the posture information of the handle of the 2nd IMU acquisition；
According to the location information of luminous point in historical track and spatial image, the three-dimensional coordinate of illuminator is determined, wherein luminous point and hair
Body of light is corresponding；According to the posture information of three-dimensional coordinate and handle, determines the six-freedom degree information of handle, realize determining for handle
Position.As it can be seen that technical solution of the present invention be different from the prior art in external camera, be by wearing taking the photograph in display equipment
As head realize wear display equipment and with the positioning of wearing the corresponding handle of display equipment, camera is with wearing display equipment
It moves and moves, orientation range is wide.That is, the positioning device in technical solution of the present invention is setting in positioning object
On, it does not need to preset external positioning device, using convenience, orientation range is wide, and user experience can be improved.
Detailed description of the invention
Fig. 1 is a kind of flow diagram of localization method for positioning object provided by one embodiment of the present invention；
Fig. 2 is a kind of structural schematic diagram for wearing display equipment provided by one embodiment of the present invention；
Fig. 3 is a kind of structural schematic diagram of positioning system for positioning object provided by one embodiment of the present invention.
Specific embodiment
Mentality of designing of the invention is: the spatial image of the acquisition of the camera in display equipment, knot are worn by being arranged in
The first posture information for wearing display equipment for being arranged in and wearing the acquisition of the first Inertial Measurement Unit IMU in display equipment is closed,
Realize the positioning for wearing display equipment；And by determining the illuminator being arranged on handle corresponding with display equipment is worn
Historical track and the 2nd IMU acquisition handle posture information, realize the positioning of handle.To make the purpose of the present invention, skill
Art scheme and advantage are clearer, and below in conjunction with attached drawing, embodiment of the present invention is described further in detail.
Fig. 1 is a kind of flow diagram of localization method for positioning object provided by one embodiment of the present invention.Such as Fig. 1 institute
Show, this method comprises:
Step S110 wears the space in space locating for display equipment by wearing the camera acquisition being arranged in display equipment
Image.
In the present embodiment, the camera worn in display equipment, which is arranged in, can be wide-angle camera, so that the present embodiment
In be applicable in orientation range it is wider.Also, in order to be obtained by spatial image and shining in image in handle positions
The three-dimensional coordinate of the corresponding illuminator of point, the number for wearing the camera in display equipment is at least 2, when display is worn in acquisition
When spatial image locating for equipment, which acquires spatial image simultaneously, and each camera obtains a space
Image.Because in the present embodiment being at least two wide-angle camera, spatial image is also at least two.
Step S120, the first Inertial Measurement Unit IMU acquisition by wearing display equipment wear the first of display equipment
Posture information obtains the six-freedom degree information for wearing display equipment according to spatial image and the first posture information.
Here Inertial Measurement Unit is Inertialmeasurementunit, hereinafter referred to as IMU.
In the present embodiment, according to the spatial structural form in spatial image, using instant positioning and map structuring
(simultaneous localization and mapping, abbreviation slam) method is calculated, can obtain and wear display
The one-movement-freedom-degree (i.e. three-dimensional coordinate) of equipment；Wear display equipment first according to the first Inertial Measurement Unit IMU acquisition
Posture information (for example, quaternary number information), can obtain wear display equipment rotational freedom (i.e. yaw angle, pitch angle and
Roll angle)；The six-freedom degree letter for wearing display equipment is assured that using above-mentioned one-movement-freedom-degree and rotational freedom
Breath realizes the positioning for wearing display equipment.
Step S130 wears display equipment and obtains the historical track of illuminator and the historical track of the 2nd IMU；Wherein, it sends out
Body of light and the 2nd IMU are arranged on handle corresponding with display equipment is worn.
In the present embodiment, the positioning of itself will not only be realized by wearing display equipment, also realize and wear display equipment
The positioning of corresponding handle.For the positioning for realizing handle, need to obtain the historical track and the 2nd IMU of the illuminator on handle
Historical track.
In the present embodiment, an illuminator is set on handle, multiple illuminators also can be set.Multiple illuminators are set
Purpose be, when carrying out the positioning of handle, to prevent from occurring in spatial image other interference luminous points, alternatively, in no handle
Historical track in the case where, can use the luminous point corresponding with multiple illuminators in spatial image the regularity of distribution carry out
The positioning of handle.
Step S140 wears the posture information that display equipment receives the handle of the 2nd IMU acquisition.
For example, the quaternary number information of the handle of the 2nd IMU acquisition.
Step S150 determines that the three-dimensional of illuminator is sat according to the location information of luminous point in historical track and spatial image
Mark, wherein luminous point is corresponding with illuminator, so, according to the location information of luminous point in image, can determine and luminous point
The three-dimensional coordinate of corresponding illuminator.
For example, in the above description, it is indicated that the number of the camera in the present embodiment is at least two, then pair obtained
Answering the image of the same illuminator just has at least two, so that it may binocular imaging principle is utilized, according to corresponding at least two image
The location information of the luminous point of the same illuminator obtains the three-dimensional coordinate of the illuminator.Here binocular imaging principle, is base
In principle of parallax and two images of the binocular camera from different position acquisition objects are utilized, by calculating corresponding points in image
Between position deviation, the method to obtain object dimensional aggregate information.The advantages of this method be it is high-efficient, precision is high, structure is simple
The advantages that single, at low cost.
Step S160 determines the six-freedom degree information of handle according to the posture information of three-dimensional coordinate and handle.
Here three-dimensional coordinate can determine the one-movement-freedom-degree information of handle；Posture information (quaternary number) can determine hand
The rotational freedom information of handle is the six-freedom degree information that can determine handle according to one-movement-freedom-degree and rotational freedom.
As it can be seen that technical solution of the present invention be different from the prior art in external camera, be by wearing display equipment
On camera realize wear display equipment and with the positioning of wearing the corresponding handle of display equipment, camera is with wearing display
The movement of equipment and move, orientation range is wide.That is, the positioning device in technical solution of the present invention is that setting is positioning
It on object, does not need to preset external positioning device, using convenience, orientation range is wide, and user experience can be improved.
In one embodiment of the invention, it is above-mentioned wear display equipment be virtual reality (Virtual Reality,
Abbreviation VR) equipment, above-mentioned handle is handle corresponding with VR equipment.
In the technical solution of we, when determining the three-dimensional coordinate of the illuminator on handle, different situations might have,
Such as, the illuminator on handle is not blocked, and has the historical track of illuminator；Illuminator on handle is not blocked, but
It is the historical track of not illuminator；Illuminator on handle is blocked, and has the historical track of illuminator.So, in determination
When the three-dimensional coordinate of illuminator, the method for use should be different.It is illustrated below for above situation.
(1) illuminator on handle is not blocked, and has the historical track of illuminator.
In one embodiment of the invention, the position according to luminous point in historical track and spatial image of step S150
Information determines that the three-dimensional coordinate of illuminator includes:
1. obtaining the linear velocity and linear acceleration of illuminator according to the historical track of illuminator；2. adding according to linear velocity and line
Speed determines the three-dimensional coordinate of illuminator, according to historical track estimation when the three-dimensional coordinate of illuminator here, is one and estimates
Calculation value；3. the two-dimensional coordinate of illuminator in spatial image is obtained by the three-dimensional coordinate back projection to spatial image of illuminator, because
Three-dimensional coordinate for illuminator is an estimated value, and the two-dimensional coordinate of the illuminator obtained here is also estimated value；4. calculating empty
Between luminous point in image two-dimensional coordinate；Wherein, the luminous point in spatial image is corresponding with illuminator, that is to say, that hair
Luminous point is corresponding to be the actual position of illuminator, so the two-dimensional coordinate of the luminous point calculated in this step should be illuminator
True two-dimensional coordinate, still, the corresponding relationship between luminous point and illuminator has not determined at this time, i.e., the luminous point tool in image
Which illuminator body, which corresponds to, has not determined；5. the two-dimensional coordinate value of the two-dimensional coordinate value of illuminator and luminous point is compared
Compared with, that is, the two-dimensional coordinate estimated value of the two-dimensional coordinate of the luminous point of calculating and the illuminator of calculating is compared；It will hair
The difference value of the two-dimensional coordinate value of the two-dimensional coordinate value and illuminator of luminous point is determined as and sends out in the luminous point of preset threshold range
Body of light has the luminous point of corresponding relationship；6. determining hair according to the two-dimensional coordinate of the luminous point with illuminator with corresponding relationship
The three-dimensional coordinate of body of light.
For example, in the above description, it is indicated that the number of the camera in the present embodiment is at least two, then according to it is same
One illuminator has the two-dimensional coordinate of the luminous point of corresponding relationship, so that it may utilize binocular imaging principle, determine illuminator
Three-dimensional coordinate, determining three-dimensional coordinate are exactly the true value of the three-dimensional coordinate of illuminator.
In above description, an illuminator can be set on handle, multiple illuminators also can be set.The above method is outstanding
The case where it is suitable for multiple illuminators.When there are multiple illuminators, and wearing display equipment includes at least two camera
When, for an illuminator on handle, the linear velocity of the illuminator is obtained according to the historical track of the illuminator and line accelerates
Degree；The three-dimensional coordinate of the illuminator is determined according to linear velocity and linear acceleration；The three-dimensional coordinate of the illuminator is distinguished into back projection
Onto each spatial image, the two-dimensional coordinate of the illuminator in each spatial image is obtained respectively；Calculate separately each space diagram
The two-dimensional coordinate of each luminous point as in；For a spatial image, by the two-dimensional coordinate of the illuminator in spatial image and
The two-dimensional coordinate of each luminous point compares, by the two-dimensional coordinate difference value of the two-dimensional coordinate of luminous point and the illuminator in default threshold
It is worth the luminous point of range, is determined as the luminous point that there is corresponding relationship with the illuminator, and so on, determine each spatial image
In with the illuminator have corresponding relationship luminous point；Using the two-dimensional coordinate of at least two luminous point of the corresponding illuminator,
The determining three-dimensional coordinate with the illuminator.The three-dimensional coordinate of each illuminator on handle is assured that by this method.
Before distinguishing the three-dimensional coordinate of illuminator in back projection to each spatial image, need to guarantee in spatial image
Luminous point occur at least two spatial image, exclude simultaneously do not appear at least two spatial image shine
Point.For example, there are 3 cameras, 3 spatial images are acquired, are spatial image 1, spatial image 2, spatial image 3 respectively, it is right
Luminous point 1 in spatial image 1 does not occur in spatial image 2 and spatial image 3, then just excluding the luminous point 1.
After determining the three-dimensional coordinate of each illuminator, using RPnP algorithm or other related algorithms in the prior art,
In conjunction with the posture information of the 2nd IMU handle acquired, determines the six-freedom degree information of handle, realize the positioning of handle.
(2) illuminator on handle is not blocked, but the historical track or historical track without illuminator are imperfect.This
Embodiment is suitable for the case where being provided with multiple illuminators on handle.
In one embodiment of the invention, the position according to luminous point in historical track and spatial image of step S150
Information determines that the three-dimensional coordinate of illuminator includes:
1. calculating the two-dimensional coordinate of each luminous point in spatial image；2. according to the distribution of the luminous point in spatial image
Rule determines the corresponding relationship of the illuminator on the luminous point and handle in spatial image；3. using shining on corresponding handle
The two-dimensional coordinate of the luminous point of body determines the three-dimensional coordinate of the illuminator on corresponding handle.Because this method is existing
There is relevant programme in technology, is no longer described in detail herein.
For example, there are four illuminators for setting on handle, it is arranged as tetrahedroid, the regularity of distribution in spatial image is four
Side shape, then, in spatial image, there are four luminous points for being arranged in similar quadrangle just to correspond to handle, and can be according to four
The relative position information between luminous point in the shape of side, determines the corresponding relationship between the illuminator on luminous point and handle.
The present embodiment, the case where suitable for being provided with multiple illuminators on handle, and the accuracy in order to guarantee positioning, this
One or more illuminators in embodiment cannot be in same plane.
(3) illuminator on handle is blocked, and has the historical track of illuminator.
When handle is because of noise jamming, or when the reasons such as blocking and can not be positioned, the temporary interruption of positioning will cause, in order to
Prevent the appearance of above situation from influencing the usage experience of user, in one embodiment of the invention, the foundation of step S150
The location information of luminous point in historical track and spatial image determines that the three-dimensional coordinate of illuminator includes:
1. according to the historical track of illuminator determine the three-dimensional position at the first moment of illuminator, the 2nd IMU first when
The unit direction vector at the speed at quarter and the 2nd first moment of IMU, the first moment here may be considered in illuminator quilt
At the time of last before blocking；2. according to the 2nd IMU acquisition posture information obtain the 2nd second moment of IMU unit direction to
Amount；3. calculating the direction change of the 2nd IMU according to the unit direction vector of the unit direction vector at the second moment and the first moment
Unit vector；4. calculating the three-dimensional coordinate at the second moment of illuminator according to three-dimensional coordinate formula.
Specifically, the first moment in the present embodiment may be considered the previous moment at the second moment.Above-mentioned three-dimensional seat
Mark formula are as follows: Pt=Pt-1+Vt × l；Wherein, Pt is the three-dimensional coordinate at the second moment of illuminator；Pt-1 is the first of illuminator
The three-dimensional coordinate at moment；Vt is the direction change unit vector of the 2nd IMU；L is the speed at the first moment of the 2nd IMU.
In the present embodiment, when handle can not be positioned, the history three-dimensional track of each illuminator in handle can use
And the 2nd the method that combines of IMU realize the positioning of handle.
Because the three-dimensional coordinate at the second moment of each illuminator be it is three-dimensional, then the l in above-mentioned formula specifically should be
The component velocity on three-dimensional at the first moment of two IMU.
In order to keep the positioning for wearing display equipment and handle more accurate, in one embodiment of the invention, in step
After the six-freedom degree information of display equipment is worn in acquisition in S120, method shown in FIG. 1 further comprises: in conjunction with first
The six-freedom degree for wearing display equipment is carried out smooth interpolation filtering by the first posture information of Inertial Measurement Unit IMU acquisition,
The final six-freedom degree of equipment is shown using the filtered six-freedom degree for wearing display equipment as wearing.
After the six-freedom degree information of the determination handle of step S160, method shown in FIG. 1 further comprises: in conjunction with
The six-freedom degree of handle is carried out smooth interpolation filtering, by filtered handle by the posture information of the handle of the 2nd IMU acquisition
The six-freedom degree six-freedom degree final as handle.
In the present embodiment, the six-freedom degree information of the six-freedom degree information for wearing display equipment and handle is carried out
Smooth interpolation filtering can use kalman filter method.Wherein, Kalman filtering is a kind of utilization linear system state equation,
Data are observed by system input and output, the algorithm of optimal estimation is carried out to system mode, due to including system in observation data
In noise and interference influence, so optimal estimation is also considered as filtering.In the present embodiment, Kalman filtering can
New measurement error is obtained to go to look into merge past measurement estimation, by after the six-freedom degree filtering for positioning object, can be obtained
More accurate data are obtained, refreshing frequency is effectively improved.
In addition, in the present embodiment, the posture information acquired in conjunction with IMU carries out smooth interpolation filtering, also for into one
Step obtains the six-freedom degree for more accurately positioning object.
It should also be noted that, when carrying out smooth interpolation filtering, what is utilized is not only the posture information of IMU acquisition, also
Need the acceleration information using IMU acquisition.Because the data of IMU acquisition not only include above-mentioned posture information, while can also
Acceleration information is exported, so being as long as extracting acceleration information from the data that IMU is acquired when carrying out smooth interpolation filtering
It can.
In the present invention, in the history for determining that the three-dimensional coordinate of the illuminator on handle needs to utilize illuminator and the 2nd IMU
Track.In one embodiment of the invention, after the six-freedom degree information of the determination handle in step S160, Fig. 1 institute
The method shown further include: save the six-freedom degree information of handle, six-freedom degree information is to record illuminator and the 2nd IMU
Historical track.
Fig. 2 is a kind of structural schematic diagram for wearing display equipment provided by one embodiment of the present invention.As shown in Fig. 2, should
Wearing display equipment 200 includes camera 210, the first Inertial Measurement Unit IMU220；Wear display equipment 200 further include:
Image acquisition units 230, for obtaining the spatial image for wearing space locating for display equipment by camera；
Positioning unit 240 wears the first posture information of display equipment by the first Inertial Measurement Unit IMU acquisition, according to
According to spatial image and the first posture information, the six-freedom degree information for wearing display equipment is obtained；
Acquiring unit 250, for obtaining the historical track of illuminator and the historical track of the 2nd IMU；Wherein, illuminator and
2nd IMU is arranged on handle corresponding with display equipment is worn；
Receiving unit 260, the posture information of the handle for receiving the 2nd IMU acquisition；
Three-dimensional coordinate determination unit 270 is determined for the location information according to luminous point in historical track and spatial image
The three-dimensional coordinate of illuminator, wherein luminous point is corresponding with illuminator；
Positioning unit 240 is also used to the posture information according to three-dimensional coordinate and handle, determines the six-freedom degree letter of handle
Breath.
In one embodiment of the invention, three-dimensional coordinate determination unit 270, specifically for the history rail according to illuminator
The linear velocity and linear acceleration of mark acquisition illuminator；The three-dimensional coordinate of illuminator is determined according to linear velocity and linear acceleration；It will hair
In the three-dimensional coordinate back projection to spatial image of body of light, the two-dimensional coordinate of illuminator in spatial image is obtained, wherein spatial image
In luminous point it is corresponding with illuminator；Calculate the two-dimensional coordinate of the luminous point in spatial image；Wherein, the hair in spatial image
Luminous point is corresponding with illuminator；The two-dimensional coordinate value of the two-dimensional coordinate value of illuminator and luminous point is compared, by luminous point
Two-dimensional coordinate value and illuminator two-dimensional coordinate value difference value in the luminous point of preset threshold range, be determined as and illuminator
Luminous point with corresponding relationship；According to the two-dimensional coordinate of the luminous point with illuminator with corresponding relationship, illuminator is determined
Three-dimensional coordinate.
In one embodiment of the invention, three-dimensional coordinate determination unit 270 is specifically used for being determined according to historical track and send out
Three-dimensional position, the speed at the first moment of the 2nd IMU and the unit side at the 2nd first moment of IMU at the first moment of body of light
To vector；The unit direction vector at the 2nd second moment of IMU is obtained according to the posture information of the 2nd IMU acquisition；When according to second
The unit direction vector at quarter and the unit direction vector at the first moment calculate the direction change unit vector of the 2nd IMU；According to three
Tie up the three-dimensional coordinate that coordinate formula calculates the second moment of illuminator.
In one embodiment of the invention, wearing display equipment 200 further comprises: filter unit, for combining the
The six-freedom degree for wearing display equipment is carried out smooth interpolation filter by the first posture information of one Inertial Measurement Unit IMU acquisition
Wave shows the final six-freedom degree of equipment using the filtered six-freedom degree for wearing display equipment as wearing；And knot
The six-freedom degree of handle is carried out smooth interpolation filtering, by filtered hand by the posture information for closing the handle of the 2nd IMU acquisition
The six-freedom degree of the handle six-freedom degree final as handle.
Historical track recording unit, for saving the six-freedom degree information of handle, six-freedom degree information record shines
The historical track of body and the 2nd IMU.
Fig. 3 is a kind of structural schematic diagram of positioning system for positioning object provided by one embodiment of the present invention.Such as Fig. 3 institute
Show, the positioning system 300 of the positioning object includes: that as shown in Figure 2 wearing shows equipment 200 and show equipment 200 with wearing
Corresponding handle 320.
Display equipment 200 is worn in the wired or wireless connection of handle 320, and to guarantee to wear display, equipment 200 is available arrives
Corresponding data.
It should be noted that shown in each embodiment shown in Fig. 2 for wearing display equipment and system shown in Fig. 3 and Fig. 1
Each embodiment of method correspond to identical, be described in detail above, details are not described herein.
In conclusion the present invention, which passes through the camera acquisition worn and be arranged in display equipment, wears space locating for display equipment
Spatial image；The first posture letter of display equipment is worn in the first Inertial Measurement Unit IMU acquisition by wearing display equipment
Breath obtains the six-freedom degree information for wearing display equipment according to spatial image and the first posture information, and realization is worn display and set
Standby positioning；Also, it wears display equipment and obtains the historical track of illuminator and the historical track of the 2nd IMU；Wherein, illuminator
It is arranged on handle corresponding with display equipment is worn with the 2nd IMU；Receive the posture information of the handle of the 2nd IMU acquisition；
According to the location information of luminous point in historical track and spatial image, the three-dimensional coordinate of illuminator is determined, wherein luminous point and hair
Body of light is corresponding；According to the posture information of three-dimensional coordinate and handle, determines the six-freedom degree information of handle, realize determining for handle
Position.As it can be seen that technical solution of the present invention be different from the prior art in external camera, be by wearing taking the photograph in display equipment
As head realize wear display equipment and with the positioning of wearing the corresponding handle of display equipment, camera is with wearing display equipment
It moves and moves, orientation range is wide.That is, the positioning device in technical solution of the present invention is setting in positioning object
On, it does not need to preset external positioning device, using convenience, orientation range is wide, and user experience can be improved.
The above description is merely a specific embodiment, under above-mentioned introduction of the invention, those skilled in the art
Other improvement or deformation can be carried out on the basis of the above embodiments.It will be understood by those skilled in the art that above-mentioned tool
Body description only preferably explains that the purpose of the present invention, protection scope of the present invention should be subject to the protection scope in claims.
Claims (8)
1. a kind of localization method for positioning object, which is characterized in that the described method includes:
The spatial image in space locating for display equipment is worn described in the camera acquisition being arranged in display equipment by wearing；
By the first posture letter for wearing display equipment described in the first Inertial Measurement Unit IMU acquisition for wearing display equipment
Breath wears the six-freedom degree information of display equipment according to the spatial image and first posture information described in acquisition；
The historical track worn display equipment and obtain the historical track and the 2nd IMU of illuminator；Wherein, the illuminator and
2nd IMU is arranged to be worn on the corresponding handle of display equipment with described；
The posture information worn display equipment and receive the handle of the 2nd IMU acquisition；
According to the location information of luminous point in the historical track and the spatial image, determine that the three-dimensional of the illuminator is sat
Mark, wherein the luminous point is corresponding with the illuminator；
According to the posture information of the three-dimensional coordinate and the handle, the six-freedom degree information of the handle is determined；
The location information according to luminous point in the historical track and the spatial image, determines the three-dimensional of the illuminator
Coordinate includes:
The linear velocity and linear acceleration of the illuminator are obtained according to the historical track of the illuminator；
The three-dimensional coordinate of the illuminator is determined according to the linear velocity and the linear acceleration；
By in the three-dimensional coordinate back projection to the spatial image of the illuminator, illuminator described in the spatial image is obtained
Two-dimensional coordinate；
Calculate the two-dimensional coordinate of the luminous point in the spatial image；Wherein, the luminous point in the spatial image and the hair
Body of light is corresponding；The two-dimensional coordinate value of the two-dimensional coordinate value of the illuminator and the luminous point is compared, by the hair
The difference value of the two-dimensional coordinate value of the two-dimensional coordinate value of luminous point and the illuminator preset threshold range the luminous point, really
It is set to the luminous point that there is corresponding relationship with the illuminator；
According to the two-dimensional coordinate of the luminous point with the illuminator with corresponding relationship, determine that the three-dimensional of the illuminator is sat
Mark.
2. the method as described in claim 1, which is characterized in that described according to being sent out in the historical track and the spatial image
The location information of luminous point determines that the three-dimensional coordinate of the illuminator includes:
The three-dimensional position at the first moment of the illuminator, the first moment of the 2nd IMU are determined according to the historical track
Speed and the unit direction vector at the first moment of the 2nd IMU；
The unit direction vector at the second moment of the 2nd IMU is obtained according to the posture information that the 2nd IMU is acquired；
According to the unit direction vector of the unit direction vector at second moment and first moment, described second is calculated
The direction change unit vector of IMU；
The three-dimensional coordinate at second moment of illuminator is calculated according to three-dimensional coordinate formula.
3. the method as described in claim 1, which is characterized in that
After the six-freedom degree information for wearing display equipment described in the acquisition, the method further includes:
In conjunction with the first posture information that the first Inertial Measurement Unit IMU is acquired, certainly by six for wearing display equipment
Smooth interpolation filtering is carried out by degree, the filtered six-freedom degree for wearing display equipment is worn into display as described in and is set
Standby final six-freedom degree；
After the six-freedom degree information of the determination handle, the method further includes:
In conjunction with the posture information of the 2nd IMU handle acquired, the six-freedom degree of the handle is smoothly inserted
Value filtering, using the six-freedom degree that the six-freedom degree of the filtered handle is final as the handle.
4. the method as described in claim 1, which is characterized in that the determination handle six-freedom degree information it
Afterwards, the method also includes:
The six-freedom degree information of the handle is saved, the six-freedom degree information is to record the illuminator and described
The historical track of two IMU.
5. one kind wears display equipment, which is characterized in that the display equipment of wearing includes camera, the first Inertial Measurement Unit
IMU；It is described to wear display equipment further include:
Image acquisition units, for the spatial image by wearing space locating for display equipment described in camera acquisition；
Positioning unit, by wearing the first posture information of display equipment described in the first Inertial Measurement Unit IMU acquisition, according to
According to the spatial image and first posture information, the six-freedom degree information of display equipment is worn described in acquisition；
Acquiring unit, for obtaining the historical track of illuminator and the historical track of the 2nd IMU；Wherein, the illuminator and institute
It states the 2nd IMU and is arranged at and worn on the corresponding handle of display equipment with described；
Receiving unit, the posture information of the handle for receiving the 2nd IMU acquisition；
Three-dimensional coordinate determination unit, for the location information according to luminous point in the historical track and the spatial image, really
The three-dimensional coordinate of the fixed illuminator, wherein the luminous point is corresponding with the illuminator；
The positioning unit is also used to the posture information according to the three-dimensional coordinate and the handle, determines the six of the handle
A freedom degree information；
The three-dimensional coordinate determination unit, specifically for obtaining the linear speed of the illuminator according to the historical track of the illuminator
Degree and linear acceleration；The three-dimensional coordinate of the illuminator is determined according to the linear velocity and the linear acceleration；It shines described
In the three-dimensional coordinate back projection to the spatial image of body, the two-dimensional coordinate of illuminator described in the spatial image is obtained,
In, the luminous point in the spatial image is corresponding with the illuminator；Calculate the two dimension of the luminous point in the spatial image
Coordinate；Wherein, the luminous point in the spatial image is corresponding with the illuminator；By the two-dimensional coordinate value of the illuminator and
The two-dimensional coordinate value of the luminous point is compared, by the two-dimensional coordinate of the two-dimensional coordinate value of the luminous point and the illuminator
The difference value of value is determined as the luminous point for having corresponding relationship with the illuminator in the luminous point of preset threshold range；
According to the two-dimensional coordinate of the luminous point with the illuminator with corresponding relationship, the three-dimensional coordinate of the illuminator is determined.
6. equipment as claimed in claim 5, which is characterized in that
The three-dimensional coordinate determination unit, three of the first moment specifically for determining the illuminator according to the historical track
Tie up position, the 2nd IMU the first moment speed and the unit direction vector at the first moment of the 2nd IMU；According to
The posture information of the 2nd IMU acquisition obtains the unit direction vector at the second moment of the 2nd IMU；When according to described second
The unit direction vector of the unit direction vector at quarter and first moment, calculate the direction change unit of the 2nd IMU to
Amount；The three-dimensional coordinate at second moment of illuminator is calculated according to three-dimensional coordinate formula.
7. equipment as claimed in claim 5, which is characterized in that the equipment further comprises:
Filter unit, the first posture information for acquiring in conjunction with the first Inertial Measurement Unit IMU, wears display for described
The six-freedom degree of equipment carries out smooth interpolation filtering, using the filtered six-freedom degree for wearing display equipment as institute
It states and wears the final six-freedom degree of display equipment；And the posture information in conjunction with the 2nd IMU handle acquired,
The six-freedom degree of the handle is subjected to smooth interpolation filtering, using the six-freedom degree of the filtered handle as described in
The final six-freedom degree of handle；
Historical track recording unit, for saving the six-freedom degree information of the handle, the six-freedom degree information record
The historical track of the illuminator and the 2nd IMU.
8. a kind of positioning system for positioning object, which is characterized in that the system comprises: as described in claim any one of 5-7
Wear display equipment and wear the corresponding handle of display equipment with described；
Display equipment is worn described in the wired or wireless connection of handle.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710370301.7A CN107153369B (en) | 2017-05-23 | 2017-05-23 | It is a kind of to position the localization method of object, system and wear display equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710370301.7A CN107153369B (en) | 2017-05-23 | 2017-05-23 | It is a kind of to position the localization method of object, system and wear display equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107153369A CN107153369A (en) | 2017-09-12 |
| CN107153369B true CN107153369B (en) | 2019-08-20 |
Family
ID=59793315
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710370301.7A Active CN107153369B (en) | 2017-05-23 | 2017-05-23 | It is a kind of to position the localization method of object, system and wear display equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107153369B (en) |
Families Citing this family (18)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107490866B (en) * | 2017-09-20 | 2019-11-29 | 中国航空工业集团公司洛阳电光设备研究所 | A kind of method of the wearable display system of pilot and its positioning head posture |
| CN108022302B (en) * | 2017-12-01 | 2021-06-29 | 深圳市天界幻境科技有限公司 | Stereo display device of Inside-Out space orientation's AR |
| CN109991740A (en) * | 2017-12-30 | 2019-07-09 | 深圳多哚新技术有限责任公司 | One kind is frivolous to wear display equipment |
| CN108762527A (en) * | 2018-05-23 | 2018-11-06 | 重庆创通联达智能技术有限公司 | A kind of recognition positioning method and device |
| CN110378339B (en) * | 2019-07-22 | 2021-08-17 | 海信视像科技股份有限公司 | Method and device for determining the position of a luminous body |
| CN110572635A (en) * | 2019-08-28 | 2019-12-13 | 重庆爱奇艺智能科技有限公司 | A method, device and system for tracking and positioning of handheld control devices |
| CN111061363A (en) * | 2019-11-21 | 2020-04-24 | 青岛小鸟看看科技有限公司 | a virtual reality system |
| CN111174683B (en) * | 2020-01-07 | 2021-11-30 | 青岛小鸟看看科技有限公司 | Handle positioning method, head-mounted display device and storage medium |
| CN111420391A (en) * | 2020-03-04 | 2020-07-17 | 青岛小鸟看看科技有限公司 | A head-mounted display system and its spatial positioning method |
| CN112286343A (en) | 2020-09-16 | 2021-01-29 | 青岛小鸟看看科技有限公司 | Positioning tracking method, platform and head mounted display system |
| CN112822480B (en) | 2020-12-31 | 2022-05-17 | 青岛小鸟看看科技有限公司 | VR system and positioning tracking method thereof |
| CN113012232B (en) * | 2021-03-01 | 2023-10-27 | 重庆工程职业技术学院 | A positioning system and method for use in mines |
| CN114681907B (en) * | 2022-04-27 | 2025-04-01 | 歌尔股份有限公司 | Positioning method, head mounted device and computer readable storage medium |
| CN117670994A (en) * | 2022-08-22 | 2024-03-08 | 华为技术有限公司 | An image processing method, calibration system and related equipment |
| CN116257134B (en) * | 2023-02-09 | 2024-04-09 | 玩出梦想(上海)科技有限公司 | Handle helmet tracking method, device, equipment and medium in non-inertial reference frame |
| CN119228877A (en) * | 2023-06-30 | 2024-12-31 | 北京字跳网络技术有限公司 | Coordinate system alignment method, device, storage medium and program product |
| CN121794653A (en) | 2023-08-25 | 2026-04-03 | 三星电子株式会社 | Electronic devices, methods, and storage media for acquiring images used for tracking |
| WO2025048131A1 (en) * | 2023-08-25 | 2025-03-06 | 삼성전자주식회사 | Electronic device, method, and storage medium for acquiring images for tracking |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6474159B1 (en) * | 2000-04-21 | 2002-11-05 | Intersense, Inc. | Motion-tracking |
| CN105487670A (en) * | 2015-12-18 | 2016-04-13 | 北京诺亦腾科技有限公司 | Method for positioning physical prop of virtual reality system and virtual reality system |
| CN105528079A (en) * | 2015-12-18 | 2016-04-27 | 北京诺亦腾科技有限公司 | Method for locating physical prop of virtual reality system and virtual reality system |
| CN105607737A (en) * | 2015-12-18 | 2016-05-25 | 北京诺亦腾科技有限公司 | Positioning method for physical prop of virtual reality system, and virtual reality system |
| CN206096621U (en) * | 2016-07-30 | 2017-04-12 | 广州数娱信息科技有限公司 | Enhancement mode virtual reality perception equipment |
-
2017
- 2017-05-23 CN CN201710370301.7A patent/CN107153369B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6474159B1 (en) * | 2000-04-21 | 2002-11-05 | Intersense, Inc. | Motion-tracking |
| CN105487670A (en) * | 2015-12-18 | 2016-04-13 | 北京诺亦腾科技有限公司 | Method for positioning physical prop of virtual reality system and virtual reality system |
| CN105528079A (en) * | 2015-12-18 | 2016-04-27 | 北京诺亦腾科技有限公司 | Method for locating physical prop of virtual reality system and virtual reality system |
| CN105607737A (en) * | 2015-12-18 | 2016-05-25 | 北京诺亦腾科技有限公司 | Positioning method for physical prop of virtual reality system, and virtual reality system |
| CN206096621U (en) * | 2016-07-30 | 2017-04-12 | 广州数娱信息科技有限公司 | Enhancement mode virtual reality perception equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107153369A (en) | 2017-09-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107153369B (en) | It is a kind of to position the localization method of object, system and wear display equipment | |
| CN106525003B (en) | A kind of attitude measurement method based on binocular vision | |
| TWI520576B (en) | Method and system for converting 2d images to 3d images and computer-readable medium | |
| CN108022302B (en) | Stereo display device of Inside-Out space orientation's AR | |
| JP4708752B2 (en) | Information processing method and apparatus | |
| US20170132806A1 (en) | System and method for augmented reality and virtual reality applications | |
| CN116342661A (en) | Binocular visual-inertial odometry method using landmark point offset pose correction | |
| Dorfmüller | Robust tracking for augmented reality using retroreflective markers | |
| US20170337701A1 (en) | Method and system for 3d capture based on structure from motion with simplified pose detection | |
| Dorfmüller-Ulhaas | Robust optical user motion tracking using a kalman filter | |
| JP6483832B2 (en) | Method and system for scanning an object using an RGB-D sensor | |
| CN102788572B (en) | A method, device and system for measuring the posture of a construction machinery hook | |
| EP2510379A2 (en) | System and method for determining geo-location(s) in images | |
| US20160225153A1 (en) | Apparatus and method for tracking eye-gaze | |
| JP6288858B2 (en) | Method and apparatus for estimating position of optical marker in optical motion capture | |
| CN107613867A (en) | Action display system and program | |
| US10110822B2 (en) | Method for tracking at least one object and method for replacing at least one object by a virtual object in a moving image signal recorded by a camera | |
| CN117611664A (en) | Image information and inertial navigation data fusion method | |
| WO2020031659A1 (en) | Position and attitude estimation system, position and attitude estimation apparatus, and position and attitude estimation method | |
| JP6205387B2 (en) | Method and apparatus for acquiring position information of virtual marker, and operation measurement method | |
| Davidson et al. | Monocular vision-based range estimation supported by proprioceptive motion | |
| Alshamaa et al. | RobCap: A mobile motion capture system mounted on a robotic arm | |
| JP2013120150A (en) | Human position detection system and human position detection method | |
| He et al. | Fusion of inertial sensing to compensate for partial occlusions in optical tracking systems | |
| CN116295327A (en) | Positioning method, device, electronic device and computer-readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |