CN116485993A - A Real-time Panoramic 3D Reconstruction Method Based on Virtual Stereo Unwrapping Method - Google Patents

A Real-time Panoramic 3D Reconstruction Method Based on Virtual Stereo Unwrapping Method Download PDF

Info

Publication number
CN116485993A
CN116485993A CN202310195255.7A CN202310195255A CN116485993A CN 116485993 A CN116485993 A CN 116485993A CN 202310195255 A CN202310195255 A CN 202310195255A CN 116485993 A CN116485993 A CN 116485993A
Authority
CN
China
Prior art keywords
camera
projector
phase
virtual
unwrapping
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202310195255.7A
Other languages
Chinese (zh)
Inventor
林斌
王恒宇
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Zhejiang University ZJU
Original Assignee
Zhejiang University ZJU
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Zhejiang University ZJU filed Critical Zhejiang University ZJU
Priority to CN202310195255.7A priority Critical patent/CN116485993A/en
Publication of CN116485993A publication Critical patent/CN116485993A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three-dimensional [3D] modelling for computer graphics
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B30/00—Optical systems or apparatus for producing three-dimensional [3D] effects, e.g. stereoscopic images
    • G—PHYSICS
    • G03—PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
    • G03B—APPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR
    • G03B35/00—Stereoscopic photography
    • G—PHYSICS
    • G03—PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
    • G03B—APPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR
    • G03B35/00—Stereoscopic photography
    • G03B35/08—Stereoscopic photography by simultaneous recording
    • G03B35/10—Stereoscopic photography by simultaneous recording having single camera with stereoscopic-base-defining system

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Computer Graphics (AREA)
  • Geometry (AREA)
  • Software Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Optics & Photonics (AREA)
  • Length Measuring Devices By Optical Means (AREA)

Abstract

The invention discloses a real-time panoramic three-dimensional reconstruction method based on a virtual stereo unwrapping method, which only comprises a camera, a projector and a turntable, wherein the system captures three-step phase shift sinusoidal fringes through projection at each angle, and after wrapping phases are obtained, the virtual camera-projector system formed under adjacent rotation angles is used for carrying out auxiliary unwrapping, so that a 360-degree three-dimensional model is finally synthesized. The method has the advantages that only a single camera is adopted, the system cost is reduced compared with a multi-camera SPU system, and more visual angles can be provided for unwrapping in the process of utilizing the virtual camera-projector system to assist unwrapping, so that the unwrapping stability is greatly improved.

Description

Real-time panoramic three-dimensional reconstruction method based on virtual stereoscopic unwrapping method
Technical Field
The invention relates to the field of optical measurement, in particular to a real-time panoramic three-dimensional reconstruction method based on a virtual stereoscopic unwrapping method.
Background
The optical three-dimensional (3D) morphology reconstruction technology has wide application in the fields of intelligent manufacturing, medical treatment and the like [1-3]. Among them, fringe Projection Profilometry (FPP) [4-5] plays an important role in it because of its high accuracy, full field reconstruction, etc. However, the three-dimensional reconstruction of the single-direction surface type cannot obtain all information of the target due to shadow shielding and other problems, and the requirements in the fields of reverse modeling, industrial detection and the like cannot be met gradually [6], and the rapid, high-precision and 360-degree three-dimensional reconstruction is increasingly focused by academia and industry [7].
To realize 360-degree three-dimensional reconstruction, point clouds of objects are required to be acquired and spliced under different view angles, the first realization method is a multi-angle stereoscopic digital correlation (DIC) system, a J.O.Teu < 8 > and the like are adopted for the first time to measure the panoramic strain of a cylindrical sample by adopting a multi-view DIC technology by adopting a 4-camera system, but the system still has a limited view field; DANA SOLAV [9] and the like measure the full-field deformation and strain of the lower limbs of the human body by adopting a 12-camera system. However, the multi-camera system increases the equipment cost and calibration of the system is difficult. Another method is that the point clouds acquired by a single measurement system under different view angles are registered, and the method of point cloud registration can be divided into algorithm registration and instrument assistance. The point cloud registration technique relies on the repeated parts of the point cloud under different view angles to splice, szymon [10] et al propose a real-time 360-degree three-dimensional model acquisition technique based on iterative nearest neighbor points (ICPs), but the method skips coarse matching to result in lower registration accuracy [7]. An improved registration strategy from thick to thin is proposed in 2019,Jiaming Qian[7, and the accuracy is greatly improved while the registration speed is ensured. However, the technology based on point cloud registration still has serious time consumption, is difficult to be suitable for a real-time matching scene, and generally requires a person to manually move an object to be measured, so that the method is difficult to be suitable for an automatic three-dimensional measurement scene. The instrument assistance comprises mirror assistance, turntable assistance and the like, the mirror assistance is used for capturing targets with three angles through a mirror to realize panoramic three-dimensional measurement [11], but the number of view angles provided by the method is limited, and the 360-degree reconstruction requirement of a complex object cannot be met. The turntable is assisted to obtain point clouds under any angle by rotating the turntable, the point clouds obtained under each angle can finish point cloud registration only by rotating the point clouds under a rotating shaft coordinate system by a corresponding angle, a precise panoramic model can be obtained by selecting smaller rotating step length for complex objects, a method for realizing automatic calibration of a rotating shaft by using a calibration plate is provided by Meiling Dai [12] and the like, and the calibration precision of the rotating shaft is improved by using an auxiliary camera by using Xiaoqi Cai [13 ]. The Xiaoli Liu [6] and the like realize panoramic reconstruction of a model by using a plurality of 3D cameras and an automatic control turntable, but the panoramic reconstruction time is longer because each sensor needs to reconstruct in a time-sharing way and each turntable needs to stop at each angle to wait for the reconstruction to finish. Because the FPP requires multi-frame projection, reconstruction can be destroyed in the rotating process of the turntable, the efficiency of panoramic reconstruction is severely limited, and the real-time three-dimensional reconstruction based on the FPP [2,14,15] starts to develop along with the development of high-speed projectors in the present year. The method for realizing real-time three-dimensional panorama reconstruction based on a turntable by using a four-camera projector system comprises the steps of projecting single-period phase shift stripes to obtain wrapping phases, and enabling four cameras to assist in unwrapping by matching with an Adaptive Depth Constraint (ADC) strategy to realize real-time three-dimensional reconstruction, wherein the four high-speed cameras become high in cost and difficult to calibrate the system by using a three-dimensional phase unwrapping (SPU) method.
In summary, the current multi-view DIC has the problems of complex system and high cost, the mirror-based 360-degree reconstruction method has the problems of small number of view angles, small field of view (FOV), and the like, and the four-camera projector-based SPU system has the problems of complex system, high implementation cost, and the like.
Disclosure of Invention
In order to solve the defects of the prior art, the invention aims to provide a real-time panoramic three-dimensional reconstruction method based on a virtual stereo unwrapping method, and provides a novel real-time 360-degree three-dimensional reconstruction method based on a virtual stereo unwrapping (VSPU) method. The method has the advantages that only a single camera is adopted, the system cost is reduced compared with a multi-camera SPU system, and more visual angles can be provided for unwrapping in the process of utilizing the virtual camera-projector system to assist unwrapping, so that the unwrapping stability is greatly improved.
In order to achieve the above object, the technical method adopted by the invention is as follows:
the invention discloses a real-time panoramic three-dimensional reconstruction method based on a virtual three-dimensional unwrapping method, which comprises the following steps: constructing a three-dimensional reconstruction system, wherein the system comprises a computer, a projector, a camera, a turntable controller, a turntable and a projector, wherein the projector, the camera and the turntable controller are respectively connected with the computer, the turntable is connected with the turntable controller, and the projector is connected with the camera;
calibrating the projector, the camera and the turntable to obtain the camera, the internal and external parameters of the projector and the position of the rotating shaft of the turntable;
sending a rotating instruction to a turntable controller, and enabling the turntable to rotate at a constant speed;
performing single virtual stereo unwrapping (VSPU);
after single virtual stereo unwrapping (VSPU) is completed, delay Δt reg Performing the next VSPU process until 360-degree scanning is completed;
the specific process of single virtual stereo unwrapping (VSPU) includes:
projection of projector shoots 2m+1 (m is larger than or equal to 1) times of multi-step phase shift map, and each time delay is deltat vspu When in each projection, the camera shoots a phase shift diagram after distortion through hard triggering;
according to the multi-step phase shift diagrams under different angles, respectively calculating the wrapping phase diagrams under the { -m, -m+1, … m-1, m } positions;
obtaining a limited number of possible k values through constant geometric constraint (CDC) according to a parcel phase diagram at a 0 position, and obtaining point clouds with different k values through a triangular ranging method;
projecting the obtained point cloud to a virtual camera target surface and a virtual projector target surface at the { -m, -m+1, -1, … m-1, m } positions respectively, obtaining wrapping phase diagrams of the points on the virtual camera and the virtual projector, and obtaining wrapping phase error diagrams at the { -m, -m+1, -1, … m-1, m } positions by making absolute values of differences;
obtaining a wrapped phase error map under different k values by summing the wrapped phase error maps, and obtaining a k value with the smallest error to obtain a final unwrapped k value map;
and obtaining point clouds under the 0-position angle through triangular ranging according to the unwrapped k-value graph, and registering the point clouds into the 360-degree panoramic model.
As a further improvement, the invention calculates the wrapping phase diagram under the positions of { -m, -m+1, … m-1, m } according to multi-step phase shift diagrams under different angles, and specifically comprises the following steps:
the projector projects the phase-shifted fringes onto the object, the camera captures a distorted fringe pattern, which can be expressed as:
I n representing projection of an nth stripe image, n epsilon 0, N; n represents the phase shift step number, in general, the larger the step number is, the higher the measurement precision is, and the three-step phase shift method is adopted in the invention;representing the pixel coordinates of the ith camera image (hereinafter, for simplicity, use +.>A representation); a represents average intensity and B represents amplitude; phi represents the phase;Representing the phase shift, the phase phi may be obtained by a phase shift algorithm:
atan2 is a signed arctangent operation, the wrapping phase phi is extended to (-pi, pi) by the sign of the numerator and denominator for three-step phase shift:
atan2 is a signed arctangent operation, which spreads the wrapping phase phi to (-pi, pi) by the sign of the numerator and denominator.
As a further improvement, the invention provides that a finite number of possible k values are obtained by constant geometric constraint (CDC) according to the parcel phase diagram at position 0, in particular:
in CDC, the system can set a rough range Z according to the size of the object min And Z max Expressed as:
Z min ≤Z w (o c ，K c )≤Z max ，
for a 360-degree measuring system, the CDC excludes some k values out of range, the depth range of the measured object at different angles can be changed greatly, and the phase ambiguity cannot be eliminated.
As a further improvement, the obtained point clouds are respectively projected to a virtual camera target surface and a virtual projector target surface at the { -m, -m+1, -1, … m-1, m } positions to obtain the wrapping phase diagrams of the points on the virtual camera and the virtual projector, and the absolute value of the difference is obtained to obtain the wrapping phase error diagram at the { -m, -m+1, -1,1 … m-1, m } positions, specifically:
taking an object to be measured as a reference system, a camera-projector system is considered to rotate around a rotary shaft of a rotary table to form a virtual camera-projector system, the phase difference angle theta of adjacent camera-projector systems is theta=v·Δt, wherein v represents the rotary speed of the rotary table, Δt represents the interval time for shooting and collecting phase-shift images, the rotary shaft of the rotary table is taken as the z-axis of a world coordinate system, and the relationship between the camera and the projector under different angles can be expressed as follows:
unlike the spatial stereoscopic unwrapping system, the projector in the virtual system is also rotated, in order to use the adjacent angle virtual camera-projector for auxiliary unwrapping, the possible points obtained by the main camera are projected to the adjacent virtual projector and virtual camera, respectively, expressed as:
wherein the method comprises the steps ofRespectively represent the projection of the kth possible point to camera c i And projector p i The coordinates of the target surface are calculated,transformation matrix respectively representing ith camera and projector under world coordinate system, A c ，A p An internal reference matrix representing the camera and projector, respectively, < >>External reference matrix representing i-th camera and projector, respectively,/->Representing the kth possible point of the main camera in the world coordinate system, the phase projected to the camera target surface>Image pairs of parcel phase images obtained by Eq (3) at the corresponding rotation angle +.>The two-dimensional interpolation is carried out, phase compensation is needed at the jump position, and the phase value projected to the target surface of the projector can be expressed as:
wherein the method comprises the steps ofRepresentation->Is u-axis, the camera-projector limit constraint axis, +.>Is within the range of [ -pi, pi]And (3) adopting the wrapping phase errors projected by the virtual projector and the camera by the possible points as a discrimination standard, screening the possible points, wherein the errors are expressed as follows:
wherein the method comprises the steps ofFor the wrapped phase difference of the kth possible point under the ith virtual camera, +.>Representing corresponding discrimination error, and actively making jump to ensure that the phase deviation is [0, pi ]]And (3) inner part.
As a further improvement, the method sums the wrapped phase error graphs to obtain wrapped phase error graphs under different k values, and takes the k value with the smallest error to obtain a final unwrapped k value graph, which specifically comprises:
the final selected k value is the minimum sum of the deviations of all virtual systems,
k result ＝minIndex(Err(k))。
wherein k is result For the k value obtained by the VSPU method, err (k) is the phase error of different k values, minIndex (·) is the index function of the minimum value, m is the number of selected adjacent virtual systems, and the number of auxiliary unwrapped systems is 2m.
As a further improvement, the single VSPU process of the invention comprises five projection shooting multi-step phase shift diagrams, and each time delay is theta vspu The method comprises the steps of carrying out a first treatment on the surface of the Delay theta after single angle VSPU process is finished reg The next VSPU process is carried out, specifically:
firstly, after the turntable is started, the turntable is rotationally accelerated to a uniform speed omega at a constant angular acceleration alpha turn Stabilization time is Δt=ω turn After that, the system starts projection shooting, each time t passes reg A set of fringe patterns for the VSPU are taken at time intervals defined by the set registration angle interval delta theta reg To be obtained Δt reg ＝Δθ reg /ω turn 。
Compared with the prior art, the invention has the beneficial effects that:
1. the setting is simple, and the cost is low: only a single camera, projector and turret are required to achieve 360 ° panoramic reconstruction, which requires significantly lower cost than the multiple camera systems of the SPU system. In addition, the method does not need synchronous triggering of the projector and a plurality of cameras, and is simpler in arrangement.
2. Easy implementation: the calibration process of the method only comprises a single camera-projector, and the camera-turntable rotating shaft calibration is simpler compared with an SPU method. In the reconstruction process, the method can flexibly select the number of the virtual systems for assisting in unwrapping in the rotation process of the detected object, and is more stable than an SPU method.
3. More application scenarios: the real-time 360-degree reconstruction system of the method can be applied to a real-time industrial detection scene, and for a single-sided structure light detection system, complete defect detection cannot be carried out on complex objects due to shadow shielding and other problems, and 360-degree panoramic defect detection can be realized by providing simple setting. In addition, the system can be applied to scenes such as reverse engineering, cultural relic protection and the like, and a complete 360-degree three-dimensional model can be obtained only by 47 s. The method needs to know the relative displacement relation between the measured object and the measuring system in advance, so the method can be applied to real-time detection of a production line, the phase damage can be caused by projecting multi-step phase shift stripes due to the high moving speed of the production line, and the method can be used for capturing single-period phase shift stripes at different moving positions for unwrapping, so that complete structured light reconstruction is realized for real-time defect detection.
Drawings
FIG. 1 is a diagram of a VSPU real-time 360 DEG three-dimensional reconstruction system of the present invention;
in the figure, 0 is a camera, a dotted line part is a virtual camera in the rotating process, 1 is a projector, the dotted line part is a virtual projector in the rotating process, 2 is a turntable, 3 is a geometric constraint measuring range of the system, and 4 is the rotating direction;
FIG. 2 is a data flow diagram of the method of the present invention;
FIG. 3 is a phase error plot of the present invention;
Detailed Description
The technical scheme of the invention is further described by the specific embodiments with reference to the accompanying drawings in the specification:
FIG. 1 is a diagram of a VSPU real-time 360 DEG three-dimensional reconstruction system of the present invention; the camera comprises a camera 0, a projector 1, an electric turntable 2, a turntable 2 controller and a computer, wherein the computer is respectively communicated with the camera 0 and the projector 1, the turntable 2 controller is connected with the camera 0 through a serial port, the turntable 2 controller is communicated with the turntable 2 through a 232 interface, the projector 1 is connected with the camera 0 through a trigger line for projection shooting, when the projector 1 projects a fringe pattern each time, a trigger signal is sent to the camera 0, and the distorted fringe pattern is captured by the camera 0 and is used for projection shooting synchronization under a high-speed scene. When the VSPU is in three-dimensional reconstruction, an object to be detected is placed on the turntable 2, the control equipment sends an instruction to the turntable 2 to rotate along the direction 4 at a constant speed, meanwhile, under the control of precise timing, the computer controls the projector 1 to project a group of single-period phase shift stripes every time a fixed time passes, and the camera 0 acquires a distorted phase shift image in a triggering mode. The object to be measured is driven to rotate by the turntable 2, and the object to be measured is taken as a reference system, which is equivalent to the rotation of the reconstruction system around the rotation axis to form a virtual system (the part of the dashed line frame in fig. 1).
FIG. 2 is a data flow diagram of the method of the present invention; comprising the following steps:
calibrating the projector 1, the camera 0 and the turntable 2 to obtain the camera 0, the internal and external parameters of the projector 1 and the position of the rotating shaft of the turntable 2; the method comprises the following steps:
the calibration process of the system comprises a projector 1, camera 0 calibration and turntable 2 calibration. The camera 0 calibration process adopts a Zhang calibration method [16], the checkerboard is placed on the turntable 2 to rotate along the direction 4 at a fixed angle, the camera 0 acquires checkerboard images at different angles, and checkerboard corner points are extracted for calculating internal parameters and distortion coefficients of the camera 0. The projector 1 is calibrated by adopting the Wei Gao 17 method, red and blue checkers are placed under different angles, the projector 1 projects phase shift images in the transverse and longitudinal directions, the angular point coordinates of the red and blue checkers are mapped to the projector 1, the projector 1 is used as a reverse camera 0 to calculate the internal parameters, external parameters and distortion coefficients of the projector 1, and the distortion correction of the projector 1 is necessary, so that the precision of a three-dimensional reconstruction system and the precision of the projection of three-dimensional points to the coordinates of the target surface of the projector 1 can be ensured.
And (3) calibrating the rotating shaft of the turntable 2 by adopting the checkerboard under different rotating angles, extracting corner points of the checkerboard by the camera 0, and calculating three-dimensional coordinates under a camera 0 coordinate system by a PnP algorithm. Circular fitting is carried out on the checkerboard coordinates under different angles by an SVD method [18], a rotation central axis is calculated, and the rotation central axis is taken as a z axis of a world coordinate system.
Sending a rotation instruction to a turntable 2 controller, and enabling the turntable 2 to rotate at a constant speed;
performing single virtual stereo unwrapping (VSPU);
the specific process of single virtual stereo unwrapping (VSPU) includes:
projector 1 projects and shoots 2m+1 (m is larger than or equal to 1) three-step phase shift map, and each time delay is deltat vspu Each of the followingDuring secondary projection, the camera 0 shoots a phase shift diagram after distortion through hard triggering;
according to the three-step phase shift diagrams under different angles, respectively calculating a wrapping phase diagram under the { -m, -m+1, … m-1, m } positions; the method comprises the following steps:
projector 1 projects phase-shifted fringes onto an object, camera 0 captures a distorted fringe pattern, and the fringe pattern captured by camera 0 can be expressed as:
I n representing projection of an nth stripe image, n epsilon 0, N; n represents the phase shift step number, in general, the larger the step number is, the higher the measurement precision is, and the three-step phase shift method is adopted in the invention;representing the pixel coordinates of the ith camera 0 image (hereinafter, for simplicity, use +.>A representation); a represents average intensity and B represents amplitude; phi represents the phase;Representing the phase shift, the phase phi may be obtained by a phase shift algorithm:
atan2 is a signed arctangent operation, the wrapping phase phi is extended to (-pi, pi) by the sign of the numerator and denominator for three-step phase shift:
from the parcel phase map at position 0, measured by constant geometric constraint (CDC)Obtaining a limited number of possible k values in the range 3, and obtaining point clouds with different k values through a triangulation method; the measurement system is set to a limited range due to projection brightness and focus depth limitations of the projector 1; in CDC, the system can set a rough range Z according to the size of the object min And Z max Expressed as:
Z min ≤Z w (o c ，K c )≤Z max ，
for a 360-degree measurement system, the CDC excludes some out-of-range k values, but the depth range of the measured object at different angles can be greatly changed, and the phase ambiguity cannot be eliminated.
Projecting the obtained point cloud to a virtual camera 0 target surface and a virtual projector 1 target surface at the { -m, -m+1, -1, … m-1, m } positions respectively, obtaining wrapping phase maps of the points on the virtual camera 0 and the virtual projector 1, and obtaining a wrapping phase error map at the { -m, -m+1, -1,..m-1, m } positions by making absolute values of differences; in the invention, m=2 is adopted, and the specific steps are as follows:
taking an object to be measured as a reference system, the camera 0-projector 1 system rotates around the rotating shaft of the turntable 2 to form a virtual camera 0-projector 1 system, the phase difference angle theta between the adjacent camera 0-projector 1 systems is theta=v·Δt, wherein v represents the rotating speed of the turntable 2, Δt represents the interval time for shooting and collecting phase-shift images, the rotating shaft is taken as the z-axis of the world coordinate system, and the relationship between the camera 0 and the projector 1 under different angles can be expressed as follows:
unlike the spatial stereoscopic unwrapping system, the projector 1 in the virtual system is also rotated, in order to use the virtual camera 0-projector 1 of adjacent angle for auxiliary unwrapping, the possible points obtained by the main camera 0 are projected to the adjacent virtual projector 1 and virtual camera 0, respectively, expressed as:
wherein the method comprises the steps ofRespectively represent the projection of the kth possible point to camera 0c i And projector 1p i The coordinates of the target surface are calculated,transformation matrices respectively representing the ith camera 0 and projector 1 in world coordinate system, A c ，A p Reference matrix representing camera 0 and projector 1, respectively,/->The extrinsic matrices representing the i-th camera 0 and projector 1, respectively, < ->Representing the kth possible point of the main camera 0 in the world coordinate system, for the correct possible point, the phase obtained by the virtual camera 0 projected to the adjacent angle should be the same as the phase of the corresponding projector 1, and for the error point, there is a larger deviation, the phase projected to the target surface of the camera 0 is +.>Image pairs of parcel phases obtainable by Eq (3) at the corresponding rotation angle +.>The two-dimensional interpolation is performed, phase compensation is needed at the jump position, and the phase value projected to the target surface of the projector 1 can be expressed as:
wherein the method comprises the steps ofRepresentation->Is the u-axis, the camera 0-projector 1 limit constraint axis, +.>Is within the range of [ -pi, pi]The wrapping phase errors of the projections of the virtual projector 1 and the camera 0 by the possible points are taken as discrimination criteria, the possible points are screened, and the errors are expressed as follows:
wherein the method comprises the steps ofFor the wrapped phase difference of the kth possible point under the ith virtual camera 0, +.>Representing the corresponding discrimination error, the phases of the camera 0 and the projector 1 may fall on different k when checking the phase deviation due to the jump of the wrapping phase, and to avoid such ambiguity, the jump is actively made to ensure the phase deviation to be 0, pi]And (3) inner part.
Obtaining a wrapped phase error map under different k values by summing the wrapped phase error maps, and obtaining a k value with the smallest error to obtain a final unwrapped k value map; the method comprises the following steps: the method utilizes the virtual projector 1-camera 0 system in the rotation process to carry out three-dimensional unwrapping, can flexibly select the number of virtual systems, ensures the stability of the method by unwrapping more virtual systems, and finally selects the k value as the minimum value of the sum of the deviations of all the virtual systems,
k result ＝minIndex(Err(k))
wherein k is result For the k value obtained by the VSPU method, err (k) is the phase error of different k values, minIndex (·) is the index function of the minimum value, m is the number of selected adjacent virtual systems, and the number of auxiliary unwrapped systems is 2m.
And obtaining point clouds under the 0-position angle through triangular ranging according to the unwrapped k-value graph, and registering the point clouds into the 360-degree panoramic model.
After single virtual stereo unwrapping (VSPU) is completed, delay Δt reg The next VSPU process is performed until a 360 degree scan is completed.
The single VSPU process comprises five projection shooting three-step phase shift diagrams, and each time delay is delta t vspu The method comprises the steps of carrying out a first treatment on the surface of the After the VSPU process of a single angle is finished, delay delta t reg The next VSPU process is carried out, specifically:
first, the turntable 2 is started and then rotationally accelerated to a constant speed omega at a constant angular acceleration alpha turn Stabilization time is Δt=ω turn After which the system starts projection shooting, each time passing Δt reg A set of fringe patterns for the VSPU are taken at time intervals defined by the set registration angle interval delta theta reg To be obtained Δt reg ＝Δθ reg /ω turn In the present invention, the turntable 2 rotates at a speed ω turn Turntable acceleration α=4°/s=8°/s 2 Registration interval Δt reg =1.5 s, registration angle interval Δθ reg =12°, VSPU angular interval Δθ vspu By means of the method, 360-degree three-dimensional reconstruction can be achieved within 47s, and 30-angle point clouds are obtained.
While only the preferred embodiments of the present invention have been described above, it should be noted that it will be apparent to those skilled in the art that various modifications and adaptations can be made without departing from the essential technical features of the present invention, and the modifications and adaptations should and are intended to be comprehended within the scope of the present invention.
Reference to the literature
[1]J.Salvi,S.Fernandez,T.Pribanic,X.Llado,A state of the art in structured light patterns for surface profilometry,Pattern recognition,43(2010)2666-2680.
[2]S.Zhang,Recent progresses on real-time 3D shape measurement using digital fringe projection techniques,Optics and lasers in engineering,48(2010)149-158.
[3]S.S.Gorthi,P.Rastogi,Fringe projection techniques:whither we are？,Optics and lasers in engineering,48(2010)133-140.
[4]S.Zhang,High-speed 3D shape measurement with structured light methods:A review,Optics and Lasers in Engineering,106(2018)119-131.
[5]S.Feng,C.Zuo,T.Tao,Y.Hu,M.Zhang,Q.Chen,G.Gu,Robust dynamic 3-d measurements with motion-compensated phase-shifting profilometry,Optics and Lasers in Engineering,103(2018)127-138.
[6]X.Liu,X.Peng,H.Chen,D.He,B.Z.Gao,Strategy for automatic and complete three-dimensional optical digitization,Optics letters,37(2012)3126-3128.
[7]J.Qian,S.Feng,T.Tao,Y.Hu,K.Liu,S.Wu,Q.Chen,C.Zuo,High-resolution real-time 360 3d model reconstruction of a handheld object with fringe projection profilometry,Optics letters,44(2019)5751-5754.
[8]J.-J.Orteu,F.Bugarin,J.Harvent,L.Robert,V.Velay,Multiple-camera instrumentation of a single point incremental forming process pilot for shape and 3D displacement measurements:methodology and results,Experimental mechanics,51(2011)625-639.
[9]D.Solav,K.M.Moerman,A.M.Jaeger,K.Genovese,H.M.Herr,MultiDIC:An open-source toolbox for multi-view 3D digital image correlation,IEEE Access,6(2018)30520-30535.
[10]S.Rusinkiewicz,O.Hall-Holt,M.Levoy,Real-time 3D model acquisition,ACM Transactions on Graphics(TOG),21(2002)438-446.
[11]W.Yin,S.Feng,T.Tao,L.Huang,S.Zhang,Q.Chen,C.Zuo,Calibration method for panoramic 3D shape measurement with plane mirrors,Optics Express,27(2019)36538-36550.
[12]M.Dai,L.Chen,F.Yang,X.J.A.o.He,Calibration of revolution axis for 360deg surface measurement,52(2013)5440-5448.
[13]X.Cai,K.Zhong,Y.Fu,J.Chen,Y.Liu,C.Huang,Calibration method for the rotating axis in panoramic 3D shape measurement based on a turntable,Measurement Science and Technology,32(2020)035004.
[14]J.Qian,T.Tao,S.Feng,Q.Chen,C.Zuo,Motion-artifact-free dynamic 3D shape measurement with hybrid Fourier-transform phase-shifting profilometry,Optics express,27(2019)2713-2731.
[15]S.Feng,Q.Chen,G.Gu,T.Tao,L.Zhang,Y.Hu,W.Yin,C.Zuo,Fringe pattern analysis using deep learning,Advanced Photonics,1(2019)025001.
[16]Z.Zhang,A flexible new technique for camera calibration,IEEE Transactions on pattern analysis and machine intelligence,22(2000)1330-1334.
[17]W.Gao,L.Wang,Z.Hu,Flexible method for structured light system calibration,Optical Engineering,47(2008)083602.
[18]T.-H.Lin,Automatic 3D color shape measurement system based on a stereo camera,Applied Optics,59(2020)2086-2096.

Claims (6)

1.一种基于虚拟立体解包裹方法的实时全景三维重建方法，其特征在于，包括：1. A real-time panoramic 3D reconstruction method based on a virtual stereoscopic unwrapping method, characterized in that it includes: 构建三维重建系统，所述的系统包括计算机、分别与计算机相连的投影仪、相机、转台控制器、与转台控制器相连的转台、所述的投影仪与相机相连；A three-dimensional reconstruction system is constructed, the system including a computer, a projector and a camera respectively connected to the computer, a turntable controller, a turntable connected to the turntable controller, and the projector and camera being connected. 对投影仪、相机和转台进行标定，得到相机，投影仪的内外参数和转台旋转轴的位置；The projector, camera, and turntable are calibrated to obtain the internal and external parameters of the camera and projector, and the position of the turntable's rotation axis. 向转台控制器发出旋转指令，转台匀速转动；A rotation command is sent to the turntable controller, and the turntable rotates at a constant speed; 进行单次虚拟立体解包裹(VSPU)；Perform a single virtual stereo unpacking (VSPU); 单次虚拟立体解包裹(VSPU)结束后，延时Δtreg进行下一次VSPU过程，直到完成360度扫描；After a single virtual stereo unpacking (VSPU) is completed, a delay of Δt reg is applied before the next VSPU process is performed, until the 360-degree scan is completed. 所述的单次虚拟立体解包裹(VSPU)的具体过程包括：The specific process of the single virtual stereo unpacking (VSPU) includes: 投影仪投影拍摄2m+1(m≥1)次多步相移图，每次的延时为Δtvspu，每次投影时，相机通过硬触发拍摄畸变后的相移图；The projector projects and captures 2m+1 (m≥1) multi-step phase-shifted images, with a delay of Δt vspu for each step. During each projection, the camera captures the distorted phase-shifted image through hard triggering. 根据不同角度下的多步相移图，分别计算{-m，-m+1，...m-1，m}位置下的包裹相位图；Based on the multi-step phase shift diagrams at different angles, calculate the wrap-around phase diagrams at positions {-m, -m+1, ..., m-1, m} respectively; 根据0位置下的包裹相位图，通过恒定几何约束(CDC)得到有限个可能的k值，并通过三角测距法得到不同k值的点云；Based on the wrap-around phase map at position 0, a finite number of possible k values are obtained through constant geometric constraints (CDC), and point clouds with different k values are obtained through triangulation. 将得到的点云分别投影到{-m，-m+1，-1，1，...m-1，m}位置的虚拟相机靶面和虚拟投影仪靶面，得到这些点在虚拟相机和虚拟投影仪上的包裹相位图，做差的绝对值得到{-m，-m+1，-1，1，...m-1，m}位置上的包裹相位误差图；The obtained point cloud is projected onto the virtual camera target surface and the virtual projector target surface at the positions {-m, -m+1, -1, 1, ..., m-1, m}, respectively, to obtain the wrapping phase map of these points on the virtual camera and the virtual projector. The absolute value of the difference is used to obtain the wrapping phase error map at the positions {-m, -m+1, -1, 1, ..., m-1, m}. 通过对包裹相位误差图取和，得到不同k值下的包裹相位误差图，取误差最小的k值，得到最终解包裹k值图；By summing the package phase error maps, we obtain package phase error maps under different k values. We then take the k value with the smallest error to obtain the final unpacking k value map. 根据解包裹的k值图，通过三角测距得到0位置角度下的点云，将这些点云注册到360度全景模型中。Based on the unpacked k-value map, point clouds at the 0 position angle are obtained through triangulation, and these point clouds are registered into the 360-degree panoramic model. 2.根据权利要求1所述的基于虚拟立体解包裹方法的实时全景三维重建方法，其特征在于，所述的根据不同角度下的多步相移图，分别计算{-m，-m+1，...m-1，m}位置下的包裹相位图，具体为：2. The real-time panoramic 3D reconstruction method based on the virtual stereo unwrapping method according to claim 1, characterized in that, the step of calculating the wrapping phase map at positions {-m, -m+1, ..., m-1, m} based on the multi-step phase shift map at different angles specifically comprises: 投影仪投影相移条纹到物体上，相机采集畸变的条纹图，相机采集到的条纹图可以被表示为：A projector projects phase-shifted fringes onto an object, and a camera captures the distorted fringe pattern. The fringe pattern captured by the camera can be represented as: In代表投影第n个条纹图像，n∈[0，N)；N表示相移步数，表示第i个相机图像像素坐标(后文为了简便，用表示)；A表示平均强度，B表示幅度；φ表示相位；表示相移，相位φ可以通过获得通过相移算法：I <sub>n </sub> represents the nth fringe image being projected, where n ∈ [0, N); N represents the phase shift step number. Represents the pixel coordinates of the i-th camera image (hereinafter, for simplicity, it will be referred to as...). (Indicates); A represents average intensity, B represents amplitude; φ represents phase; The phase shift, φ, can be obtained through a phase shift algorithm: atab2为有符号的反正切操作，通过分子和分母的符号将包裹相位φ扩展为(-π，π]。atab2 is a signed arctangent operation that expands the wrapping phase φ to (-π, π) by the signs of the numerator and denominator. 3.根据权利要求1或2所述的基于虚拟立体解包裹方法的实时全景三维重建方法，其特征在于，根据0位置下的包裹相位图，通过恒定几何约束(CDC)得到有限个可能的k值，具体为：3. The real-time panoramic 3D reconstruction method based on the virtual stereo unwrapping method according to claim 1 or 2, characterized in that, based on the wrapping phase map at position 0, a finite number of possible k values are obtained through constant geometric constraints (CDC), specifically: 在CDC中，系统根据物体的大小可以设置一个大致的范围Zmin和Zmax，被表示为：In CDC, the system can set an approximate range Zmin and Zmax based on the size of the object, which are represented as: Zmin≤Zw(oc，Kc)≤Zmax，Z min ≤ Z w (o c , K c ) ≤ Z max , 对于360°测量系统，CDC排除掉一些超出范围的k值，被测物体在不同角度下的深度范围会产生较大变化，无法消除相位模糊。For 360° measurement systems, CDC excludes some k values that are out of range. The depth range of the measured object will vary greatly at different angles, and phase ambiguity cannot be eliminated. 4.根据权利要求3所述的基于虚拟立体解包裹方法的实时全景三维重建方法，其特征在于，所述的将得到的点云分别投影到{-m，-m+1，-1，1，...m-1，m}位置的虚拟相机靶面和虚拟投影仪靶面，得到这些点在虚拟相机和虚拟投影仪上的包裹相位图，做差的绝对值得到{-m，-m+1，-1，1，...m-1，m}位置上的包裹相位误差图，具体为：4. The real-time panoramic 3D reconstruction method based on virtual stereo unwrapping method according to claim 3, characterized in that the step of projecting the obtained point cloud onto the virtual camera target surface and the virtual projector target surface at positions {-m, -m+1, -1, 1, ..., m-1, m} respectively, to obtain the wrapping phase map of these points on the virtual camera and the virtual projector, and taking the absolute value of the difference to obtain the wrapping phase error map at positions {-m, -m+1, -1, 1, ..., m-1, m}, specifically: 以待测物体作为参考系，被认为相机-投影仪系统绕着转台转轴进行旋转形成虚拟相机-投影仪系统，相邻的相机-投影仪系统相差角度θ为θ＝v·Δt，其中v表示转台旋转速度，Δt表示拍摄采集相移图像的间隔时间，以转台旋转轴作为世界坐标系的z轴，不同角度下相机、投影仪的关系可以表示为：Using the object under test as a reference frame, a virtual camera-projector system is formed by rotating around the turntable axis. The phase difference θ between adjacent camera-projector systems is θ = v·Δt, where v represents the turntable rotation speed and Δt represents the interval between capturing phase-shifted images. Using the turntable rotation axis as the z-axis of the world coordinate system, the relationship between the camera and projector at different angles can be expressed as: 与空间立体解包裹系统不同的是，虚拟系统中投影仪也被旋转了，为了采用相邻角度的虚拟相机-投影仪进行辅助解包裹，将主相机得到的可能点分别投影到相邻的虚拟投影仪和虚拟相机，被表示为：Unlike spatial 3D unwrapping systems, the projector in the virtual system is also rotated. To use virtual camera-projector pairs at adjacent angles for assisted unwrapping, the possible points obtained by the main camera are projected onto adjacent virtual projectors and virtual cameras, respectively, as shown below: 其中分别表示第k个可能点投影到相机ci和投影仪pi靶面的坐标，分别表示世界坐标系下第i个相机和投影仪的变换矩阵，Ac，Ap分别表示相机和投影仪的内参矩阵，分别表示第i个相机和投影仪的外参矩阵，表示世界坐标系下主相机第k个可能点，投影到相机靶面的相位通过对应旋转角度下Eq(3)得到的包裹相位图像对进行二维插值得到，在跳变位置需要进行相位补偿，投影到投影仪靶面的相位值可以被表示为：in Let represent the coordinates of the k-th possible point projected onto the camera c <sub>i</sub> and the projector p<sub>i</sub> target surface, respectively. Let A<sub>c</sub> and A<sub>p</sub> represent the transformation matrices of the i-th camera and projector in the world coordinate system, respectively, and let A <sub>c</sub> and A<sub>p</sub> represent the intrinsic parameter matrices of the camera and projector, respectively. Let these represent the extrinsic parameter matrices of the i-th camera and the projector, respectively. This represents the phase of the k-th possible point of the main camera projected onto the camera target surface in the world coordinate system. The wrapping phase image obtained by Eq(3) at the corresponding rotation angle is used to... Two-dimensional interpolation reveals that phase compensation is required at the transition position. The phase value projected onto the projector target surface can be expressed as: 其中表示的横轴坐标，相机-投影仪极限约束轴为u轴，的范围是[-π，π]，采用可能点对虚拟投影仪和相机投影的包裹相位误差作为判别标准，对可能点进行筛选，误差表示为：in express The horizontal axis is the coordinate of the camera-projector limit constraint, and the U-axis is the coordinate of the camera-projector limit constraint. The range is [-π, π]. The possible points are filtered using the wrap-around phase error between the virtual projector and camera projections as the criterion. The error is expressed as: 其中为第k个可能点在第i个虚拟相机下的包裹相位差，表示相应判别误差，主动做跳变保证相位偏差在[0，π]内。in Let the package phase difference of the k-th possible point under the i-th virtual camera be denoted as . This indicates the corresponding discrimination error, and the phase shift is actively performed to ensure that the phase deviation is within [0, π]. 5.根据权利要求1或2或4所述的基于虚拟立体解包裹方法的实时全景三维重建方法，其特征在于，所述的对包裹相位误差图取和，得到不同k值下的包裹相位误差图，取误差最小的k值，得到最终解包裹k值图，具体为：5. The real-time panoramic 3D reconstruction method based on virtual stereo unwrapping as described in claim 1, 2, or 4, characterized in that, the summation of the wrapping phase error maps to obtain wrapping phase error maps under different k values, and the selection of the k value with the smallest error to obtain the final unwrapping k-value map, specifically comprises: 最终选择的k值为所有虚拟系统的偏差之和最小值，The final chosen value of k is the minimum sum of the deviations of all virtual systems. kresult＝minIndex(Err(k))。k result =minIndex(Err(k)). 其中kresult为VSPU方法得到的k值，Err(k)为不同k值的相位误差，minIndex(·)为最小值的索引函数，m为选择相邻虚拟系统的数量，辅助解包裹的系统数为2m。Where k result is the k value obtained by the VSPU method, Err(k) is the phase error of different k values, minIndex(·) is the index function of the minimum value, m is the number of adjacent virtual systems selected, and the number of systems to assist in unpacking is 2m. 6.根据权利要求5基于虚拟立体解包裹方法的实时全景三维重建方法，其特征在于，所述的单次VSPU过程包括五次投影拍摄多步相移图，每次的延时为θvspu；单个角度的VSPU过程结束后，延时θreg进行下一次VSPU过程，具体为：6. The real-time panoramic 3D reconstruction method based on the virtual stereo unwrapping method according to claim 5, characterized in that the single VSPU process includes five projections to capture multi-step phase-shift images, with a delay of θvspu for each step; after the VSPU process at a single angle is completed, the next VSPU process is performed after a delay of θreg , specifically as follows: 首先转台启动后以恒定角加速度σ旋转加速到匀速ωturn，稳定时间为Δt＝ωturn/α，之后系统开始进行投影拍摄，每次经过treg时间就拍摄一组用于VSPU的条纹图，该时间间隔是通过设定的登记角度间隔Δθreg来得到的Δtreg＝Δθreg/ωturn。First, after the turntable starts, it rotates at a constant angular acceleration σ to a uniform speed ω turn . The settling time is Δt = ω turn / α. Then, the system starts to project images. Each time t reg is elapsed, a set of stripe patterns for VSPU is captured. This time interval is obtained by setting the registered angle interval Δθ reg , where Δt reg = Δθ reg / ω turn .
CN202310195255.7A 2023-03-03 2023-03-03 A Real-time Panoramic 3D Reconstruction Method Based on Virtual Stereo Unwrapping Method Pending CN116485993A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310195255.7A CN116485993A (en) 2023-03-03 2023-03-03 A Real-time Panoramic 3D Reconstruction Method Based on Virtual Stereo Unwrapping Method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310195255.7A CN116485993A (en) 2023-03-03 2023-03-03 A Real-time Panoramic 3D Reconstruction Method Based on Virtual Stereo Unwrapping Method

Publications (1)

Publication Number Publication Date
CN116485993A true CN116485993A (en) 2023-07-25

Family

ID=87216705

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310195255.7A Pending CN116485993A (en) 2023-03-03 2023-03-03 A Real-time Panoramic 3D Reconstruction Method Based on Virtual Stereo Unwrapping Method

Country Status (1)

Country Link
CN (1) CN116485993A (en)

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100188400A1 (en) * 2009-01-23 2010-07-29 National Taipei University Of Technology Method for simultaneous hue phase-shifting and system for 3-d surface profilometry using the same
CN110514143A (en) * 2019-08-09 2019-11-29 南京理工大学 A mirror-based calibration method for fringe projection system

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100188400A1 (en) * 2009-01-23 2010-07-29 National Taipei University Of Technology Method for simultaneous hue phase-shifting and system for 3-d surface profilometry using the same
CN110514143A (en) * 2019-08-09 2019-11-29 南京理工大学 A mirror-based calibration method for fringe projection system

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
LIN BIN ET AL.: "Real-Time 360° 3D shape measurement based on VSPU", 《SSRN ELECTRONIC JOURNAL》, 16 December 2022 (2022-12-16) *

Similar Documents

Publication Publication Date Title
CN108921901B (en) A large field of view camera calibration method based on precision two-axis turntable and laser tracker
CN107063129B (en) A kind of array parallel laser projection three-dimensional scan method
CN103292699B (en) A kind of 3 D scanning system and method
CN110880185A (en) High-precision dynamic real-time 360-degree omnidirectional point cloud acquisition method based on fringe projection
WO2020199439A1 (en) Single- and dual-camera hybrid measurement-based three-dimensional point cloud computing method
CN110793464B (en) Large-field fringe projection vision three-dimensional measurement system and method
CN110849268B (en) A Fast Phase-Altitude Mapping Calibration Method
CN111353997B (en) Real-time three-dimensional surface defect detection method based on fringe projection
Lei et al. A phase-angle inspired calibration strategy based on MEMS projector for 3D reconstruction with markedly reduced calibration images and parameters
CN106595528A (en) Digital speckle-based telecentric microscopic binocular stereoscopic vision measurement method
CN113551617A (en) Binocular and dual-frequency complementary three-dimensional surface measurement method based on fringe projection
CN108036740A (en) Three-dimensional colour measuring system and method during a kind of high-precision real based on various visual angles
CN111811433A (en) Structured light system calibration method, device and application based on red and blue orthogonal stripes
CN113724337B (en) A camera dynamic external parameter calibration method and device that does not depend on the angle of the gimbal
CN117611684A (en) A structural parameter optimization and calibration method for biprism virtual binocular vision system
CN106568394A (en) Hand-held three-dimensional real-time scanning method
CN117073579A (en) Binocular three-dimensional measurement system and method based on stripe projection structured light
CN113298882A (en) Camera calibration device and method for microscopic three-dimensional topography measurement system
Ma et al. Flexible high-resolution continuous 3-D scanning for large-scale industrial components
CN107941147B (en) Non-contact online measurement method for three-dimensional coordinates of large-scale system
Hu et al. Microscopic fringe projection profilometry systems in Scheimpflug condition and performance comparison
Wang et al. High-speed 360° 3D shape measurement based on virtual stereo phase unwrapping method
CN116485993A (en) A Real-time Panoramic 3D Reconstruction Method Based on Virtual Stereo Unwrapping Method
CN118189857A (en) Three-dimensional measurement method and system based on digital image correlation of single camera-projector system
CN202853566U (en) Projective calibration target based optical measurement system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination