CN116182732B - Imaging methods, devices, storage media, and terminals - Google Patents
Imaging methods, devices, storage media, and terminalsInfo
- Publication number
- CN116182732B CN116182732B CN202111435763.5A CN202111435763A CN116182732B CN 116182732 B CN116182732 B CN 116182732B CN 202111435763 A CN202111435763 A CN 202111435763A CN 116182732 B CN116182732 B CN 116182732B
- Authority
- CN
- China
- Prior art keywords
- phase shift
- target
- pixel value
- exposure time
- pixel
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G01B11/00—Measuring arrangements characterised by the use of optical techniques
- G01B11/24—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures
- G01B11/2433—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures for measuring outlines by shadow casting
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Studio Devices (AREA)
- Transforming Light Signals Into Electric Signals (AREA)
Abstract
本申请实施例公开了一种成像方法、装置、存储介质以及终端，所述方法包括：获取相机在至少两个曝光时间下，分别采集的目标在多步相移结构光下对应的各初始图像，针对每步相移，获取所述各初始图像中同一像素点在不同曝光时间下的像素值所组成的相移向量，并对各所述相移向量中的各所述像素值进行线性校正，得到校正后的各目标相移向量，基于所述各目标相移向量在各所述曝光时间中确定目标曝光时间，将所述各目标相移向量中所述目标曝光时间对应的像素值作为目标像素值，由各所述目标像素值组成所述目标在所述每步相移下对应的目标图像。本申请可以有效抑制成像过程中随时间波动的随机噪声，可以提高结构光测量技术中的成像质量。
This application discloses an imaging method, apparatus, storage medium, and terminal. The method includes: acquiring initial images of a target under multi-step phase-shift structured light, captured by a camera at at least two exposure times; for each phase shift, acquiring a phase shift vector composed of pixel values of the same pixel in each initial image at different exposure times; linearly correcting each pixel value in each phase shift vector to obtain corrected target phase shift vectors; determining a target exposure time based on each target phase shift vector in each exposure time; using the pixel value corresponding to the target exposure time in each target phase shift vector as the target pixel value; and composing a target image of the target corresponding to the target under each phase shift by the target pixel values. This application can effectively suppress random noise that fluctuates over time during imaging and can improve the imaging quality in structured light measurement technology.
Description
Technical Field
The present application relates to the field of computer vision, and in particular, to an imaging method, an imaging device, a storage medium, and a terminal.
Background
Currently, in the industries of printed circuit boards, flat panel displays, semiconductors, photovoltaics, automotive electronics, and the like, automatic inspection (Automated Optical Inspection, AOI) technology based on optical imaging and machine vision algorithms is becoming a key means for product quality control in related industries. AOI techniques can be classified into 2D AOI and 3D AOI, depending on the dimension of the data being measured. But 2D AOI is difficult to compete with tasks involving depth information, severely limiting the range of use of the relevant devices. The 3D AOI technology based on structured light imaging can realize a more perfect detection function by accurately measuring depth information, and the structured light 3D measurement technology is widely applied in a plurality of fields by virtue of flexibility, high efficiency and reliability.
Disclosure of Invention
The embodiment of the application provides an imaging method, an imaging device, a computer storage medium and a terminal, which can improve imaging quality in a structured light three-dimensional measurement technology. The technical scheme is as follows:
in a first aspect, an embodiment of the present application provides an imaging method, including:
acquiring initial images corresponding to targets acquired by a camera under multi-step phase shift structural light under at least two exposure times;
for each step of phase shift, obtaining phase shift vectors formed by pixel values of the same pixel point in each initial image under different exposure time, and carrying out linear correction on each pixel value in each phase shift vector to obtain each corrected target phase shift vector;
determining a target exposure time in each of the exposure times based on each of the target phase shift vectors;
and taking the pixel value corresponding to the target exposure time in each target phase shift vector as a target pixel value, and forming a target image corresponding to the target under each step of phase shift by each target pixel value.
In a second aspect, an embodiment of the present application provides an imaging apparatus, including:
The image acquisition module is used for acquiring each initial image of the target, which is acquired by the camera respectively under the condition of at least two exposure times, corresponding to the multi-step phase shift structure light;
The pixel correction module is used for obtaining phase shift vectors formed by pixel values of the same pixel point in each initial image under different exposure time aiming at each step of phase shift, and carrying out linear correction on each pixel value in each phase shift vector to obtain each corrected target phase shift vector;
an exposure determination module for determining a target exposure time in each of the exposure times based on each of the target phase shift vectors;
and the pixel imaging module is used for taking the pixel value corresponding to the target exposure time in each target phase shift vector as a target pixel value, and forming a target image corresponding to the target under each step of phase shift by each target pixel value.
In a third aspect, embodiments of the present application provide a computer storage medium having a plurality of instructions adapted to be loaded by a processor and to perform the above-described method steps.
In a fourth aspect, embodiments of the present application provide a terminal, which may include a memory and a processor, wherein the memory stores a computer program adapted to be loaded by the memory and to perform the above-described method steps.
The technical scheme provided by the embodiment of the application has the beneficial effects that at least:
When the scheme of the embodiment of the application is executed, each initial image of a target, which is acquired by a camera under at least two exposure times, under multi-step phase shift structure light is acquired, a phase shift vector formed by pixel values of the same pixel point in each initial image under different exposure times is acquired for each step of phase shift, each pixel value in each phase shift vector is subjected to linear correction, each corrected target phase shift vector is obtained, the target exposure time is determined in each exposure time based on each target phase shift vector, the pixel value corresponding to the target exposure time in each target phase shift vector is taken as a target pixel value, and each target pixel value forms a target image corresponding to the target under each step of phase shift. According to the application, the fringe images which are acquired by the camera and subjected to target modulation under different exposure time are acquired, and the pixel values in the fringe images are corrected in the time domain, so that the filtering processing of the pixel values in the time domain can be equivalent, random noise which fluctuates along with time can be effectively restrained, then the pixel value corresponding to the proper exposure time is selected for each pixel point to serve as the final pixel value of the pixel point in the image corresponding to each phase shift, and the imaging quality in the structured light measuring technology can be improved.
Drawings
In order to more clearly illustrate the embodiments of the invention or the technical solutions in the prior art, the drawings that are necessary for the description of the embodiments or the prior art will be briefly described, it being obvious that the drawings in the following description are only some embodiments of the invention and that other drawings may be obtained from them without inventive effort for a person skilled in the art.
FIG. 1 is a flow chart diagram of an imaging method provided by an embodiment of the present application;
fig. 2 is a schematic diagram of an application scenario of an imaging method according to an embodiment of the present application;
FIG. 3 is a flow chart of another imaging method according to an embodiment of the present application;
fig. 4 is a schematic structural view of an image forming apparatus according to an embodiment of the present application;
Fig. 5 is a schematic structural diagram of a terminal according to an embodiment of the present application.
Detailed Description
In order to make the objects, features and advantages of the embodiments of the present application more obvious and understandable, the technical solutions of the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is apparent that the described embodiments are only some embodiments of the present application, not all embodiments of the present application. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.
In the description of the present application, it should be understood that the terms "first," "second," and the like are used for descriptive purposes only and are not to be construed as indicating or implying relative importance. In the description of the present application, it should be noted that, unless expressly specified and limited otherwise, "comprise" and "have" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those listed steps or elements but may include other steps or elements not listed or inherent to such process, method, article, or apparatus. The specific meaning of the above terms in the present application will be understood in specific cases by those of ordinary skill in the art. Furthermore, in the description of the present application, unless otherwise indicated, "a plurality" means two or more. "and/or" describes an association relationship of an association object, and indicates that there may be three relationships, for example, a and/or B, and may indicate that there are three cases of a alone, a and B together, and B alone. The character "/" generally indicates that the context-dependent object is an "or" relationship.
In the related art, when the structured light 3D measurement technology is applied to an actual scene (such as an industrial scene), random noise is inevitably introduced in the imaging process due to large reflectivity difference of a target to be measured, and the problem that the pixel points obtained after imaging are oversaturated or underexposed due to the fact that a camera collects images by adopting fixed exposure time is caused, so that imaging quality is low.
The present application will be described in detail with reference to specific examples.
In the following method embodiments, for convenience of explanation, only the execution subject of each step is described as a terminal.
Fig. 1 is a schematic flow chart of an imaging method according to an embodiment of the application. As shown in fig. 1, the method according to the embodiment of the present application may include the following steps:
S101, acquiring initial images of targets acquired by a camera respectively under at least two exposure times and corresponding to the targets under multi-step phase shift structured light.
It can be understood that the application scenario of the embodiment of the present application can be referred to as a scenario diagram shown in fig. 2, and is applied to a structured light three-dimensional measurement technology, where fig. 2 includes a projector, a camera, a target (object to be measured), and a terminal. The terminal programming generates sine stripes, the sine stripes are projected to a target through a projector, then a camera is utilized to shoot the bending degree of the sine stripes modulated by the target, namely, the camera shoots the bending stripes, the terminal demodulates the bending stripes, the phase corresponding to the pixel points on the target can be obtained, and then three-dimensional data corresponding to the target can be obtained through a phase-distance mapping model, namely, the structured light three-dimensional measurement technology.
In a specific embodiment, when applied to the scene shown in fig. 2, the camera may be adjusted to capture the fringe images modulated by the target at a plurality of different exposure times, respectively, while maintaining the structured light mode and the projection intensity of the multi-step phase-shift structured light unchanged, and for convenience of description, the fringe images corresponding to different numbers of steps modulated by the target may be referred to as an initial image. It will be appreciated that the structured light pattern of the multi-step phase-shifted structured light may be kept unchanged, as may the sinusoidal pattern and sinusoidal period of the structured light. It will be appreciated that the plurality of different exposure times of the camera may be set to a set of values of an arithmetic distribution, for example m exposure times, with a number of steps of N, each step of phase shift corresponding to m initial images corresponding to m exposure times. Thus, m×n initial images can be obtained during imaging.
S102, for each phase shift, obtaining a phase shift vector formed by pixel values of the same pixel point in each initial image under different exposure time, and carrying out linear correction on each pixel value in each phase shift vector to obtain each corrected target phase shift vector.
It will be appreciated that, in theory, when the pixel values of the pixel points are not oversaturated, the relationship between the pixel values and the exposure time is linear, and that the pixel values are oversaturated refers to when the pixel values are stored in 8 bits of data, the pixel values greater than 255 are referred to as oversaturated. However, in practical industrial application, due to the large reflectivity difference of the target, random noise possibly introduced in the imaging process, and the exposure time of the camera, the problem of oversaturation or underexposure of the imaged pixel value can be caused, so that a nonlinear relationship is formed between the pixel value and the exposure time.
In a specific embodiment, for example, m exposure times are used, the phase shift step number is N, and for m initial images corresponding to each phase shift step, the pixel value of each pixel point in the m initial images respectively, that is, the pixel value of each pixel point corresponding to m exposure times respectively, may be determined. Taking a pixel as an example, m pixel values corresponding to a pixel may be formed into a phase shift vector, for example, a phase shift vector stored in a column vector form or a phase shift vector stored in a row vector form. In the practical application process, the m pixel values and the m exposure times of a pixel point may be in a nonlinear relationship, so that the m pixel values and the m exposure times may be linearly fitted to obtain a linear straight line corresponding to the pixel point, in the linear straight line, the exposure time may be used as an independent variable, and the pixel values may be used as function values, so that function values corresponding to the m exposure times respectively may be obtained. Further, the m pixel values may be corrected based on the function values corresponding to the m exposure times, that is, the m pixel values are replaced by the m function values according to the exposure times, so that the m function values corresponding to one pixel point may be obtained to form a target phase shift vector.
S103, determining target exposure time in each exposure time based on each target phase shift vector.
It will be appreciated that for one pixel, the target exposure time for that pixel may be determined, and for different pixels, there may be different target exposure times. For determining the target exposure time corresponding to each pixel point, the pixel value, which is not oversaturated and has the largest value, of n target phase shift vectors of the same pixel point can be found, further, the exposure time corresponding to each of the n pixel values can be determined, and the minimum one of the exposure times can be taken as the target exposure time.
S104, taking the pixel value corresponding to the target exposure time in each target phase shift vector as a target pixel value, and forming a target image corresponding to the target under each step of phase shift by each target pixel value.
It will be appreciated that for a pixel point that corresponds to a target exposure time, then the target pixel value for each phase shift for that pixel point at the target exposure time can be determined. Obviously, for the n-th phase shift, all pixel points have a target pixel value corresponding to a target exposure time, and then the target pixel value can form a target image corresponding to the n-th phase shift. That is, in one target image, the pixel values of all the pixel points are target pixel values of the pixel points at the target exposure time.
When the scheme of the embodiment of the application is executed, each initial image of a target, which is acquired by a camera under at least two exposure times, under multi-step phase shift structure light is acquired, a phase shift vector formed by pixel values of the same pixel point in each initial image under different exposure times is acquired for each step of phase shift, each pixel value in each phase shift vector is subjected to linear correction, each corrected target phase shift vector is obtained, the target exposure time is determined in each exposure time based on each target phase shift vector, the pixel value corresponding to the target exposure time in each target phase shift vector is taken as a target pixel value, and each target pixel value forms a target image corresponding to the target under each step of phase shift. According to the application, the fringe images which are acquired by the camera and subjected to target modulation under different exposure time are acquired, and the pixel values in the fringe images are corrected in the time domain, so that the filtering processing of the pixel values in the time domain can be equivalent, random noise which fluctuates along with time can be effectively restrained, then the pixel value corresponding to the proper exposure time is selected for each pixel point to serve as the final pixel value of the pixel point in the image corresponding to each phase shift, and the imaging quality in the structured light measuring technology can be improved.
Fig. 3 is a schematic flow chart of an imaging method according to an embodiment of the application. As shown in fig. 3, the method according to the embodiment of the present application may include the following steps:
s301, acquiring initial images of targets acquired by a camera respectively under at least two exposure times and corresponding to the targets under multi-step phase shift structured light.
Specifically, reference may be made to S101 in fig. 1, which is not described herein.
S302, for each phase shift, a phase shift vector composed of pixel values of the same pixel point in each initial image under different exposure time is obtained.
In a specific embodiment, assuming m exposure times and N-step phase shifts, the exposure times are noted as t k (k=0, 1, 2., m), for the nth (n=1, 2,3, the combination of N) step phase shift, an explanation will be given by taking one pixel point (i, j) in the initial image as an example. The pixel value corresponding to the pixel point (i, j) when the exposure time is t k is recorded asRespectively taking pixel values corresponding to pixel points (i, j) in m initial imagesA column vector V i,j is formed, and V i,j may be referred to as a phase shift vector. Then N V i,j can be obtained.
S303, based on different exposure time in each phase shift vector and pixel values corresponding to the different exposure time, obtaining a fitting straight line taking the exposure time as an independent variable and the pixel value as the dependent variable.
In a specific embodiment, for the calculation of the fitting line, the difference between adjacent exposures may be denoted as δ, t k-tk-1 =δ, and for the nth phase shift, the difference between the pixel values corresponding to the exposure time t k and the exposure time t k-1, respectively, may be denoted asIn theory, however, as long as the pixel values are not oversaturatedI MAX represents the maximum possible gray value of the camera sample (I MAX is 255 when the gray value is stored in 8 bits of data), the pixel value and the exposure time are linear, when k takes different values,In the practical application process, due to factors such as random noise, when k takes different values,The values of (c) randomly fluctuate around the exact value. Further, a least square method can be adopted to fit the pixel value and the exposure time in the vector V i,j to obtain a linear straight line
S304, initial pixel values corresponding to the different exposure times in the phase shift vectors are obtained, and corrected pixel values of the exposure times in the fitting straight line are obtained.
S305, replacing the initial pixel values corresponding to the exposure time with the correction pixel values one by one, and forming a target phase shift vector corresponding to each phase shift by the correction pixel values.
S304 and S305 are explained below.
In a specific embodiment, taking pixel (i, j) as an example, pixel values corresponding to m exposure times in vector V i,j can be obtained, let us note that t k (k=1, 2,., m) can be substituted into a linear straight line as the initial pixel valueThe corrected pixel value corresponding to t k is obtained, the initial pixel value corresponding to t k can be replaced by the corrected pixel value corresponding to t k, which is the process of correcting the initial pixel value, and the vector formed by the corrected pixel values corresponding to different exposure times obtained after correction can be used as the target phase shift vector.
S306, respectively selecting the pixel value with the largest unsaturated value from each target phase shift vector as each first pixel value, and respectively selecting the pixel value corresponding to the minimum exposure time from each target phase shift vector as each second pixel value.
In a specific embodiment, the pixel value that is not oversaturated and has the greatest value may be selected from the target phase shift vector, and may be referred to asCan be used forReferred to as a first pixel value, the first pixel value in each target phase shift vector can be found to obtainForming an N-dimensional row vectorIn addition, the pixel value corresponding to t 0 (minimum exposure time) can be selected from each target phase shift vector and can be recorded asCan be used forReferred to as a second pixel value, resulting in Can be combined into an N-dimensional row vector
S307, calculating the difference value between the first pixel value and the second pixel value corresponding to each phase shift, and obtaining the time sequence number corresponding to each first pixel value.
In a specific embodiment, the method can be based on a linear straight lineIs calculated to be satisfiedIs denoted as Δi n. For the n-th phase shift, the formula can beCalculated to obtainDue toIt is known that,Knowing ΔI n, one can calculateWherein, the Representation ofThe corresponding sequence number of the exposure time, i.e. the corresponding sequence number of the first pixel value.
S308, for each time sequence number, calculating the product of the value of each time sequence number and the difference value of the adjacent exposure time, calculating the sum value of the product and the minimum exposure time, and taking the sum value as the first exposure time corresponding to the first pixel value.
In a specific embodiment, can be made ofThe calculated t k,n,tk,n represents the first exposure time corresponding to the first pixel value, δ represents the difference between adjacent exposure times, the value of δ is a fixed value, t 0,n represents the minimum exposure time, and the value of t 0,n is a fixed value.
S309, based on the corresponding relation between the different time sequence numbers and the different exposure times, obtaining the first exposure time corresponding to each first pixel value.
In a specific embodiment, it is possible to provideAnd the correspondence between the exposure times,Representation ofThe corresponding sequence number of the exposure time can be searched in the corresponding relationThe corresponding exposure time is the first exposure time corresponding to the first pixel value.
S310, taking the minimum value in each first exposure time as a target exposure time, taking the pixel value corresponding to the target exposure time in each target phase shift vector as a target pixel value, and forming a target image corresponding to the target under each phase shift by each target pixel value.
In a specific embodiment, the minimum value may be selected from the first exposure times, and the minimum first exposure time is taken as the target exposure time, and the value of the target exposure time may be different for each pixel point. Further, for the pixel point (i, j), a target pixel value corresponding to the target exposure time may be determined from the n target phase shift vectors, respectively. Obviously, for each pixel point, its corresponding target pixel value at the n-th phase shift can be determined. Therefore, for the pixel values of all the pixel points in the target image corresponding to the phase shift of the nth step, the pixel values of all the pixel points corresponding to the nth step may be target pixel values of each of the pixel points.
When the scheme of the embodiment of the application is executed, each initial image of a target, which is acquired by a camera under at least two exposure times, under multi-step phase shift structure light is acquired, a phase shift vector formed by pixel values of the same pixel point in each initial image under different exposure times is acquired for each step of phase shift, each pixel value in each phase shift vector is subjected to linear correction, each corrected target phase shift vector is obtained, the target exposure time is determined in each exposure time based on each target phase shift vector, the pixel value corresponding to the target exposure time in each target phase shift vector is taken as a target pixel value, and each target pixel value forms a target image corresponding to the target under each step of phase shift. According to the application, the fringe images which are acquired by the camera and are modulated by the targets under different exposure time are acquired, and the pixel values in the fringe images are corrected in the time domain, so that the filtering processing of the pixel values in the time domain can be equivalent, random noise which fluctuates along with time can be effectively restrained, then the pixel value corresponding to the proper exposure time is selected for each pixel point to serve as the final pixel value of the pixel point in the image corresponding to each phase shift, the imaging quality in the structured light measuring technology can be improved, and the accuracy of three-dimensional data corresponding to the targets can be further improved. In addition, the application selects the pixel value corresponding to the most suitable exposure time for each pixel point, and can meet the condition that the contrast between the pixel values corresponding to the asynchronous number is as large as possible while the pixel value is oversaturated.
Fig. 4 is a schematic structural diagram of an imaging device according to an embodiment of the application. The imaging apparatus 400 may be implemented as all or a portion of a terminal by software, hardware, or a combination of both.
The apparatus 400 comprises:
an image acquisition module 410, configured to acquire initial images of the target, which are acquired by the camera under the multi-step phase shift structure light, under at least two exposure times;
The pixel correction module 420 is configured to obtain, for each phase shift, a phase shift vector formed by pixel values of the same pixel point in each initial image under different exposure times, and perform linear correction on each pixel value in each phase shift vector, so as to obtain each corrected target phase shift vector;
an exposure determination module 430, configured to determine a target exposure time in each of the exposure times based on each of the target phase shift vectors;
the pixel imaging module 440 is configured to take a pixel value corresponding to the target exposure time in each target phase shift vector as a target pixel value, and form a target image corresponding to the target under each phase shift by using each target pixel value.
Optionally, the pixel correction module 420 includes:
the fitting unit is used for obtaining fitting straight lines corresponding to the phase shift vectors respectively;
and the correction unit is used for respectively correcting pixel values corresponding to different exposure times in the phase shift vectors based on the fitting straight lines to obtain corrected target phase shift vectors.
Optionally, the fitting unit includes:
And the fitting subunit is used for obtaining a fitting straight line taking the exposure time as an independent variable and the pixel value as a dependent variable based on different exposure times in the phase shift vectors and the pixel values respectively corresponding to the different exposure times.
Optionally, the fitting subunit includes:
the first fitting unit is used for obtaining initial pixel values corresponding to the different exposure times in the phase shift vectors respectively and obtaining corrected pixel values of the exposure times in the fitting straight line;
And the second fitting subunit is used for respectively replacing the initial pixel values corresponding to the exposure time with the correction pixel values in a one-to-one correspondence manner, and forming each target phase shift vector corresponding to each phase shift step by the correction pixel values.
Optionally, the exposure determination module 430 includes:
The first determining unit is used for respectively selecting an unsaturated pixel value with the largest value from the target phase shift vectors as each first pixel value, and respectively selecting a pixel value corresponding to the minimum exposure time from the target phase shift vectors as each second pixel value;
the second determining unit is used for obtaining first exposure time corresponding to each first pixel value respectively based on each first pixel value and each second pixel value;
And a third determining unit configured to take a minimum value in each of the first exposure times as a target exposure time.
Optionally, the second determining unit includes:
The first calculating unit is used for calculating the difference value between the first pixel value and the second pixel value corresponding to each step of phase shift to obtain the time sequence number corresponding to each first pixel value;
and the second calculation unit is used for obtaining the first exposure time corresponding to each first pixel value respectively based on each time sequence number.
Optionally, the second computing unit includes:
A first time determination subunit for calculating, for each time sequence number, a product between a value of the each time sequence number and a difference value of adjacent exposure times, and calculating a sum of the product and the minimum exposure time, the sum being taken as a first exposure time corresponding to the first pixel value, or,
And the second time determining subunit is used for obtaining the first exposure time corresponding to each first pixel value respectively based on the corresponding relation between different time sequence numbers and different exposure times.
When the scheme of the embodiment of the application is executed, each initial image of a target, which is acquired by a camera under at least two exposure times, under multi-step phase shift structure light is acquired, a phase shift vector formed by pixel values of the same pixel point in each initial image under different exposure times is acquired for each step of phase shift, each pixel value in each phase shift vector is subjected to linear correction, each corrected target phase shift vector is obtained, the target exposure time is determined in each exposure time based on each target phase shift vector, the pixel value corresponding to the target exposure time in each target phase shift vector is taken as a target pixel value, and each target pixel value forms a target image corresponding to the target under each step of phase shift. According to the application, the fringe images which are acquired by the camera and subjected to target modulation under different exposure time are acquired, and the pixel values in the fringe images are corrected in the time domain, so that the filtering processing of the pixel values in the time domain can be equivalent, random noise which fluctuates along with time can be effectively restrained, then the pixel value corresponding to the proper exposure time is selected for each pixel point to serve as the final pixel value of the pixel point in the image corresponding to each phase shift, and the imaging quality in the structured light measuring technology can be improved.
Referring to fig. 5, fig. 5 is a schematic structural diagram of a terminal according to an embodiment of the present application. As shown in fig. 5, terminal 1300 may include at least one processor 1301, at least one network interface 1304, a user interface 1303, a memory 1305, and at least one communication bus 1302.
Wherein a communication bus 1302 is used to enable connected communications between these components.
The user interface 1303 may include a Display screen (Display) and a Camera (Camera), and the optional user interface 1303 may further include a standard wired interface and a standard wireless interface.
The network interface 1304 may optionally include a standard wired interface, a wireless interface (e.g., WI-FI interface), among others.
Processor 1301 may include one or more processing cores, among other things. The processor 1301 connects various parts within the entire terminal 1300 using various interfaces and lines, and performs various functions of the terminal 1300 and processes data by executing or executing instructions, programs, code sets, or instruction sets stored in the memory 1305, and calling data stored in the memory 1305. Alternatively, the processor 1301 may be implemented in at least one hardware form of digital signal Processing (DIGITAL SIGNAL Processing, DSP), field-Programmable gate array (Field-Programmable GATE ARRAY, FPGA), programmable logic array (Programmable Logic Array, PLA). Processor 1301 may integrate one or a combination of several of a central processing unit (Central Processing Unit, CPU) and a modem, etc. Wherein the CPU primarily processes operating systems and application programs, etc. It will be appreciated that the modem may not be integrated into the processor 1301 and may be implemented by a single chip.
The Memory 1305 may include a random access Memory (Random Access Memory, RAM) or a Read-Only Memory (RAM). Optionally, the memory 1305 includes a non-transitory computer readable medium (non-transitory computer-readable storage medium). The memory 1305 may be used to store instructions, programs, code sets, or instruction sets. The memory 1305 may include a stored program area that may store instructions for implementing an operating system, instructions for at least one function (e.g., a touch function, a sound playing function, an image playing function, etc.), instructions for implementing the various method embodiments described above, etc., and a stored data area that may store data related to the various method embodiments described above, etc. Memory 1305 may also optionally be at least one storage device located remotely from the aforementioned processor 1301. As shown in fig. 5, a program of an operating system, a network communication module, a user interface module, and an imaging method may be included in the memory 1305 as one type of computer storage medium.
In the terminal 1300 shown in fig. 5, the user interface 1303 is mainly used for providing an input interface for a user to acquire data input by the user, and the processor 1301 may be used for calling a program of an imaging method stored in the memory 1305 and specifically performing the following operations:
acquiring initial images corresponding to targets acquired by a camera under multi-step phase shift structural light under at least two exposure times;
for each step of phase shift, obtaining phase shift vectors formed by pixel values of the same pixel point in each initial image under different exposure time, and carrying out linear correction on each pixel value in each phase shift vector to obtain each corrected target phase shift vector;
determining a target exposure time in each of the exposure times based on each of the target phase shift vectors;
and taking the pixel value corresponding to the target exposure time in each target phase shift vector as a target pixel value, and forming a target image corresponding to the target under each step of phase shift by each target pixel value.
In one embodiment, when performing the step of linearly correcting each pixel value in each phase shift vector to obtain each corrected target phase shift vector, the processor 1301 specifically performs the following operations:
obtaining fitting straight lines corresponding to the phase shift vectors respectively;
And correcting pixel values corresponding to different exposure times in the phase shift vectors based on the fitting straight lines respectively to obtain corrected target phase shift vectors.
In one embodiment, when executing the step of obtaining the fitting straight line corresponding to each phase shift vector, the processor 1301 specifically executes the following operations:
and obtaining a fitting straight line taking the exposure time as an independent variable and the pixel value as the dependent variable based on different exposure time in each phase shift vector and the pixel value respectively corresponding to the different exposure time.
In one embodiment, when the step of correcting the pixel values corresponding to different exposure times in each of the phase shift vectors based on each of the fitting straight lines to obtain corrected target phase shift vectors is performed by the processor 1301, the following operations are specifically performed:
Acquiring initial pixel values corresponding to the different exposure times in the phase shift vectors respectively, and acquiring corrected pixel values of the exposure times in the fitting straight line;
And respectively replacing the initial pixel values corresponding to the exposure time with the correction pixel values in a one-to-one correspondence manner, and forming each target phase shift vector corresponding to each phase shift by the correction pixel values.
In one embodiment, the processor 1301, when executing the step of determining the target exposure time in each of the exposure times based on the target phase shift vectors, specifically executes the following operations:
Respectively selecting an unsaturated pixel value with the largest value from the target phase shift vectors as each first pixel value, and respectively selecting a pixel value corresponding to the minimum exposure time from the target phase shift vectors as each second pixel value;
obtaining first exposure time corresponding to each first pixel value based on each first pixel value and each second pixel value;
and taking the minimum value in each first exposure time as a target exposure time.
In one embodiment, when executing the step of obtaining the first exposure time corresponding to each first pixel value based on each first pixel value and each second pixel value, the processor 1301 specifically executes the following operations:
calculating the difference value between the first pixel value and the second pixel value corresponding to each phase shift step to obtain the time sequence number corresponding to each first pixel value;
and obtaining the first exposure time corresponding to each first pixel value based on each time sequence number.
In one embodiment, when executing the step of obtaining the first exposure time corresponding to each first pixel value based on each time sequence number, the processor 1301 specifically executes the following operations:
for each time sequence number, calculating the product between the value of each time sequence number and the difference between adjacent exposure times, and calculating the sum of the product and the minimum exposure time, taking the sum as the first exposure time corresponding to the first pixel value, or,
And obtaining the first exposure time corresponding to each first pixel value respectively based on the corresponding relation between the different time sequence numbers and the different exposure times.
In addition, those skilled in the art will appreciate that the configuration of the terminal 1300 illustrated in the above-described figures does not constitute a limitation of the terminal 1300, and the terminal may include more or less components than illustrated, or may combine certain components, or may have a different arrangement of components. For example, the terminal 1300 further includes radio frequency circuits, audio circuits, wiFi components, power supplies, bluetooth components, and the like, which are not described herein.
Embodiments of the present application also provide a computer-readable storage medium storing at least one instruction for execution by a processor to implement the imaging method described in the above embodiments.
Embodiments of the present application also provide a computer program product storing at least one instruction that is loaded and executed by the processor to implement the imaging method as described in the various embodiments above.
Those skilled in the art will appreciate that in one or more of the examples described above, the functions described in the embodiments of the present application may be implemented in hardware, software, firmware, or any combination thereof. When implemented in software, these functions may be stored on or transmitted over as one or more instructions or code on a computer-readable medium. Computer-readable media includes both computer storage media and communication media including any medium that facilitates transfer of a computer program from one place to another. A storage media may be any available media that can be accessed by a general purpose or special purpose computer.
The foregoing description of the preferred embodiments of the present application is not intended to limit the application, but rather, the application is to be construed as limited to the appended claims.
Claims (9)
1. A method of imaging, the method comprising:
acquiring initial images corresponding to targets acquired by a camera under multi-step phase shift structural light under at least two exposure times;
for each step of phase shift, obtaining phase shift vectors formed by pixel values of the same pixel point in each initial image under different exposure time, and carrying out linear correction on each pixel value in each phase shift vector to obtain each corrected target phase shift vector;
Respectively selecting an unsaturated pixel value with the largest value from the target phase shift vectors as each first pixel value, respectively selecting a pixel value corresponding to the minimum exposure time from the target phase shift vectors as each second pixel value, and obtaining first exposure time respectively corresponding to each first pixel value based on each first pixel value and each second pixel value, and taking the minimum value in each first exposure time as the target exposure time;
and taking the pixel value corresponding to the target exposure time in each target phase shift vector as a target pixel value, and forming a target image corresponding to the target under each step of phase shift by each target pixel value.
2. The method of claim 1, wherein the linearly correcting each of the pixel values in each of the phase shift vectors to obtain corrected target phase shift vectors comprises:
obtaining fitting straight lines corresponding to the phase shift vectors respectively;
And correcting pixel values corresponding to different exposure times in the phase shift vectors based on the fitting straight lines respectively to obtain corrected target phase shift vectors.
3. The method according to claim 2, wherein the obtaining a fitting line corresponding to each of the phase shift vectors includes:
and obtaining a fitting straight line taking the exposure time as an independent variable and the pixel value as the dependent variable based on different exposure time in each phase shift vector and the pixel value respectively corresponding to the different exposure time.
4. A method according to claim 2 or 3, wherein said correcting pixel values corresponding to different exposure times in each of said phase shift vectors based on each of said fitted lines, respectively, to obtain corrected target phase shift vectors, comprises:
Acquiring initial pixel values corresponding to the different exposure times in the phase shift vectors respectively, and acquiring corrected pixel values of the exposure times in the fitting straight line;
And respectively replacing the initial pixel values corresponding to the exposure time with the correction pixel values in a one-to-one correspondence manner, and forming a target phase shift vector corresponding to each phase shift by the correction pixel values.
5. The method according to claim 1, wherein obtaining the first exposure time respectively corresponding to each first pixel value based on each first pixel value and each second pixel value includes:
calculating the difference value between the first pixel value and the second pixel value corresponding to each phase shift step to obtain the time sequence number corresponding to each first pixel value;
and obtaining the first exposure time corresponding to each first pixel value based on each time sequence number.
6. The method of claim 5, wherein obtaining the first exposure time respectively corresponding to each of the first pixel values based on each of the time sequence numbers comprises:
for each time sequence number, calculating the product between the value of each time sequence number and the difference between adjacent exposure times, and calculating the sum of the product and the minimum exposure time, taking the sum as the first exposure time corresponding to the first pixel value, or,
And obtaining the first exposure time corresponding to each first pixel value respectively based on the corresponding relation between the different time sequence numbers and the different exposure times.
7. An imaging apparatus, the apparatus comprising:
The image acquisition module is used for acquiring each initial image of the target, which is acquired by the camera respectively under the condition of at least two exposure times, corresponding to the multi-step phase shift structure light;
The pixel correction module is used for obtaining phase shift vectors formed by pixel values of the same pixel point in each initial image under different exposure time aiming at each step of phase shift, and carrying out linear correction on each pixel value in each phase shift vector to obtain each corrected target phase shift vector;
The exposure determining module is used for respectively selecting an unsaturated pixel value with the largest value from the target phase shift vectors as each first pixel value, respectively selecting a pixel value corresponding to the minimum exposure time from the target phase shift vectors as each second pixel value, and obtaining first exposure time corresponding to each first pixel value based on each first pixel value and each second pixel value and taking the minimum value in each first exposure time as the target exposure time, wherein the first pixel value is the pixel value of the pixel point in the initial image acquired under the first exposure time;
and the pixel imaging module is used for taking the pixel value corresponding to the target exposure time in each target phase shift vector as a target pixel value, and forming a target image corresponding to the target under each step of phase shift by each target pixel value.
8. A computer storage medium storing a plurality of instructions adapted to be loaded by a processor and to perform the method steps of any one of claims 1 to 6.
9. A terminal comprising a processor and a memory, wherein the memory stores a computer program adapted to be loaded by the processor and to perform the method steps of any of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111435763.5A CN116182732B (en) | 2021-11-29 | 2021-11-29 | Imaging methods, devices, storage media, and terminals |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111435763.5A CN116182732B (en) | 2021-11-29 | 2021-11-29 | Imaging methods, devices, storage media, and terminals |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116182732A CN116182732A (en) | 2023-05-30 |
| CN116182732B true CN116182732B (en) | 2026-03-27 |
Family
ID=86444761
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111435763.5A Active CN116182732B (en) | 2021-11-29 | 2021-11-29 | Imaging methods, devices, storage media, and terminals |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116182732B (en) |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2010057142A (en) * | 2008-08-29 | 2010-03-11 | Sharp Corp | Calibration device, calibration method, sensor, display device, calibration processing program and recording medium recording the calibration processing program |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11509880B2 (en) * | 2012-11-14 | 2022-11-22 | Qualcomm Incorporated | Dynamic adjustment of light source power in structured light active depth sensing systems |
| US9554057B2 (en) * | 2013-07-16 | 2017-01-24 | Texas Instruments Incorporated | Wide dynamic range depth imaging |
| US10785463B2 (en) * | 2013-07-16 | 2020-09-22 | Texas Instruments Incorporated | Super-resolution in structured light imaging |
| CN107592472B (en) * | 2017-10-20 | 2019-12-20 | 思特威(上海)电子科技有限公司 | Imaging correction method of multi-exposure image sensor |
| CN108827184B (en) * | 2018-04-28 | 2020-04-28 | 南京航空航天大学 | Structured light self-adaptive three-dimensional measurement method based on camera response curve |
-
2021
- 2021-11-29 CN CN202111435763.5A patent/CN116182732B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2010057142A (en) * | 2008-08-29 | 2010-03-11 | Sharp Corp | Calibration device, calibration method, sensor, display device, calibration processing program and recording medium recording the calibration processing program |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116182732A (en) | 2023-05-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108932698B (en) | Image distortion correction method, device, electronic equipment and storage medium | |
| US8422827B2 (en) | Image correction apparatus and image correction method | |
| JP6598673B2 (en) | Data processing apparatus and method | |
| CN114125411B (en) | Projection device correction method, projection device correction device, storage medium and projection device | |
| CN112669429A (en) | Image distortion rendering method and device | |
| US9451165B2 (en) | Image processing apparatus | |
| CN110728644B (en) | Image generation method and device, electronic equipment and readable storage medium | |
| JP5259291B2 (en) | White balance correction device and signal processing program | |
| US20150187051A1 (en) | Method and apparatus for estimating image noise | |
| US8737758B2 (en) | Apparatus and method of reducing noise | |
| JP6395455B2 (en) | Inspection device, inspection method, and program | |
| JP5391053B2 (en) | 3D shape measuring method and 3D shape measuring apparatus | |
| CN116182732B (en) | Imaging methods, devices, storage media, and terminals | |
| US20180204351A1 (en) | Image processing apparatus and image processing method | |
| CN104215215A (en) | Ranging method | |
| JP2015076754A5 (en) | ||
| JP6532328B2 (en) | Image processing apparatus, control method therefor, and control program | |
| US20140152862A1 (en) | Image processing apparatus, image pickup apparatus, image pickup system, image processing method, and non-transitory computer-readable storage medium | |
| CN109523481B (en) | Method, device and computer-readable storage medium for correcting image distortion of projector | |
| JP2017156311A (en) | Three-dimensional measurement device, three-dimensional measurement system, three-dimensional measurement method, and program | |
| JPWO2018051492A1 (en) | Optical flow accuracy calculation apparatus and optical flow accuracy calculation method | |
| JP2010177832A (en) | Image processing apparatus and image processing method | |
| JP2016191773A (en) | Projection system, projector device, imaging apparatus and program | |
| JP6468751B2 (en) | Image processing apparatus, image processing method, and program | |
| CN116147530B (en) | A surface imaging method, storage medium, and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant |