CN108254295B - A method and device for locating and characterizing spherical particles - Google Patents

Abstract

The invention discloses a method and a device for positioning and representing spherical particles, wherein the method comprises the steps of S1 collecting a dark field image, S2 collecting a bright field image under uniform irradiation of a light source, S3 placing a solution sample containing the spherical particles above a sensor, collecting a hologram sequence and simultaneously adjusting the exposure time of a camera, S4 estimating an image of a sensor plane under a blank field to be used as a background image, S5 carrying out flat field correction on the shot hologram and the background image, S6 obtaining a holographic image formed by the normalized light source irradiation sample, and S7 fitting the holographic image to an expression described by a scattering function and an incident light field to realize high-precision three-dimensional positioning on the shot particles and accurately represent the size and refractive index information of the particles.

Description

A method and device for locating and characterizing spherical particles

technical field

The invention belongs to the field of lensless microscopy, and in particular relates to a method and device for positioning and characterizing spherical particles.

Background technique

Numerous existing and emerging applications will benefit from localizing, characterizing or tracking particle motion, such as localization and tracking of colloidal spheres, nanorods, protein aggregates, etc. in the fields of biomedicine, fluid mechanics and soft matter, water quality testing Characterization of contaminants, etc.

Previous studies on particle tracking and characterization were based on standard inverted optical microscopes, using collimated, attenuated HeNe lasers to replace traditional incandescent illuminators and concentrators. A conventional eyepiece was used to magnify the interference pattern, and a grayscale camera was used to record the hologram. However, this technique suffers from a trade-off between field-of-view (FOV) and imaging resolution. The high magnification of standard optical microscopes represents a smaller field of view, thus hindering their application in situations where a large field of view is required to locate, identify, and track multiple particles. To obtain images with high resolution and a large field of view, mechanical scanning and stitching are required to enlarge the limited field of view, which not only complicates the imaging process but also significantly increases the overall cost of these systems. Even so, the temporal resolution still suffers.

Lensless holographic microscopy has emerged as a new imaging technique in recent years. In contrast to conventional lens-based holographic microscopes, lensless holographic microscopes directly sample the light passing through the object without using any imaging lenses between the object and the sensor plane, so the spatial bandwidth product is no longer related to spatial resolution. Lensless holographic microscopes with single magnification (where the sample is so close to the sensor plane that there is almost no magnification) have a field of view the same size as the imaging sensor, without the need for any lenses and other intermediate optics. This allows the imaging device to be further simplified, while effectively avoiding optical aberrations and dispersion that are unavoidable in conventional lens-based imaging systems. Furthermore, the entire system is compact and cost-effective, providing a possible solution for simultaneous localization and characterization of multiple particles in a large field of view in resource-constrained environments.

In the current study, the angular spectrum method and the autofocus algorithm are used for the image captured by the lensless holographic microscope to realize the rough localization of the sample, and the rough estimation of the sample size is realized according to the image back-propagated to the focal plane. This method can only achieve rough positioning and size estimation of the sample, and cannot obtain relevant information about the composition of the sample.

SUMMARY OF THE INVENTION

In view of the above defects in the prior art, the purpose of the present invention is to provide a method for positioning and characterizing spherical particles, which can accurately locate and characterize each spherical particle in a large field of view, and obtain the size and refraction of each particle. rate information. Another object of the present invention is to provide an apparatus for implementing the method.

The technical solution that realizes the object of the present invention is:

A method for positioning and characterizing spherical particles, comprising the following steps:

S1, turn off the light source, and collect dark field images;

S2, turn on the light source, and collect a brightfield image under the uniform illumination of the light source;

S3, place a solution sample containing spherical particles above the sensor, and ensure that the distance from the sample to the sensor is much smaller than the distance from the sample to the light source; turn on the light source, collect the hologram sequence at a certain time interval, and adjust the camera exposure time at the same time to avoid particles without particles. Blurring of interference patterns caused by regular Brownian motion;

S4, estimating the image of the sensor plane under the empty field of view as a background image, that is, an image formed by incident light irradiating a solution without particles and propagating to the sensor plane;

S5, performing flat field correction on the hologram in step S3 and the background image estimated in step S4;

S6, normalizing the flat-field-corrected hologram according to the flat-field-corrected background image to obtain a holographic image formed by illuminating the sample with the normalized light source;

S7, fit the holographic image obtained in step S6 to the expression described by the scattering function and the incident light field, realize high-precision three-dimensional positioning of the captured particles, and accurately characterize the size and refractive index information of the particles at the same time.

The present invention is a device for positioning and characterizing spherical particles, comprising a coherent light source, an optical fiber, a small hole and an image acquisition system. The sample is placed above the sensor plane of the image acquisition system, and the distance between the sample and the light source is much greater than the distance between the sample and the sensor plane. distance; the coherent light source is coupled into the optical fiber and incident on the sample through the small hole, and the sample scatters the incident light; on the sensor plane of the image acquisition system, the incident light and the scattered light interfere, and the image acquisition system records and outputs the interference pattern, that is, the hologram .

In the prior art, the system using a lens has a very small field of view, and the observed particles are very limited, and it is impossible to observe and locate the particles in a large field of view; while the method of using a lensless system for positioning, its positioning accuracy and size estimation The accuracy is very low, and the refractive index cannot be calculated. The significant advantages of the present invention are: the lensless holographic microscopy method of the present invention can meet the requirements of large field of view and high resolution at the same time, and the hologram captured by the lensless microscopic device can be combined with the Lorenz-Mie theory, and the The spherical particles in the consistent size field of view of the chip enable high-precision positioning at the sub-micron level, and can obtain precise size and refractive index information. The method of the present invention is applicable to microparticles in the entire range of particle size and composition to which the Lorenz-Mie theory applies.

Description of drawings

Figure 1 is a flow chart of the method of the present invention for locating and characterizing spherical particles.

2 is a schematic structural diagram of the device for positioning and characterizing spherical particles according to the present invention, wherein 1-coherent light source, 2-optical fiber, 3-small hole, 4-sample, 5-image acquisition system, 6-sensor.

FIG. 3 is a large field of view hologram photographed by the device proposed by the present invention.

Fig. 4 is the light intensity distribution diagram that adopts the method proposed by the present invention to propagate back to the height of the particle, (a) uses the angular spectrum method and the autofocus algorithm to detect the height of the particle center; (b) the light of the height plane of the particle center Strong distribution map.

5 is a fitting result diagram of the intercepted single-particle and multi-particle holograms using the method proposed by the present invention, (a) is the intercepted single-particle hologram, (b) is a fitting result diagram for (a) diagram, (c) is an intercepted multiparticulate hologram, and (d) is a fitting result of (c).

Fig. 6 is the result of using the method proposed by the present invention to locate and characterize the particles of a certain region of interest in the hologram, (a) is the intercepted particle hologram of the region of interest, (b) is the image of (a) Fitting result graph, (c) is the localization and characterization result of all particles in (a).

Detailed ways

Referring to FIG. 1 , the method of the present invention for locating and characterizing spherical particles can precisely locate the three-dimensional position of the particles with sub-micron precision in the large field-of-view hologram of the sample containing spherical particles captured by a lensless holographic microscope device, and To characterize particle size and refractive index information, the specific steps are as follows:

S1: Turn off the light source, and use the image acquisition system 5 to capture a dark field image under dark room conditions (without ambient stray light). The lensless holographic microscope device for capturing images is shown in FIG. 2 , including a coherent light source 1 , an image acquisition system 5 , and the like. The coherence of the incident light can be enhanced through the small hole 3 , and the field of view is consistent with the chip size of the sensor 6 of the image acquisition system 5 . The illumination range of the light source covers the entire sensor plane of the image acquisition system 5 .

S2: Turn on the light source, and collect a brightfield image under the uniform illumination of the light source under dark room conditions (without ambient stray light).

S3: Place the sample 4 (a solution containing particles) above the sensor 6. It can be dropped directly on the silicon dioxide on the surface of the sensor 6, or it can be dropped on a clean glass sheet and then fixed on the sensor 6. The distance from the sample 4 to the sensor 6 is much smaller than the distance from the sample 4 to the light source 1. On the one hand, the incident wave propagating from the sample 4 to the plane of the sensor 6 can be regarded as a plane wave. On the other hand, the single lensless holographic microscope is guaranteed. The multiple magnification (ie, there is basically no magnification for the sample 4), provides a large field of view (FOV, field-of-view,) consistent with the size of the chip, and does not require any other optical elements. Turn on the light source 1, the linearly polarized laser beam is incident on the plane of the sample 4, the sample 4 scatters the incident light, and the incident light and the scattered light on the plane of the sensor 6 interfere, and the image acquisition system 5 records and outputs the interference pattern, that is, the hologram. Manually shoot and save or develop the camera to automatically shoot and save pictures, and collect hologram sequences at certain time intervals (the time interval is as short as possible, no equal time interval is required, and the number of sequences should be as much as 15 or more). Irregular Brownian motion, while adjusting the camera exposure time to ensure that the image brightness is moderate, try to avoid the blurring of the interference pattern caused by the Brownian motion of the particles. The photographed hologram is shown in Figure 3.

S4: Estimate the image of the sensor plane in the empty field of view (the case where no particles exist in the solution) as a background image, that is, an image formed by incident light irradiating the solution without particles and propagating to the sensor plane. In this step, the method for estimating the background image is to estimate the pixel value of the entire image by taking the median value of the same pixel point in the entire hologram sequence as the pixel value of the point. Because the existence of particles in the solution causes diffraction, bright or dark stripes are formed on the sensor plane, and the particles are in a state of random Brownian motion during the shooting. Selecting the median value of the same point in multiple frames of images can effectively estimate the empty field of view (solution). The image of the sensor plane without the presence of particles) is used as the background image.

S5: Perform flat field correction on any hologram to be calculated and the estimated background image.

In this step, select any hologram of the hologram sequence and the estimated background image, denote the dark field image collected in step S1 as I _d , and denote the bright field image collected in step S2 as I ₀ , denote any hologram that needs to be calculated in the hologram sequence collected in step S3 as I, and denote the background image estimated in step S4 as I _b , then perform flat-field correction on I for the images I_c and The image I _{b_c} after flat-field correction for I _b is expressed as:

A relative value image is obtained by flat-field correction on the holographic image, which will not adversely affect the image processing, and can eliminate the uneven response of each pixel and alleviate the problem of image unevenness.

S6: normalizing the flat-field corrected hologram according to the flat-field corrected background image to obtain a holographic image formed by irradiating the sample with a normalized incident wave (with an amplitude of 1);

In this step, the normalized hologram B is expressed as:

It is regarded as a hologram formed by the normalized incident wave (that is, the amplitude is 1) irradiating the sample, scattering at the sample surface, and interfering with the incident wave at the sensor plane.

S7: Fit the hologram obtained by the normalized light source incident to the expression described by the scattering function and the incident light field, locate the three-dimensional position of the particle in the large field of view captured by the lensless microscope with high precision, and Accurately characterize particle size and refractive index information.

First of all, it needs to be clear that the solution must have a certain depth, so the particles in the solution will not all be in the same plane. Using the angular spectrum method of the scalar diffraction theory, the normalized hologram obtained in step S6 is reversely propagated to different height planes. The reverse propagation is relative to the direction of light propagation. =0 propagates towards the sample plane parallel to it at distance z. The angular spectrum method treats the propagation phenomenon of light as a linear spatial filter, and the transfer function H(f _X , f _Y ) is expressed as:

Among them, (f _X , f _Y ) represents the spatial frequency, and λ represents the wavelength of the incident light.

In this way, the light intensity distribution of different height planes in space is obtained. The Autofocusing algorithm is used to judge the sharpness of images backpropagated to different heights by calculating the variance of the derivative of the image brightness (usually using the sobel operator to find the first derivative), and the image with the largest variance is considered to be the most For a clear image, its corresponding height is considered to be the height of the particle, that is, the distance between the particle and the sensor. In this way, the height of the plane where each particle is located is obtained, that is, the distance between each particle and the sensor plane;

Secondly, when the incident wave is regarded as a linearly polarized plane wave, for an isotropic uniform sample, its normalized hologram B can be described as:

表示取实部操作，E_s(r,0)表示在样本平面的散射波电场的幅度，r表示传感器平面内任一点的坐标，k表示在微粒周围介质中的波数，z_p表示微粒中心和传感器平面的距离，u₀(r)表示平面内振幅分布。微粒处在相对于传感器平面中心位置r_p＝(0,0,z_p)处，那么在传感器平面(z＝0)内坐标r＝(x,y)位置处的散射波的电场E_s(r,0)可以表示为：in

represents the operation of taking the real part, E _s (r, 0) represents the amplitude of the scattered wave electric field in the sample plane, r represents the coordinates of any point in the sensor plane, k represents the wave number in the medium around the particle, z _p represents the particle center and The distance from the sensor plane, u ₀ (r) represents the in-plane amplitude distribution. The particle is at the position r _p =(0,0,z _p ) relative to the center of the sensor plane, then the electric field _Es ( r,0) can be expressed as:

E _s (r,0)=u ₀ (r _p )f _s (k(rr _p ))

where f _s (kr) is the normalized Mie scattering function. Therefore, the normalized hologram can be expressed as:

表示入射光的偏振方向。给出f_s(kr)的解析表达式，可以以微粒的三维位置r_p、尺寸和折射率为变量信息为未知参数，将归一化的全息图逐像素拟合至B。根据Lorenz-Mie散射理论，散射函数可以展开成如下矢量球谐函数的表达式：in

Indicates the polarization direction of incident light. Given the analytical expression of f _s (kr), the normalized hologram can be fitted to B pixel by pixel with the three-dimensional position _rp , size and refractive index of the particles as variable information as unknown parameters. According to the Lorenz-Mie scattering theory, the scattering function can be expanded into the expression of the following vector spherical harmonic function:

和

是矢量球谐函数，散射系数α_n和β_n与物体的尺寸、形状、组成、放置方向以及入射波的结构有关。

and

is a vector spherical harmonic function, and the scattering coefficients α _n and β _n are related to the size, shape, composition, placement direction of the object and the structure of the incident wave.

The scattering coefficient decreases rapidly as the order n increases, and the order is

post-convergence, where a _p represents the particle radius.

The wave scattered by the particle propagates dispersively, and its amplitude when it reaches the sensor plane is significantly smaller than the intensity of the incident light when it reaches the sensor plane, so the third term of the normalized hologram B is negligible compared with the other two, that is,

For the normalized hologram B obtained in step S6, the result obtained by subtracting all the pixel values of B by 1 is approximately considered to be an image formed by scattering light propagating to the sensor plane. Backpropagating it to the height where the particles are located, the intensity distribution of the scattered light in this plane is obtained, see Figure 4. The position of the particle in the hologram is detected by the method of coordinate weighting. The method of coordinate weighting is as follows: set a certain threshold, subtract the threshold from the hologram that propagates back to the height of the particle, and find a connected area, assuming a certain connected area There are n pixels, the value of each pixel is v(i), (i=1,2,...n), and the coordinates are (x(i),y(i)), according to the pixel value for each pixel Point setting weight w(i)=v(i)/∑v(i), (i=1,2,...n), then there is a particle in this area, the center coordinates are (x ₀ , y ₀ ), x ₀ =∑w(i)*x(i), y ₀ =∑w(i)*y(i), perform the same operation on each connected region, and obtain the center coordinates of all particles;

Again, a set of parameters uniformly distributed at 1.0-2.0 (adjusted according to the actual situation) are generated as possible distribution values of the refractive index n _p , and a set of parameters uniformly distributed at 0-10 μm (adjusted according to the actual situation) are used as particles The possible distribution values of the radius a _p and the estimation of the three-dimensional position of the particles have been determined in the previous step. Using these parameter sets and the known wavelength of the light source, pixel size, and refractive index of the medium, the forward model is calculated, that is, the hologram is generated:

表示入射光偏振方向的单位向量，f_si是Lorenz-Mie散射函数，表示振幅为1的线偏振平面波照射单个球形微粒产生的散射光，α_i≈1表示入射光强度的波动，n表示微粒数目；计算生成的全息图B_s与步骤S6得到的归一化后的全息图像B之间的均方误差，选择使得两者的均方误差最小的参数对{n_p，a_p}作为微粒折射率和半径的粗略估值；

represents the unit vector of the polarization direction of the incident light, f _si is the Lorenz-Mie scattering function, represents the scattered light generated by a linearly polarized plane wave with an amplitude of 1 irradiating a single spherical particle, α _i ≈ 1 represents the fluctuation of the intensity of the incident light, and n represents the number of particles Calculate the mean square error between the generated hologram B_s and the normalized hologram B obtained in step S6, and select the parameter pair {n _p , a _p } that minimizes the mean square error of the two as the particle refractive index and a rough estimate of the radius;

Then, the obtained rough estimation of each parameter is taken as the mean value, and each parameter is sampled from the Gaussian distribution of the given variance as the initial value of the least square fitting, and a random subset of the hologram is selected at the same time, and the three-dimensional position of the particle, With the index of refraction and radius unknown, a random subset of the hologram was fitted pixel-wise with Levenberg–Marquardt least squares to the expression described by the Lorenz-Mie scattering function and the incident light field:

A set of fitting results with the smallest mean square error is selected as the local optimal solution of the unknown parameters of the particle;

Next, take the result obtained in the previous step as the mean value, sample each parameter from the Gaussian distribution with reduced variance as the new initial value of the least squares fitting, and at the same time increase the volume of the selected random subset, using Levenberg–Marquardt least squares Multiply pixel-by-pixel fitting, and select a set of fitting results with the smallest mean square error as the local optimal solution of particle parameters. By gradually increasing the number of fitting pixels and reducing the initial value selection distribution, the robust convergence of the fitting process is guaranteed. This process can be repeated;

Finally, the volume fraction of the fitted subset is selected as 1, that is, the whole hologram is fitted, and the set of results obtained in the previous step is taken as the initial value to obtain the particle size of each particle in the large field of view captured by the lensless microscope. Accurate 3D position, size and refractive index information. See Figure 5 for the fitting results of the intercepted single-particle or multi-particle holograms. The results of locating and characterizing all particles in a region of interest in a hologram are shown in Figure 6.

Claims (4)

1. a method for positioning and characterizing spherical particles, is characterized in that, comprises the steps: S1, turn off the light source, and collect dark field images; S2, turn on the light source, and collect a brightfield image under the uniform illumination of the light source; S3, place a solution sample containing spherical particles above the sensor, and ensure that the distance from the sample to the sensor is much smaller than the distance from the sample to the light source; turn on the light source, collect the hologram sequence at a certain time interval, and adjust the camera exposure time at the same time to avoid particles without particles. Blurring of interference patterns caused by regular Brownian motion; S4, estimating the image of the sensor plane under the empty field of view as a background image, that is, an image formed by incident light irradiating a solution without particles and propagating to the sensor plane; S5, performing flat field correction on the hologram in step S3 and the background image estimated in step S4; S6, normalizing the flat-field-corrected hologram according to the flat-field-corrected background image to obtain a holographic image formed by illuminating the sample with the normalized light source; S7, fitting the holographic image obtained in step S6 to the expression described by the scattering function and the incident light field, to achieve high-precision three-dimensional positioning of the captured particles, and to accurately characterize the size and refractive index information of the particles at the same time; the specific process is: S71, the normalized holographic image B obtained in step S6 is back-propagated to planes with different heights by using the angular spectrum method of scalar diffraction theory, and the height of the plane where each particle is located is obtained by using an automatic focusing algorithm, that is, the distance of each particle from the sensor plane the distance; S72, subtracting all the pixel values of the normalized holographic image B obtained in step S6 by 1, and then backpropagating to the plane height where the particles are located; setting a certain threshold, subtracting the hologram backpropagating to the particle height by this Threshold, find the connected area, and use the coordinate weighting method to detect the position of each particle in the holographic image for each connected area; S73 , generating a set of parameters uniformly distributed within a certain refractive index range as possible distribution values of particle refractive index n _p , and a set of parameters uniformly distributed within a certain particle radius range as possible distribution values of particle radius a _p , the estimation of the three-dimensional position of the particle has been determined in step S72, and the forward model is calculated using these parameter sets and the known wavelength of the light source, pixel size, and refractive index of the medium, that is, the hologram is generated:

represents the unit vector of the polarization direction of the incident light, f _si is the Lorenz-Mie scattering function, represents the scattered light generated by a linearly polarized plane wave with an amplitude of 1 irradiating a single spherical particle, α _i ≈ 1 represents the fluctuation of the intensity of the incident light, and n represents the number of particles Calculate the mean square error between the generated hologram B_s and the normalized hologram B obtained in step S6, and select the parameter pair {n _p , a _p } that minimizes the mean square error of the two as the particle refractive index and a rough estimate of the radius; S74, take the obtained rough estimation of each parameter as the mean value, sample each parameter from the Gaussian distribution of the given variance as the initial value of the least square fitting, and select a random subset of the hologram at the same time, and use the three-dimensional position of the particle, The refractive index and radius are unknown, and a random subset of the hologram is fitted to the formula of the hologram B_s pixel by pixel using the Levenberg–Marquardt least squares method, and a set of fitting results with the smallest mean square error is selected as the local particle parameter. Optimal solution; S75, taking the result obtained in step S74 as the mean value, sampling each parameter from the Gaussian distribution with reduced variance as the new initial value of the least squares fitting, while increasing the volume of the selected random subset, using the Levenberg–Marquardt least squares method Pixel-by-pixel fitting, selecting a set of fitting results with the smallest mean square error as the local optimal solution of particle parameters; this process is repeated; S76 , the volume fraction of the fitting subset is finally selected to be 1, that is, the entire hologram is fitted, and the set of results obtained in step S75 is used as the initial value to obtain the precise three-dimensional position, size and refractive index information of each particle captured. 2. A method for locating and characterizing spherical particles according to claim 1, wherein in step S4, the method for estimating the image of the sensor plane under the empty field of view is: The median value of the pixel point is used as the pixel value of this point, so as to estimate the pixel value of the entire hologram. 3. a kind of method for positioning and characterizing spherical particles according to claim 1, is characterized in that, in step S5, the concrete method of flat field correction is: The collected dark field image is represented as I _d , the collected bright field image is represented as I ₀ , any hologram in the collected hologram sequence is represented as I, and the background image is represented as I _b , then the hologram I is performed. The image I_c after flat-field correction and the image I _{b_c} after flat-field correction on the background image I _b are respectively expressed as:

and

4. The method for locating and characterizing spherical particles according to claim 1, wherein in step S6, the holographic image B formed by the normalized light source irradiating the sample is expressed as:

Among them, I represents any hologram in the acquired hologram sequence, I _d represents the acquired dark field image, and I ₀ represents the acquired bright field image.

Images