CN105488780A - Monocular vision ranging tracking device used for industrial production line, and tracking method thereof - Google Patents
Monocular vision ranging tracking device used for industrial production line, and tracking method thereof Download PDFInfo
- Publication number
- CN105488780A CN105488780A CN201510132219.1A CN201510132219A CN105488780A CN 105488780 A CN105488780 A CN 105488780A CN 201510132219 A CN201510132219 A CN 201510132219A CN 105488780 A CN105488780 A CN 105488780A
- Authority
- CN
- China
- Prior art keywords
- image
- information
- module
- computer controller
- defocus
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Landscapes
- Image Processing (AREA)
Abstract
Description
技术领域technical field
本发明属于工业机器人技术领域,具体涉及一种用于工业生产线的单目视觉测距追踪装置及其追踪方法。The invention belongs to the technical field of industrial robots, and in particular relates to a monocular vision ranging tracking device and a tracking method for industrial production lines.
背景技术Background technique
近年来,随着人们对大脑视觉皮层(VisualCortex)功能结构的深入了解,计算机视觉这门新兴的学科也得到了越来越多的关注,并有了长足而深入的发展。视觉测量及3D运动定位追踪技术在相应的诸多领域也被广泛应用,比如应用于:娱乐游戏产业中基于定位控制的运动追踪游戏;军工产业得益于视觉实时定位技术,对目标对象进行识别与捕获;医疗产业中需要模拟人机交互操作;在人工智能中使用视觉操控机器人系统等领域。In recent years, with the in-depth understanding of the functional structure of the brain's visual cortex (VisualCortex), the emerging discipline of computer vision has also received more and more attention, and has made considerable and in-depth development. Visual measurement and 3D motion positioning and tracking technologies are also widely used in many corresponding fields, such as: motion tracking games based on positioning control in the entertainment game industry; the military industry benefits from visual real-time positioning technology to identify and capture; in the medical industry, human-computer interaction needs to be simulated; in artificial intelligence, vision is used to control robotic systems and other fields.
目前,大部分测距定位系统都采用双目视觉,其基于双目立体视觉理论,该理论建立在对人类视觉系统研究的基础上,通过双目视差获取场景信息。Marr、Poggio以及Grimson最早提出并实现了一种基于人类视觉系统的计算视觉模型及算法。虽然双目视觉是最接近于人类视觉的三维数据恢复方式,但事实上,传统双目定位系统存在一定的问题,例如:传统双目定位系统遮挡盲区现象,特征点匹配过程复杂以及要求高精度的双目协作。At present, most ranging and positioning systems use binocular vision, which is based on the theory of binocular stereo vision, which is based on the research on the human visual system and obtains scene information through binocular parallax. Marr, Poggio and Grimson first proposed and implemented a computational vision model and algorithm based on the human visual system. Although binocular vision is the 3D data recovery method closest to human vision, in fact, there are certain problems in the traditional binocular positioning system, such as: traditional binocular positioning system covers the blind area, the feature point matching process is complicated, and high precision is required binocular collaboration.
发明内容Contents of the invention
本发明的目的为了克服现有的双目视觉测距追踪系统中存在的:遮挡盲区现象,在测距算法执行中需要复杂的特征点匹配过程以及在控制操作中需要高精度的双目协作一系列问题,提出了一种用于工业生产线的单目视觉测距追踪装置及其追踪方法。The purpose of the present invention is to overcome the existing binocular vision distance measurement and tracking system: the phenomenon of occlusion of blind areas, the need for complex feature point matching processes in the execution of distance measurement algorithms and the need for high-precision binocular cooperation in control operations. A series of problems, a monocular vision ranging tracking device and tracking method for industrial production lines are proposed.
一种用于工业生产线的单目视觉测距追踪装置,包括:图像传感单元,控制校正单元,用户图形接口界面和数据存储模块;A monocular vision ranging tracking device for industrial production lines, including: an image sensing unit, a control and correction unit, a graphical user interface and a data storage module;
图像传感单元包括图像采集模块和图像预处理模块,用于图像获取。The image sensing unit includes an image acquisition module and an image preprocessing module for image acquisition.
其中,图像采集模块为图像传感单元的硬件采集部分,对特征捕捉对象进行图像采集,并将采集到的原始图像信息输出给图像预处理模块;图像预处理模块为软件算法处理部分,利用图像预处理算法,对原始图像信息进行图像处理:光照归一化处理,白平衡处理,曝光控制处理和图像卷积;将经过处理的图像信息输出给控制校正单元。Among them, the image acquisition module is the hardware acquisition part of the image sensing unit, which collects the image of the feature capture object, and outputs the collected original image information to the image preprocessing module; the image preprocessing module is the software algorithm processing part, which utilizes the image The preprocessing algorithm performs image processing on the original image information: illumination normalization processing, white balance processing, exposure control processing and image convolution; the processed image information is output to the control correction unit.
控制校正单元包括计算机控制器和执行终端模块,用于接收图像传感单元输出的图像信息,并根据相应的用户指令,控制执行终端模块执行相应的操作。The control and correction unit includes a computer controller and an execution terminal module, which is used to receive the image information output by the image sensing unit, and control the execution terminal module to perform corresponding operations according to corresponding user instructions.
计算机控制器接收图像预处理模块输出的图像信息,并接收从用户图形接口界面得到的用户指令,根据指令对图像进行处理及数据提取挖掘操作,将图像处理数据生成控制信号,以控制流的形式输出至执行终端模块。此外,计算机控制器还依据用户指令,将图像处理数据存储至相应的数据存储模块。The computer controller receives the image information output by the image preprocessing module, and receives the user instruction obtained from the user graphical interface interface, processes the image and extracts and mines the data according to the instruction, and generates a control signal from the image processing data in the form of a control flow Output to the execution terminal block. In addition, the computer controller also stores the image processing data into the corresponding data storage module according to the user's instruction.
执行终端模块连接图像采集模块,接收计算机控制器输出的图像处理数据进行校正、调整。The execution terminal module is connected to the image acquisition module, and receives the image processing data output by the computer controller for correction and adjustment.
用户图形接口界面输送用户指令给计算机控制器,同时从计算机控制器接收用户需要进行的图像处理指令和操作指令;The user graphical interface interface transmits user instructions to the computer controller, and at the same time receives image processing instructions and operation instructions required by the user from the computer controller;
数据存储模块存储计算机控制器输出的图像处理数据。The data storage module stores the image processing data output by the computer controller.
一种用于工业生产线的单目视觉测距追踪方法,具体步骤如下:A monocular vision ranging tracking method for industrial production lines, the specific steps are as follows:
步骤一、用户图形接口界面输出用户指令给计算机控制器,控制执行终端模块上的图像采集模块对特征捕捉对象进行图像采集;Step 1, the user graphical interface interface outputs user instructions to the computer controller, and controls the image acquisition module on the execution terminal module to perform image acquisition on the feature capture object;
步骤二、图像预处理模块利用图像预处理算法,对采集的图像进行处理,将图像信息输出给计算机控制器;Step 2, the image preprocessing module uses the image preprocessing algorithm to process the collected image, and outputs the image information to the computer controller;
步骤三、计算机控制器通过对图像进行处理及数据提取挖掘操作的方法,计算特征捕捉对象与执行终端模块之间的距离,实现单目视觉测距追踪;Step 3, the computer controller calculates the distance between the feature capture object and the execution terminal module by processing the image and data extraction and mining operations, so as to realize monocular vision distance measurement and tracking;
所述的计算机控制器对图像进行处理及数据提取挖掘操作的方法,具体步骤如下:The method for the computer controller to process images and extract and mine data, the specific steps are as follows:
步骤1:根据图像采集模块获取散焦模糊图像,建立数学模型。Step 1: Obtain a defocused blurred image according to the image acquisition module, and establish a mathematical model.
散焦模糊图像的形成过程的数学模型为:The mathematical model of the formation process of the defocused blurred image is:
f(ε,η)*h(x,y)+n(x,y)=g(x,y)f(ε,η)*h(x,y)+n(x,y)=g(x,y)
其中f(ε,η)为聚焦平面上得到的真实输入散焦模糊图像,h(x,y)为散焦退化核函数,n(x,y)为环境噪音,g(x,y)为实际输出的散焦模糊图像;Where f(ε,η) is the real input defocus blur image obtained on the focus plane, h(x,y) is the defocus degradation kernel function, n(x,y) is the environmental noise, g(x,y) is The actual output defocused blurred image;
其中散焦退化核函数采用点扩散函数模型,即圆形点扩散退化函数形式:Among them, the defocus degradation kernel function adopts the point spread function model, that is, the circular point spread degradation function form:
r为退化核函数的点扩散半径,定量地说明散焦图像的模糊程度;x和y为图像中对应扩散点的笛卡尔坐标系下二维坐标值。r is the point diffusion radius of the degenerate kernel function, which quantitatively illustrates the blurring degree of the defocused image; x and y are the two-dimensional coordinate values in the Cartesian coordinate system of the corresponding diffusion point in the image.
步骤2:采用倒谱算法对数学模型进行操作,分离真实输入图像信息与散焦退化核函数的信息。Step 2: Use the cepstrum algorithm to operate the mathematical model to separate the real input image information from the information of the defocus degradation kernel function.
步骤201、将实际输出散焦模糊图像信息数据表示为:真实输入图像信息数据与散焦退化核函数卷积形式;Step 201, express the actual output defocus blurred image information data as: the convolution form of the real input image information data and the defocus degradation kernel function;
g(x,y)=f(x,y).*h(x,y)g(x,y)=f(x,y).*h(x,y)
步骤202、利用傅里叶变换,将步骤201中的卷积结果转化为在频域上相乘的形式;Step 202, using Fourier transform to convert the convolution result in step 201 into a multiplication form in the frequency domain;
DFT(f(x,y).*h(x,y))=DFT(f(x,y))*DFT(h(x,y))DFT(f(x,y).*h(x,y))=DFT(f(x,y))*DFT(h(x,y))
步骤203、利用对数函数的性质,将步骤202中频域相乘结果转化为相加的形式;Step 203, utilizing the properties of the logarithmic function, converting the multiplication result in the frequency domain in step 202 into an addition form;
log(DFT(f(x,y)))+log(DFT(h(x,y)))log(DFT(f(x,y)))+log(DFT(h(x,y)))
步骤204、相加形式将真实输入图像信息与散焦退化核函数的信息分离;Step 204, the addition form separates the real input image information from the information of the defocus degradation kernel function;
步骤205、对步骤204得到的结果再次进行逆傅里叶变换,完成对图像的倒谱算法。Step 205, perform inverse Fourier transform again on the result obtained in step 204, and complete the cepstrum algorithm for the image.
对步骤204得到的对数频域图像处理结果再次进行逆傅里叶变换,完成对图像的倒谱算法,从而将其转换至倒谱域内,在倒谱域内对其特征信息进行分析。The inverse Fourier transform is performed on the logarithmic frequency domain image processing result obtained in step 204 again to complete the cepstrum algorithm for the image, thereby converting it into the cepstrum domain, and analyzing its characteristic information in the cepstrum domain.
步骤3:通过步骤2中分离出的散焦退化核函数相关信息,得到说明散焦图像的模糊程度的点扩散半径,进一步定量地。Step 3: Through the information related to the defocus degradation kernel function separated in step 2, the point diffusion radius indicating the degree of blur of the defocused image is obtained, further quantitatively.
步骤301、将步骤2中经过倒谱算法获取的数据三维显示;Step 301, three-dimensionally display the data acquired through the cepstrum algorithm in step 2;
步骤302、通过三维图像在二维平面上的包络结果,获取离中心点最近的环槽半径;Step 302, according to the envelope result of the three-dimensional image on the two-dimensional plane, the radius of the ring groove closest to the center point is obtained;
步骤303、所得到的环槽半径与点扩散半径r成正比关系。Step 303, the obtained ring groove radius is proportional to the point diffusion radius r.
步骤4:通过步骤3得到特征捕捉对象的距离信息结合对特征捕捉对象的2D定位,最终得到特征捕捉对象的3D定位坐标。Step 4: Obtain the distance information of the feature capture object through step 3 combined with the 2D positioning of the feature capture object, and finally obtain the 3D positioning coordinates of the feature capture object.
本发明的优点在于:The advantages of the present invention are:
(1)一种用于工业生产线的单目视觉测距追踪方法,不需要初始化标定及标记点,只需要保证对象物体为均匀着色的不透明体即可。(1) A monocular vision ranging tracking method for industrial production lines, which does not require initialization calibration and marking points, and only needs to ensure that the object is an opaque body with uniform coloring.
(2)一种用于工业生产线的单目视觉测距追踪装置,只使用单个中端工业相机,大大降低了硬件成本,并消除了上述背景中所述传统双目视觉测距定位装置的赘余及缺点,从而能够简单方便地进行测距追踪。(2) A monocular vision distance measurement and tracking device for industrial production lines, which only uses a single mid-range industrial camera, which greatly reduces hardware costs and eliminates the redundancy of the traditional binocular vision distance measurement and positioning device described in the background above. The advantages and disadvantages are eliminated, so that distance measurement and tracking can be performed simply and conveniently.
(3)一种用于工业生产线的单目视觉测距追踪方法,有效地去除诸如光照和微小振动等环境因素对系统的影响。(3) A monocular vision ranging tracking method for industrial production lines, which effectively removes the influence of environmental factors such as light and micro vibration on the system.
附图说明Description of drawings
图1是本发明一种用于工业生产线的单目视觉测距追踪装置示意图;Fig. 1 is a schematic diagram of a monocular vision distance measuring and tracking device used in an industrial production line according to the present invention;
图2是本发明单目视觉测距追踪装置中的图像传感单元示意图;Fig. 2 is a schematic diagram of the image sensing unit in the monocular vision ranging tracking device of the present invention;
图3是本发明散焦模糊图像的数学模型示意图;Fig. 3 is a schematic diagram of a mathematical model of a defocused blurred image of the present invention;
图4是本发明定量计算捕捉对象的距离信息示意图;Fig. 4 is a schematic diagram of the distance information of the object to be quantitatively calculated and captured by the present invention;
图5是本发明单目视觉测距追踪的原理图;Fig. 5 is a principle diagram of monocular vision distance measurement and tracking in the present invention;
图6是本发明一种用于工业生产线的单目视觉测距追踪方法流程图;Fig. 6 is a flow chart of a monocular vision distance measurement and tracking method used in an industrial production line according to the present invention;
图7是本发明计算机控制器对图像进行处理及数据提取挖掘操作的方法流程图;Fig. 7 is the flow chart of the method for image processing and data extraction and mining operation by the computer controller of the present invention;
图8是本发明单目视觉测距追踪方法中2D定位算法的流程图;Fig. 8 is a flow chart of the 2D positioning algorithm in the monocular vision ranging tracking method of the present invention;
具体实施方式detailed description
下面结合附图对本发明的具体实施方式作进一步说明。The specific embodiments of the present invention will be further described below in conjunction with the accompanying drawings.
一种用于工业生产线的单目视觉测距追踪装置,被应用于工业生产线上工业机器人运动操作的测距追踪。如图1所示,包括:图像传感单元,控制校正单元,用户图形接口界面和数据存储模块;A monocular vision ranging and tracking device for industrial production lines is applied to the ranging and tracking of industrial robot motion operations on industrial production lines. As shown in Figure 1, it includes: an image sensing unit, a control correction unit, a graphical user interface and a data storage module;
图像传感单元包括图像采集模块和图像预处理模块,用于图像获取;The image sensing unit includes an image acquisition module and an image preprocessing module for image acquisition;
其中,图像采集模块为图像传感单元的硬件采集部分,运用图像采集设备,对特征捕捉对象进行图像采集,并将采集到的原始图像信息经过图像数据总线,输出给图像预处理模块,进行下一步图像预处理步骤。Among them, the image acquisition module is the hardware acquisition part of the image sensing unit, which uses the image acquisition equipment to acquire images of the feature capture objects, and outputs the acquired original image information to the image preprocessing module through the image data bus for the following One step image preprocessing step.
图像采集设备优选照相机,摄像机或网络摄像头;本实施例中采用百万像素级工业定焦相机,具有成像畸变校正功能。其中,工业定焦相机采用固定焦距镜头,镜头的参数如表1所示,处理器内核为Linux3.0内核及以上,控制操作系统为Ubuntu11.04及以上,并具有相应的GigE千兆以太网接口。The image acquisition device is preferably a camera, video camera or network camera; in this embodiment, a megapixel-level industrial fixed-focus camera is used, which has an imaging distortion correction function. Among them, the industrial fixed-focus camera adopts a fixed focal length lens. The parameters of the lens are shown in Table 1. The processor core is Linux3.0 kernel and above, the control operating system is Ubuntu11.04 and above, and has corresponding GigE Gigabit Ethernet interface.
表1是工业定焦相机的固定焦距镜头参数Table 1 is the fixed focal length lens parameters of industrial fixed focus cameras
图像预处理模块为图像传感单元的软件算法处理部分。The image preprocessing module is the software algorithm processing part of the image sensing unit.
对于现有的任何一种图像采集设备,都需要一套软件包(SDK)来支持,与采集设备配套的视觉软件包通常包括以下模块:图像预处理部分;图像处理部分;字符识别部分;数据提取部分;图像资源管理部分;显示功能部分和其他功能部分。For any existing image acquisition device, a set of software package (SDK) is required to support it. The vision software package matched with the acquisition device usually includes the following modules: image preprocessing part; image processing part; character recognition part; data Extraction part; image resource management part; display function part and other function part.
如图2所示,图像采集模块运用工业定焦相机的定焦镜头和GigE千兆以太网,采集特征捕捉对象的图像,并传输给计算机控制器;同时图像预处理模块应用工业定焦相机自带软件开发工具包SDK中的图像预处理算法,对图像采集模块输出的原始图像信息进行处理,包括:设置图像捕捉模式,初始化图像质量以及初步简单的图像预处理功能;其中简单的图像处理过程,包括:光照归一化处理,白平衡处理,曝光控制处理和图像卷积处理;As shown in Figure 2, the image acquisition module uses the fixed-focus lens of the industrial fixed-focus camera and GigE Gigabit Ethernet to collect the image of the feature-captured object and transmits it to the computer controller; at the same time, the image pre-processing module uses the industrial fixed-focus camera to automatically With the image preprocessing algorithm in the software development kit SDK, it processes the original image information output by the image acquisition module, including: setting the image capture mode, initializing the image quality, and initial simple image preprocessing functions; the simple image processing process , including: illumination normalization processing, white balance processing, exposure control processing and image convolution processing;
光照归一化处理:去除光照条件不同对后续图像处理的影响;Illumination normalization processing: remove the influence of different illumination conditions on subsequent image processing;
白平衡处理:感知图像采集周围管径,调整色彩平衡,对图像采集模块输出的图像信号进行修正调节;White balance processing: perceive the diameter of the tube around the image acquisition, adjust the color balance, and correct and adjust the image signal output by the image acquisition module;
曝光控制处理:调整图像整体亮度;Exposure control processing: adjust the overall brightness of the image;
图像卷积:对图像进行边缘处理。Image convolution: performs edge processing on images.
图像预处理模块具体的处理步骤如下:The specific processing steps of the image preprocessing module are as follows:
第一步、设置Gamma校正,改善光线对获取的图像质量的影响;The first step is to set Gamma correction to improve the influence of light on the quality of the acquired image;
根据工业定焦相机的AnalogControls功能控制区中的Gammacorrection功能函数,设置GammaEnable让相机在获取图像并进行预处理时,自动对图像进行Gamma校正。According to the Gammacorrection function in the AnalogControls function control area of the industrial fixed-focus camera, set GammaEnable to allow the camera to automatically perform Gamma correction on the image when acquiring the image and performing preprocessing.
第二步、设置自动白平衡,根据不同环境条件下的光照情况相应调节白平衡;The second step is to set the automatic white balance, and adjust the white balance according to the lighting conditions under different environmental conditions;
工业定焦相机AnalogControls功能控制区中的设置白平衡的功能函数,分为手动设置和自动设置两种,在手动设置模式下:通过设置BalanceRatio选取需要的白平衡值;在自动设置模式下,选择BalanceWhiteAuto选项使相机在捕捉图像进行图像预处理时自动进行白平衡调节。The function of setting white balance in the AnalogControls function control area of industrial fixed-focus cameras is divided into two types: manual setting and automatic setting. In manual setting mode: select the required white balance value by setting BalanceRatio; in automatic setting mode, select The BalanceWhiteAuto option enables the camera to automatically perform white balance adjustments when capturing images for image preprocessing.
第三步、设置自动曝光,根据不同环境条件下的光照强度相应调节最优曝光时间。The third step is to set the automatic exposure, and adjust the optimal exposure time according to the light intensity under different environmental conditions.
工业定焦相机自带的AcquisitionControls功能控制区,封装了设置曝光模式/时间/时间基数的功能函数,分为手动设置和自动设置两种,在手动设置模式下,通过设置ExposureMode/Time/Timebase等选项设置调整用户所需要的曝光参数;在自动设置模式下,选择ExposureAuto选项来使相机在捕捉图像时,依据环境自动设置最优曝光参数。The AcquisitionControls function control area that comes with the industrial fixed-focus camera encapsulates the function of setting the exposure mode/time/time base, which is divided into manual setting and automatic setting. In the manual setting mode, by setting ExposureMode/Time/Timebase, etc. Option settings to adjust the exposure parameters required by the user; in the automatic setting mode, select the ExposureAuto option to enable the camera to automatically set the optimal exposure parameters according to the environment when capturing images.
最终将经过上述图像预处理之后的误差噪声较小的预处理图像,输出给下一层控制校正单元。Finally, the preprocessed image with less error and noise after the above image preprocessing is output to the control correction unit of the next layer.
控制校正单元包括计算机控制器和执行终端模块,用于接收图像传感单元输出的图像信息,并根据相应的用户指令,控制执行终端模块执行相应的操作。The control and correction unit includes a computer controller and an execution terminal module, which is used to receive the image information output by the image sensing unit, and control the execution terminal module to perform corresponding operations according to corresponding user instructions.
计算机控制器获取由图像预处理模块输出的图像信息包,并接收从用户图形接口界面得到的用户指令,根据指令对图像信息包进行特定的后期图像处理及数据提取挖掘操作,并生成相应的控制信号,将其以控制流的形式,通过数据总线输出至执行终端模块。The computer controller obtains the image information package output by the image preprocessing module, and receives user instructions obtained from the user graphical interface interface, performs specific post-image processing and data extraction and mining operations on the image information package according to the instructions, and generates corresponding control The signal is output to the execution terminal module through the data bus in the form of control flow.
此外,计算机控制器还依据用户指令,对用户感兴趣的数据进行存储,并将该数据存储至相应的数据存储模块,以便用户在需要的时候,调用数据存储模块中的数据进行查看以及其他的处理操作。In addition, the computer controller also stores the data that the user is interested in according to the user's instruction, and stores the data in the corresponding data storage module, so that the user can call the data in the data storage module for viewing and other functions when needed. Processing operations.
执行终端模块接收计算机控制器输出的校正及控制信息,对终端的姿态、位置等位姿信息进行校正、调整,最终达成用户指定的操作目的。The execution terminal module receives the correction and control information output by the computer controller, and corrects and adjusts the posture information such as the posture and position of the terminal, so as to finally achieve the operation purpose specified by the user.
用户图形接口界面属于辅助部分,用以产生友好的人机交互界面;一方面输送用户指令给计算机控制器,同时从计算机控制器接收用户需要进行的图像处理指令和操作指令;The graphical user interface belongs to the auxiliary part, which is used to generate a friendly human-computer interaction interface; on the one hand, the user instruction is sent to the computer controller, and at the same time, the image processing instruction and operation instruction required by the user are received from the computer controller;
数据存储模块属于辅助部分,用于存储计算机控制器输出的实时数据。The data storage module belongs to the auxiliary part and is used for storing the real-time data output by the computer controller.
一种用于工业生产线的单目视觉测距追踪方法,被应用于工业控制领域生产线上;单目视觉测距追踪的原理,如图5所示,采用离焦透镜成像系统的物理光学成像原理,其中,图像采集设备被简化为定焦透镜,由于距离不同,特征捕捉对象无法准确聚焦在清晰聚焦平面之上,从而产生一定程度的散焦扩散退化。获取到的实际输出图像信息出现一定程度的散焦点扩散效应,对于散焦点扩散效应的定量表示为点扩散半径,由物理光学透镜成像规律公式:A monocular vision distance measurement and tracking method used in industrial production lines is applied to the production line in the field of industrial control; the principle of monocular vision distance measurement and tracking, as shown in Figure 5, uses the physical optical imaging principle of the defocus lens imaging system , where the image acquisition device is simplified as a fixed-focus lens. Due to the different distances, the feature capture objects cannot be accurately focused on the clear focus plane, resulting in a certain degree of defocus diffusion degradation. The actual output image information obtained has a certain degree of defocusing point diffusion effect, and the quantitative expression of the defocusing point diffusion effect is the point diffusion radius, which is determined by the physical optics lens imaging rule formula:
其中,u为物距,是特征捕捉对象与透镜之间的距离;f为透镜焦距;v为聚焦成像像距。Among them, u is the object distance, which is the distance between the feature capture object and the lens; f is the focal length of the lens; v is the focusing imaging image distance.
由相似三角形的比例可知点扩散半径与物距u之间存在定量关系:当特征捕捉对象在聚焦平面上清晰成像时,在清晰聚焦平面前,实际成像像距为v1,点扩散半径为R1;From the ratio of similar triangles, it can be seen that there is a quantitative relationship between the point diffusion radius and the object distance u: when the feature capture object is clearly imaged on the focal plane, the actual imaging image distance is v 1 before the clear focal plane, and the point diffusion radius is R 1 ;
在清晰聚焦平面后,实际成像像距为v2,点扩散半径为R2;After the clear focus plane, the actual imaging image distance is v 2 , and the point diffusion radius is R 2 ;
其中,d为透镜直径;Among them, d is the lens diameter;
根据点扩散半径,得到特征捕捉对象与透镜之间的距离u,进行捕捉对象的测距追踪。According to the point diffusion radius, the distance u between the feature capturing object and the lens is obtained, and the ranging tracking of the capturing object is performed.
该软件算法基于信息学理论中现代控制方法的系统思想,设定所得到的模糊图像为聚焦平面之上的清晰图像经过散焦退化核函数的模糊作用相应得到,其中还需要考虑成像过程中所受到的环境噪音。The software algorithm is based on the system idea of modern control methods in informatics theory, and the obtained blurred image is set to be a clear image above the focus plane through the blurring effect of the defocus degradation kernel function, which also needs to consider the imaging process. exposure to ambient noise.
如图6所示,该单目视觉测距追踪方法具体步骤如下:As shown in Figure 6, the specific steps of the monocular vision ranging tracking method are as follows:
步骤一、用户图形接口界面输出用户指令给计算机控制器,控制执行终端模块上的图像采集模块对特征捕捉对象进行图像采集;Step 1, the user graphical interface interface outputs user instructions to the computer controller, and controls the image acquisition module on the execution terminal module to perform image acquisition on the feature capture object;
步骤二、图像预处理模块利用图像预处理算法,对采集的图像进行处理,将图像信息输出给计算机控制器;Step 2, the image preprocessing module uses the image preprocessing algorithm to process the collected image, and outputs the image information to the computer controller;
步骤三、计算机控制器通过对图像进行处理及数据提取挖掘操作的方法,计算特征捕捉对象与执行终端模块之间的距离,实现单目视觉测距追踪;Step 3, the computer controller calculates the distance between the feature capture object and the execution terminal module by processing the image and data extraction and mining operations, so as to realize monocular vision distance measurement and tracking;
所述的计算机控制器对图像进行处理及数据提取挖掘操作的方法,如图7所示,具体步骤如下:The method for image processing and data extraction mining operation by the computer controller is as shown in Figure 7, and the specific steps are as follows:
步骤1:根据图像采集模块获取散焦模糊图像,建立数学模型。Step 1: Obtain a defocused blurred image according to the image acquisition module, and establish a mathematical model.
该数学模型基于图像退化的思想,即真实输入图像经过图像采集模块形成实际输出图像,由于图像采集模块自身的物理性质及环境噪声的影响,真实输入图像与实际输出图像并不相同。The mathematical model is based on the idea of image degradation, that is, the actual input image is formed by the image acquisition module to form the actual output image. Due to the physical properties of the image acquisition module itself and the influence of environmental noise, the actual input image is not the same as the actual output image.
如图3所示:散焦模糊图像的形成过程的数学模型为:As shown in Figure 3: the mathematical model of the formation process of defocus blur image is:
f(ε,η)*h(x,y)+n(x,y)=g(x,y)f(ε,η)*h(x,y)+n(x,y)=g(x,y)
其中f(ε,η)为聚焦平面上得到的真实输入图像,h(x,y)为散焦退化核函数,n(x,y)为环境噪音,g(x,y)为实际输出的散焦模糊图像。Where f(ε,η) is the real input image obtained on the focus plane, h(x,y) is the defocus degradation kernel function, n(x,y) is the environmental noise, g(x,y) is the actual output Defocus blurred image.
其中散焦退化核函数采用点扩散函数模型,即圆形点扩散退化函数形式:Among them, the defocus degradation kernel function adopts the point spread function model, that is, the circular point spread degradation function form:
r为退化核函数的点扩散半径,定量地说明散焦图像的模糊程度;x和y为图像中对应扩散点的笛卡尔坐标系下二维坐标值。r is the point diffusion radius of the degenerate kernel function, which quantitatively illustrates the blurring degree of the defocused image; x and y are the two-dimensional coordinate values in the Cartesian coordinate system of the corresponding diffusion point in the image.
控制校正单元接收图像采集模块获取的散焦模糊图像,结合用户图形接口界面得到的用户指令校正及控制信息,对终端的姿态、位置等位姿信息进行校正、调整,最终达成用户指定的操作目的。The control correction unit receives the defocused blurred image obtained by the image acquisition module, and combines the user command correction and control information obtained from the user graphical interface interface to correct and adjust the pose information of the terminal such as posture and position, and finally achieve the operation purpose specified by the user .
步骤2:采用倒谱算法对数学模型进行操作,分离真实输入图像信息与散焦退化核函数的信息。Step 2: Use the cepstrum algorithm to operate the mathematical model to separate the real input image information from the information of the defocus degradation kernel function.
基于倒谱算法求取点扩散半径r的具体步骤如下:The specific steps to obtain the point diffusion radius r based on the cepstrum algorithm are as follows:
步骤201、将实际输出散焦模糊图像信息数据表示为:真实输入图像信息数据与散焦退化核函数卷积形式;Step 201, express the actual output defocus blurred image information data as: the convolution form of the real input image information data and the defocus degradation kernel function;
g(x,y)=f(x,y).*h(x,y)g(x,y)=f(x,y).*h(x,y)
步骤202、利用傅里叶变换,将步骤201中的卷积结果转化为在频域上相乘的形式;Step 202, using Fourier transform to convert the convolution result in step 201 into a multiplication form in the frequency domain;
DFT(f(x,y).*h(x,y))=DFT(f(x,y))*DFT(h(x,y))DFT(f(x,y).*h(x,y))=DFT(f(x,y))*DFT(h(x,y))
步骤203、利用对数函数的性质,将步骤202中频域相乘结果转化为相加的形式;Step 203, utilizing the properties of the logarithmic function, converting the multiplication result in the frequency domain in step 202 into an addition form;
log(DFT(f(x,y)))+log(DFT(h(x,y)))log(DFT(f(x,y)))+log(DFT(h(x,y)))
步骤204、相加形式将真实输入图像信息与散焦退化核函数的信息分离;Step 204, the addition form separates the real input image information from the information of the defocus degradation kernel function;
步骤205、对步骤204得到的结果再次进行逆傅里叶变换,完成对图像的倒谱算法。Step 205, perform inverse Fourier transform again on the result obtained in step 204, and complete the cepstrum algorithm for the image.
对步骤204得到的对数频域图像处理结果再次进行逆傅里叶变换,完成对图像的倒谱算法,从而将其转换至倒谱域内,在倒谱域内对其特征信息进行分析。The inverse Fourier transform is performed on the logarithmic frequency domain image processing result obtained in step 204 again to complete the cepstrum algorithm for the image, thereby converting it into the cepstrum domain, and analyzing its characteristic information in the cepstrum domain.
数据存储模块对真实输入图像信息与散焦退化核函数的信息进行存储,并与用户图形接口界面进行交互。The data storage module stores the real input image information and the information of the defocus degradation kernel function, and interacts with the user graphical interface.
步骤3:通过对步骤2中分离出的散焦退化核函数相关信息的分析,得到说明散焦图像的模糊程度的点扩散半径。Step 3: By analyzing the relevant information of the defocus degradation kernel function separated in step 2, a point diffusion radius indicating the degree of blur of the defocused image is obtained.
步骤301、将步骤2中经过倒谱算法获取的数据三维显示;Step 301, three-dimensionally display the data acquired through the cepstrum algorithm in step 2;
该数据利用处理OpenCV软件进行处理。The data were processed using OpenCV software.
步骤302、通过三维图像在二维平面上的包络结果,获取离中心点最近的环槽半径;Step 302, according to the result of the envelope of the three-dimensional image on the two-dimensional plane, the radius of the ring groove closest to the center point is obtained;
所述的二维平面指X-Z/Y-Z平面。The two-dimensional plane refers to the X-Z/Y-Z plane.
步骤303、所得到的环槽半径与点扩散半径r成正比关系。Step 303, the obtained ring groove radius is proportional to the point diffusion radius r.
如图4所示,将纵坐标为y=0设为倒谱域零平面,简称零线,x≈641处为倒谱域处理结果图像中心峰值处;不同曲线代表利用倒谱算法得到的,具有不同r的散焦退化核函数处理结果二维图像,不同r的散焦退化核函数曲线与零线的交点为零点,零点中距离中心峰值x≈641最近的交点距中心峰值的水平距离为环槽半径的值,该环槽半径正比于点扩散半径r。As shown in Figure 4, the ordinate is set to y=0 as the zero plane in the cepstrum domain, referred to as the zero line, and x≈641 is the center peak of the cepstrum domain processing result image; different curves represent those obtained by using the cepstrum algorithm, The two-dimensional image of the processing result of the defocus degradation kernel function with different r, the intersection point of the defocus degradation kernel function curve and the zero line of different r is the zero point, and the horizontal distance from the intersection point closest to the central peak x≈641 of the zero point to the central peak value is The value of the groove radius, which is proportional to the point spread radius r.
本发明中,该环槽半径为点扩散半径r的2倍。In the present invention, the radius of the annular groove is twice the point diffusion radius r.
至此,利用环槽半径定量地表示点扩散半径r,从而进一步定量计算捕捉对象的距离信息。So far, the point diffusion radius r is expressed quantitatively by the radius of the ring groove, so as to further quantitatively calculate the distance information of the captured object.
数据存储模块对捕捉对象的距离信息进行存储,并与用户图形接口界面进行交互。The data storage module stores the distance information of the captured object and interacts with the graphical user interface.
步骤4:通过步骤3得到特征捕捉对象的距离信息结合对特征捕捉对象的2D定位,最终得到特征捕捉对象的3D定位坐标。Step 4: Obtain the distance information of the feature capture object through step 3 combined with the 2D positioning of the feature capture object, and finally obtain the 3D positioning coordinates of the feature capture object.
对于特征捕捉对象的2D定位采用可自动校正的提取算法。2D定位算法执行自动重定位功能,如图8所示,具体步骤如下:For 2D positioning of feature-captured objects, an automatically correctable extraction algorithm is used. The 2D positioning algorithm performs automatic relocation function, as shown in Figure 8, and the specific steps are as follows:
步骤401、设置泊松核函数对原始获取图像进行预处理;Step 401, setting a Poisson kernel function to preprocess the original acquired image;
步骤402、追踪初始化设定的特征捕捉对象,根据该特征捕捉对象得到点扩散半径;Step 402, tracking the feature capture object set by initialization, and obtaining the point diffusion radius according to the feature capture object;
步骤403、获取时域上连续的图像相关性和差异性信息;Step 403, acquiring continuous image correlation and difference information in the time domain;
步骤404、根据图像相关性和差异性信息,判断连续的图像是否稳定,如果稳定,进入步骤405,否则进入步骤406;Step 404, according to the image correlation and difference information, judge whether the continuous images are stable, if stable, go to step 405, otherwise go to step 406;
步骤405、根据稳定的连续图像,确认点扩散半径,获取捕捉特征对象的3D定位坐标;Step 405, according to the stable continuous image, confirm the point diffusion radius, and obtain the 3D positioning coordinates of the captured feature object;
根据稳定的连续图像,确认点扩散半径并输出给用户图形接口界面,捕捉特征对象,获取该特征捕捉对象的3D定位坐标;According to the stable continuous image, confirm the point diffusion radius and output it to the user graphical interface, capture the feature object, and obtain the 3D positioning coordinates of the feature capture object;
步骤406、所获取的图像高度不稳定,则该特征捕捉对象的点扩散半径错误,返回步骤402。Step 406 , if the acquired image is highly unstable, then the point diffusion radius of the feature capturing object is wrong, and return to step 402 .
软件算法部分利用单目近景测距思想,基于运动散焦模糊算法的测距算法,通过所获取的一系列散焦模糊图像的模糊程度反映特征捕捉对象的距离信息。与此同时,应用信息学理论,基于频域倒谱算法获取散焦模糊图像点扩散核函数的点扩散半径,进而得到图像模糊退化模型的退化系统函数,从而准确且定量地计算得出所述特征捕捉对象的距离信息,最终辅以相应的2D自动校正定位算法,获得特征捕捉对象的测距及定位坐标信息。The software algorithm part uses the idea of monocular close-range distance measurement, based on the distance measurement algorithm of the motion defocus blur algorithm, and reflects the distance information of the feature capture object through the blur degree of a series of defocus blur images obtained. At the same time, applying informatics theory, based on the frequency domain cepstrum algorithm, the point diffusion radius of the point diffusion kernel function of the defocused blurred image is obtained, and then the degradation system function of the image blur degradation model is obtained, so as to accurately and quantitatively calculate the The distance information of the feature capture object is finally supplemented with the corresponding 2D automatic correction positioning algorithm to obtain the distance measurement and positioning coordinate information of the feature capture object.
由于该系统结合运用了一种新的算法处理思想,且主要应用于工业机器人领域,故必须考虑其在工业机器人领域使用过程中的特殊要求及可靠性问题。工业机器人操作系统需要较小的波动性和较高的稳定性,基于上述原因,在本发明的算法中加入了平滑滤波算法,从而对控制器提供相对平滑稳定的定位数据。Since the system uses a new algorithm processing idea and is mainly used in the field of industrial robots, it must consider its special requirements and reliability issues in the field of industrial robots. The operating system of an industrial robot requires less fluctuation and higher stability. Based on the above reasons, a smoothing filter algorithm is added to the algorithm of the present invention to provide relatively smooth and stable positioning data for the controller.
Claims (5)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510132219.1A CN105488780A (en) | 2015-03-25 | 2015-03-25 | Monocular vision ranging tracking device used for industrial production line, and tracking method thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510132219.1A CN105488780A (en) | 2015-03-25 | 2015-03-25 | Monocular vision ranging tracking device used for industrial production line, and tracking method thereof |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN105488780A true CN105488780A (en) | 2016-04-13 |
Family
ID=55675746
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510132219.1A Pending CN105488780A (en) | 2015-03-25 | 2015-03-25 | Monocular vision ranging tracking device used for industrial production line, and tracking method thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105488780A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107369135A (en) * | 2017-06-22 | 2017-11-21 | 广西大学 | A kind of micro imaging system three-dimensional point spread function space size choosing method based on Scale invariant features transform algorithm |
| CN109060819A (en) * | 2018-07-06 | 2018-12-21 | 中国飞机强度研究所 | Error correcting method in visual field in a kind of measurement of vibration component crackle |
| CN109102525A (en) * | 2018-07-19 | 2018-12-28 | 浙江工业大学 | A kind of mobile robot follow-up control method based on the estimation of adaptive pose |
| CN111242861A (en) * | 2020-01-09 | 2020-06-05 | 浙江光珀智能科技有限公司 | Method and device for removing stray light of TOF camera, electronic equipment and storage medium |
| CN117876429A (en) * | 2024-03-12 | 2024-04-12 | 潍坊海之晨人工智能有限公司 | Real standard platform of sports type industry vision |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101662566A (en) * | 2008-08-27 | 2010-03-03 | 索尼株式会社 | Information processing apparatus, information processing method, and program |
| CN102997891A (en) * | 2012-11-16 | 2013-03-27 | 上海光亮光电科技有限公司 | Device and method for measuring scene depth |
| CN104299246A (en) * | 2014-10-14 | 2015-01-21 | 江苏湃锐自动化科技有限公司 | Production line object part motion detection and tracking method based on videos |
-
2015
- 2015-03-25 CN CN201510132219.1A patent/CN105488780A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101662566A (en) * | 2008-08-27 | 2010-03-03 | 索尼株式会社 | Information processing apparatus, information processing method, and program |
| CN102997891A (en) * | 2012-11-16 | 2013-03-27 | 上海光亮光电科技有限公司 | Device and method for measuring scene depth |
| CN104299246A (en) * | 2014-10-14 | 2015-01-21 | 江苏湃锐自动化科技有限公司 | Production line object part motion detection and tracking method based on videos |
Non-Patent Citations (3)
| Title |
|---|
| HUADONG SUN等: "Depth From Defocus and Blur for Signal Image", 《VISUAL COMMUNICATION AND IMAGE PROCESSING》 * |
| 李秀怡: "一种基于倒谱相关特性的散焦模糊图像复原方法", 《计算机技术与发展》 * |
| 董杰: "基于单目视觉的散焦测距算法的研究", 《中国博士论文全文数据库》 * |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107369135A (en) * | 2017-06-22 | 2017-11-21 | 广西大学 | A kind of micro imaging system three-dimensional point spread function space size choosing method based on Scale invariant features transform algorithm |
| CN109060819A (en) * | 2018-07-06 | 2018-12-21 | 中国飞机强度研究所 | Error correcting method in visual field in a kind of measurement of vibration component crackle |
| CN109060819B (en) * | 2018-07-06 | 2021-03-30 | 中国飞机强度研究所 | Method for correcting errors in field of view in measurement of cracks of vibration component |
| CN109102525A (en) * | 2018-07-19 | 2018-12-28 | 浙江工业大学 | A kind of mobile robot follow-up control method based on the estimation of adaptive pose |
| CN109102525B (en) * | 2018-07-19 | 2021-06-18 | 浙江工业大学 | A mobile robot following control method based on adaptive pose estimation |
| CN111242861A (en) * | 2020-01-09 | 2020-06-05 | 浙江光珀智能科技有限公司 | Method and device for removing stray light of TOF camera, electronic equipment and storage medium |
| CN111242861B (en) * | 2020-01-09 | 2023-09-12 | 浙江光珀智能科技有限公司 | Methods and devices, electronic equipment and storage media for removing TOF camera stray light |
| CN117876429A (en) * | 2024-03-12 | 2024-04-12 | 潍坊海之晨人工智能有限公司 | Real standard platform of sports type industry vision |
| CN117876429B (en) * | 2024-03-12 | 2024-06-07 | 潍坊海之晨人工智能有限公司 | Real standard system of sports type industry vision |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106778867B (en) | Target detection method and device, neural network training method and device | |
| CN110147721B (en) | Three-dimensional face recognition method, model training method and device | |
| JP5871862B2 (en) | Image blur based on 3D depth information | |
| CN108363995B (en) | Method and apparatus for generating data | |
| CN112423191B (en) | Video call device and audio gain method | |
| CN106570899B (en) | Target object detection method and device | |
| CN118570312B (en) | A multi-camera collaborative calibration method and application for dynamic vision sensors | |
| CN106600650A (en) | A method for acquiring binocular vision depth information based on deep learning | |
| CN105488780A (en) | Monocular vision ranging tracking device used for industrial production line, and tracking method thereof | |
| CN106524909B (en) | Three-dimensional image acquisition method and device | |
| CN109035307B (en) | Target tracking method and system for setting area based on natural light binocular vision | |
| CN105190229A (en) | Three-dimensional shape measurement device, three-dimensional shape measurement method, and three-dimensional shape measurement program | |
| CN113283329B (en) | Eye tracking system, eye tracker, eye tracking method, eye tracking device, and medium | |
| CN115103094A (en) | Camera module far-view angle adjusting method and system based on fixation point | |
| CN114363522A (en) | Photographing method and related device | |
| KR20150069739A (en) | Method measuring fish number based on stereovision and pattern recognition system adopting the same | |
| CN110544278A (en) | rigid body motion capture method and device and AGV pose capture system | |
| CN116823876A (en) | Mobile target positioning method, equipment, storage medium and device | |
| CN116506731A (en) | Focus shooting method, device, storage medium and electronic equipment | |
| CN109328459B (en) | Intelligent terminal, 3D imaging method and 3D imaging system thereof | |
| CN107563333A (en) | A kind of binocular vision gesture identification method and device based on ranging auxiliary | |
| CN112073640A (en) | Panoramic information acquisition pose acquisition method, device and system | |
| CN111541888A (en) | AR implementation method based on display surface | |
| WO2024191729A1 (en) | Image based reconstruction of 3d landmarks for use in generation of personalized head-related transfer functions | |
| CN206583440U (en) | A kind of projected image sighting distance detecting system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| DD01 | Delivery of document by public notice |
Addressee: Ao Bo (Beijing) Technology Co. Ltd. Document name: Notice of non patent agency |
|
| DD01 | Delivery of document by public notice |
Addressee: Ao Bo (Beijing) Technology Co. Ltd. Document name: Notification to Make Rectification |
|
| C06 | Publication | ||
| PB01 | Publication | ||
| DD01 | Delivery of document by public notice |
Addressee: Ao Bo (Beijing) Technology Co. Ltd. Document name: Notification of Passing Examination on Formalities |
|
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20160413 |
|
| RJ01 | Rejection of invention patent application after publication |