CN114638860A - Method, device and readable storage medium for tracking traffic signal - Google Patents

Abstract

The application provides a tracking method and a tracking device for a traffic signal lamp and a readable storage medium, which are used for solving the problem that an electric alarm camera in the prior art is low in accuracy in judging the state of the signal lamp. The method comprises the following steps: determining a position parameter and a first characteristic parameter of a first area where a traffic signal lamp is located in a first traffic image; the first area comprises coordinates of a center point of a traffic signal lamp in a first traffic image; determining first information of a first area in the first traffic image based on the first characteristic parameter; determining a second area in a second traffic image and second information of the second area based on the position parameter of the first area; and taking the first information as a template, determining the correlation between the template and the second information, and determining the position of a pixel point corresponding to the maximum correlation value as the central position of a signal lamp in the second traffic image.

Description

Method, device and readable storage medium for tracking traffic signal

technical field

The present application relates to the technical field of image processing, and in particular, to a traffic signal tracking method, device and readable storage medium.

Background technique

As a signal sign, traffic lights play an important role in the regulation of traffic. In order to build a smart city, an electric police camera is generally set up in the setting area facing the traffic lights to capture images (frames) of irregular driving such as vehicles running through red lights. After the electric police camera captures the image, it needs to first judge the status of the signal lights in the image to further determine whether the vehicle in the image is in an irregular driving state. However, the resolution of the images (frames) captured by the electric police camera greatly reduces the judgment efficiency of the signal light state, and cannot meet the real-time requirements of the signal light state judgment. Then, before judging the status of the signal light, a small area of interest needs to be set for the electric police camera to judge the status of the signal light. However, because the electric police cameras and traffic lights are exposed to the open air for a long time, they are easily affected by strong winds or other natural phenomena, and then shift, which leads to the fact that the area of interest selected by the electric police cameras according to the setting rules does not include signal lights. As a result, it cannot be accurately or even impossible to judge the status of the signal light.

Therefore, in the prior art, there is a problem that the electric police camera has low accuracy in judging the state of the signal light.

SUMMARY OF THE INVENTION

The present application provides a traffic signal tracking method, device and readable storage medium, which are used to solve the problem of low accuracy in judging the status of the signal light in the prior art.

In a first aspect, the present application provides a method for tracking traffic lights, including:

Determine the position parameter and the first characteristic parameter of the first area where the traffic signal light is located in the first traffic image; wherein, the first area includes the coordinates of the center point of the traffic signal in the first traffic image, and the first characteristic parameter indicates the Grayscale features and gradient features of pixels in the first region;

Based on the first characteristic parameter, determine the first information of the first area in the first traffic image; wherein, the first information indicates that each pixel in the first area in the first traffic image Grayscale features, gradient features, and the relationship between the grayscale of each pixel point and the grayscale of other pixel points;

A second area in the second traffic image and second information of the second area are determined based on the location parameters of the first area; wherein the second area includes the coordinates of the center point, the second area The information indicates the grayscale feature and gradient feature of each pixel in the second area, and the relationship feature between the grayscale of each pixel and the grayscale of other pixels;

Using the first information as a template, determine the correlation between the template and the second information, and determine that the position of the pixel corresponding to the maximum correlation value is the center position of the signal light in the second traffic image; Wherein, the correlation indicates the correlation between all the pixels in the first area in the first traffic image and all the pixels in the second area in the second traffic image.

The above method obtains the first information as a template by pre-extracting the position parameter and the first feature parameter in the first traffic image, and obtains the correlation between the template and the second information of the second area in the second traffic image obtained subsequently, Further, the center coordinates of the second traffic signal are determined, so as to track the traffic signal and find out the deviation of the position of the traffic signal in time, thereby avoiding the occurrence of misjudgment or inability to judge the status of the traffic signal.

A possible implementation manner, the determining the correlation between the template and the second information includes:

determining the correlation between the template and the second information based on the target tracking algorithm and the signal correlation principle;

The maximum value in the correlation is compared with the set threshold, and when the maximum value in the correlation is greater than the set threshold, it is determined that the traffic signal tracking is successful.

A possible implementation manner, the determining the position parameter and the first characteristic parameter of the first area where the traffic signal light is located in the first traffic image includes:

Using the circumscribed rectangle of the traffic light as a selection frame, frame the first area in the first traffic image;

determining the standard location information of the first area; wherein the standard location information includes the coordinates of the center point of the first area;

Extracting a set number of the first traffic images, determining at least one first area in each of the first traffic images, and determining whether the location information of the at least one first area is consistent with the standard location information ;

If yes, take the position information of the first area in any one of the first traffic images as the location parameter of the first area; and determine the location within the first area in any of the first traffic images The gray value and gradient value of all pixel points are obtained to obtain the first characteristic parameter of the first region.

By comparing the position information of the first area in the first traffic image with the standard position information, the above method can ensure that the first area selected in the first traffic image is the first area on the target position, thereby improving the success of the method. Probability of tracking traffic lights.

A possible implementation manner, the first information M is:

λ为预设常数，

σ为预设标准差参数，☉为点乘运算，F(·)为傅里叶变换，F^-1(·)为傅里叶反变换，X为所述第一特征参数，X′为X的转置，w为所述第一区域的长，h为所述第一区域的宽，(i，j)为所述第一区域内的像素点坐标，(i_c，j_c)为所述第一区域内的中心像素点坐标。in,

λ is a preset constant,

σ is the preset standard deviation parameter, ☉ is the dot multiplication operation, F(·) is the Fourier transform, F ⁻¹ (·) is the inverse Fourier transform, X is the first characteristic parameter, and X′ is X The transpose of , w is the length of the first area, h is the width of the first area, (i, j) is the pixel coordinates in the first area, ( _{ic , j c )} is the The coordinates of the center pixel point in the first area.

A possible implementation manner, after determining that the location of the pixel point corresponding to the maximum correlation value is the center position of the signal light in the second traffic image, the method includes:

Using the length and width of the first area in the first traffic image, determine a third area in the second traffic image with the center position of the signal light in the second traffic image as the center of the area;

Determine third information of the third area; wherein the third information indicates the grayscale feature of each pixel in the third area, and the relationship between the grayscale of each pixel and the grayscale of other pixels ;

The template and the third information are respectively multiplied and summed with corresponding weights to obtain fourth information, and the template is updated based on the fourth information for the tracking of signal lights in the next frame of traffic images.

In a second aspect, the present application provides a tracking device for a traffic signal, comprising:

Parameter unit: used to determine the position parameter and the first characteristic parameter of the first area where the traffic light is located in the first traffic image; wherein, the first area includes the coordinates of the center point of the traffic signal in the first traffic image, and the first The feature parameter indicates the grayscale feature and gradient feature of the pixel in the first region;

Information unit: used to determine the first information of the first area in the first traffic image based on the first characteristic parameter; wherein, the first information indicates that in the first traffic image, the first area The grayscale feature and gradient feature of each pixel, and the relationship feature between the grayscale of each pixel and the grayscale of other pixels;

Determining unit: for determining a second area in the second traffic image and second information of the second area based on the position parameter of the first area; wherein, the second area includes the coordinates of the center point , the second information indicates the grayscale feature and gradient feature of each pixel in the second area, and the relationship feature between the grayscale of each pixel and the grayscale of other pixels;

Central unit: used to use the first information as a template, determine the correlation between the template and the second information, and determine that the location of the pixel corresponding to the maximum correlation value is in the second traffic image. The center position of the signal light; wherein the correlation indicates the correlation between all the pixels in the first area in the first traffic image and all the pixels in the second area in the second traffic image.

A possible implementation manner, the central unit is specifically configured to determine the correlation between the template and the second information based on the target tracking algorithm and the signal correlation principle; Threshold value comparison, when the maximum value in the correlation is greater than the set threshold value, it is determined that the traffic signal tracking is successful.

A possible implementation manner, the parameter unit is specifically configured to use the circumscribed rectangle of the traffic signal as a selection box to select the first area in the first traffic image; determine the standard position of the first area information; wherein, the standard location information includes the coordinates of the center point of the first area; extracting a set number of the first traffic images, and determining at least one first area in each of the first traffic images, and determine whether the location information of the at least one first area is consistent with the standard location information; if so, use the location information of the first area in any one of the first traffic images as the location information of the first area. position parameters; and determine the grayscale values and gradient values of all pixels in the first area in any of the first traffic images to obtain the first feature parameters of the first area.

A possible implementation manner, the first information M is:

λ为预设常数，

σ为预设标准差参数，☉为点乘运算，F(·)为傅里叶变换，F^-1(·)为傅里叶反变换，X为所述第一特征参数，X′为X的转置，w为所述第一区域的长，h为所述第一区域的宽，(i，j)为所述第一区域内的像素点坐标，(i_c，j_c)为所述第一区域内的中心像素点坐标。in,

λ is a preset constant,

σ is the preset standard deviation parameter, ☉ is the dot multiplication operation, F(·) is the Fourier transform, F ⁻¹ (·) is the inverse Fourier transform, X is the first characteristic parameter, and X′ is X The transpose of , w is the length of the first area, h is the width of the first area, (i, j) is the pixel coordinates in the first area, ( _{ic , j c )} is the The coordinates of the center pixel point in the first area.

In a possible implementation manner, the device further includes an update unit, which is specifically configured to use the length and width of the first area in the first traffic image to determine the second traffic image in the second traffic image. The center position of the signal light in the middle is the third area in the center of the area; the third information of the third area is determined; wherein the third information indicates the grayscale characteristics, gradient characteristics of each pixel in the third area, and all Describe the relationship between the grayscale of each pixel point and the grayscale of other pixel points; multiply and sum the template and the third information and the corresponding weight respectively to obtain the fourth information, and update based on the fourth information The template is used for the tracking of traffic lights in the next frame of traffic images.

In a third aspect, the present application provides a readable storage medium, comprising,

memory,

The memory is used to store instructions that, when executed by a processor, cause an apparatus including the readable storage medium to perform the method according to the first aspect and any possible implementation manner.

Description of drawings

1 is a flowchart of a method for tracking traffic lights provided by an embodiment of the present application;

2 is a schematic diagram of a first deep learning model used when a first area is framed for a first traffic image according to an embodiment of the present application;

FIG. 3 is a schematic diagram of tracking a signal light on a new traffic image by using a tracking method for a traffic signal provided by an embodiment of the present application;

FIG. 4 is a schematic structural diagram of a traffic signal light tracking device according to an embodiment of the present application.

Detailed ways

In view of the problem of low accuracy in judging the status of signal lights by electric police cameras in the prior art, the present application proposes a tracking method for traffic lights: based on target tracking algorithm and signal correlation principle, by tracking the coordinates of the center of traffic lights in traffic images, tracking Traffic lights; thus solving the problem that the electric police camera does not detect the deviation of the traffic lights, and still judges the status of the lights based on the original coordinates of the traffic lights, resulting in errors in judgment.

In order to better understand the above technical solutions, the technical solutions of the present application will be described in detail below through the accompanying drawings and specific embodiments. It is not a limitation on the technical solutions of the present application, and the embodiments of the present application and the technical features in the embodiments may be combined with each other under the condition of no conflict.

Referring to FIG. 1 , an embodiment of the present application provides a method for tracking traffic lights, which solves the problem of low accuracy in judging the state of a signal light by an electric police camera in the prior art. The method specifically includes the following implementation steps:

Step 101: Determine the position parameter and the first feature parameter of the first area where the traffic signal light is located in the first traffic image.

Wherein, the first area includes the coordinates of the center point of the traffic signal in the first traffic image, and the first feature parameter indicates the grayscale feature and gradient feature of the pixel in the first area.

Specifically, in order to speed up the image processing efficiency, after the first traffic image is acquired, the first area including the center point of the traffic signal can be framed and selected by the first deep learning model, and then the first area is processed. The first deep learning model may be a single-stage object detection model, which includes a backbone network, an augmentation network, and a detection network. As shown in Figure 2, in the backbone network, a CSP (Cross Stage Partial Network) residual structure can be set to extract high-dimensional features of the first traffic image. In the CSP structure, 5 stages can be set, each stage samples the first traffic image, and the resolution is doubled compared to the previous stage, and the number of channels is doubled; until the fifth stage, the first traffic image is sampled. The resolution of image sampling is reduced by a factor of 13, while the channels are expanded by a factor of 512. In the enhanced network, a PAN (Path Aggregation Network) structure can be set. In the PAN structure, two paths, top-down and bottom-up, are set for feature extraction and feature aggregation. The second function and the third function deeply fuse the abstract high-level features with the specific low-level features, which can further improve the accuracy of the image features expressed by the backbone network. Further, the first traffic image features obtained in the enhanced network can be input into the detection network, the detection network includes the first data header, the second data header, and the third data header, and the first area is obtained through the detection network. The characteristic parameters of the first area, the characteristic parameters may be the position coordinates of the first area; the position coordinates may include but are not limited to the following information: the center coordinates of the first area, the width and height of the first area, the status of the signal lights, and the confidence level; the confidence level Indicates the accuracy of the judged signal status. Further, the above-mentioned position coordinates are used as position parameters, and the purpose of frame selection of the first area in the first traffic image can be achieved according to the position parameters of the first area.

Further, before the feature extraction is performed on the first area, the first deep learning model may be trained to avoid errors in the first deep learning model, resulting in frame selection of the wrong first area in the first traffic image. A training method provided in the embodiment of the present application is as follows: first, the circumscribing rectangle of a traffic signal light is used as a selection frame, and the first area in the first traffic image is selected. Then, the standard location information of the first area is determined; wherein, the standard location information includes the coordinates of the center point of the first area. The standard location information may be the average value of the first area in the first traffic image of a set number selected by the box. It should be noted that, the extraction of the first traffic images may be continuous extraction, or may be extracted every set time until a set number of first traffic images are extracted. For example, for example, set the set number to 200, use the first deep learning model, select the first area in a box for these 200 first traffic images, and calculate the average width to length ratio of the first area as 1.0; The horizontal and vertical coordinates in the coordinates of the center point of each first region are averaged respectively, and the coordinates of the center point of the first region are determined to be (50, 40). Then, the width-length ratio interval can be set as (0.8-1.2) in the standard position information, and the coordinates of the center point are (50, 40). Finally, a set number of the first traffic images are continuously extracted, at least one first area is determined in the set number of first traffic images, and it is determined whether the location information of the at least one first area is consistent with the The standard location information is consistent. If not, it means that the parameters of the first deep learning model need to be further adjusted and optimized. If yes, use the location information of the first area in any first traffic image as the location parameter of the first area; and determine all the pixels in the first area in the any first traffic image The gray value and gradient value of the point are obtained to obtain the first characteristic parameter of the first region. Wherein, when determining whether the position information of the at least one first area is consistent with the standard position information, the method used may be to directly compare whether the coordinates of the center point meet the error requirements; or to calculate the intersection and ratio, and compare the intersection ratio with Set a threshold for comparison determination.

It is worth noting that the frame selection in the above-mentioned first traffic image includes the first area of the traffic signal light, but is not limited to the frame selection of the first area of the red, green and yellow lights in the traffic signal. For the identification of the color of traffic lights, identification rules can be set in the above detection network, that is, the image features obtained in the backbone network are mapped to classification information and corresponding confidence levels through the identification rules, and the confidence level is the largest and the confidence level exceeds the threshold. The classification information is determined as the type of traffic lights in the first area. The classification information refers to color information of traffic lights. Determining the type (color type) of the traffic light in this way can improve the efficiency and accuracy of photo forensics of the auxiliary electric police camera for illegal acts such as vehicle running through a red light.

Further, after the correct first region is selected by the box, the position parameter and the first characteristic parameter of the first region can be determined. Wherein, the representation method of the position parameter of the first region may be determined based on the shape of the first region. When the shape of the first area is a circle, the position parameter may be expressed based on the coordinates of the center point and the radius. When the shape of the first area is a square, the position parameter may be expressed based on the coordinates of the four vertices, or may be expressed based on the coordinates of the center point and the distance from the center point to the four vertices. The first feature parameter indicates the grayscale feature and the gradient feature of the pixel points in the first region; here, the gradient feature can be represented by the difference feature between the pixel points in the first region. The following provides methods for determining the grayscale features and gradient features of pixels in the first region:

Assuming that the coordinate of any pixel in the first area is (i, j), the calculation method of the grayscale feature I of the pixel whose coordinate is (i, j) is:

Among them, floor() is a round-down function, and r, g, and b correspond to the three channel data of the image, respectively.

The calculation method of gradient feature G is:

G _i (i,j)=I(i+1,j)-I(i-1,j);

G _j (i,j)=I(i,j+1)-I(i,j-1);

Among them, Gi is the gradient in the row direction, Gj is the gradient in the column direction, and G is the gradient value of the pixel point whose coordinates are (i, j) in the first region.

It can be seen that the gradient feature can be used to represent the change feature of the gray level in the first region. Both the grayscale feature and the gradient feature are composed of grayscale features and gradient features of all pixels in the first region, that is, the grayscale features and gradient features of the first region are both expressed in matrix form.

According to the grayscale feature and the gradient feature, the first feature parameter X of the pixel point (i, j) can be determined as:

X(i,j)=I(i,j)+G(i,j);

In the same way, the first characteristic parameter of each pixel in the first area can be calculated by the above formula, and the matrix formed by the first characteristic parameter of all the pixels in the first area can represent the first characteristic of the first area. parameter.

Step 102: Determine the first information of the first area in the first traffic image based on the first characteristic parameter.

The first information indicates, in the first traffic image, the grayscale feature and gradient feature of each pixel in the first area, and the relationship between the grayscale of each pixel and the grayscales of other pixels feature.

Specifically, in the embodiment of the present application, the first information is set as the target template, and the first information M is:

λ为预设常数，

σ为预设标准差参数，☉为点乘运算，F(·)为傅里叶变换，F^-1(·)为傅里叶反变换，X为所述第一特征参数，X′为X的转置，w为所述第一区域的长，h为所述第一区域的宽，(i，j)为所述第一区域内的像素点坐标，(i_c，j_c)为所述第一区域内的中心像素点坐标。上式中使用傅里叶变换可以起到提升计算效率的效果。in,

λ is a preset constant,

σ is the preset standard deviation parameter, ☉ is the dot multiplication operation, F(·) is the Fourier transform, F ⁻¹ (·) is the inverse Fourier transform, X is the first characteristic parameter, and X′ is X The transpose of , w is the length of the first area, h is the width of the first area, (i, j) is the pixel coordinates in the first area, ( _{ic , j c )} is the The coordinates of the center pixel point in the first area. The use of Fourier transform in the above formula can improve the computational efficiency.

Step 103: Determine a second area in the second traffic image and second information of the second area based on the location parameter of the first area.

Wherein, the second area includes the coordinates of the center point, and the second information indicates the grayscale characteristics and gradient characteristics of each pixel in the second area, as well as the grayscale of each pixel and other pixels in the second area. Grayscale relational features.

Specifically, according to the position parameters of the first area, a second area at the same position can be located in the second traffic image, and referring to the method of step 102, the characteristic parameters of the second area are extracted, and based on the characteristic parameters of the second area, the The second information M ₂ of the second area can be determined.

Step 104: Using the first information as a template, determine the correlation between the template and the second information, and determine that the location of the pixel corresponding to the maximum correlation value is the position of the signal light in the second traffic image. Central location.

Wherein, the correlation indicates the correlation between all the pixels in the first area in the first traffic image and all the pixels in the second area in the second traffic image.

Specifically, in this embodiment of the present application, the correlation between the template and the second information can be determined by using the target tracking algorithm and the principle of signal correlation. Based on the target tracking algorithm and the principle of signal correlation, the response of the correlation can use the following formula:

为第二区域的第二特征参数，

为相关性响应图所对应的高斯分布矩阵。in,

is the second characteristic parameter of the second region,

is the Gaussian distribution matrix corresponding to the correlation response map.

中值最大的元素为相关性最大值；若该值大于设定阈值，则确定交通信号灯跟踪成功。那么，该值所对应的像素点的所在位置为所述第二交通图像中信号灯的中心位置。若该值不大于设定阈值，则确定交通信号灯跟踪失败，第二区域中不包括交通信号灯的中心，则重新调整第一深度学习模型，以获取包括交通信号灯中心的第二区域。According to the principle that the Gaussian distribution signal starts to decay from the center of the traffic light to the surrounding area; as well as the target tracking algorithm and signal correlation: use the Gaussian kernel to calculate the correlation between two adjacent frames, and take the point with the largest response, that is, the maximum correlation value The principle that the corresponding pixel point is the predicted target center. In the embodiment of the present application, the first area in the first traffic image is used as a template, and in the template, the center of the traffic signal coincides with the center point of the first area. Then, to determine the point with the largest response between the first area and the second area in the second traffic image, it is actually to determine such a pixel point in the second area: the pixel point and the midpoint of the first area (that is, the midpoint of the traffic signal) The pixel features and gradient features are consistent, and the attenuation relationship between the feature intensity of the pixel and the feature intensities of other pixels in the region is consistent with the feature intensity relationship between the pixel at the center position in the first region and other pixels in the region. Further, the Gaussian distribution matrix in the second region

The element with the largest median value is the maximum correlation value; if the value is greater than the set threshold, it is determined that the traffic light tracking is successful. Then, the position of the pixel corresponding to this value is the center position of the signal light in the second traffic image. If the value is not greater than the set threshold, it is determined that the traffic signal tracking fails, and the center of the traffic signal is not included in the second area, and the first deep learning model is readjusted to obtain the second area including the center of the traffic signal.

Further, when the traffic light is tracked successfully and it is determined that the traffic light is deviated, the above template (ie, the first information) can be updated, thereby avoiding the problem of inaccurate judgment by the electric police camera due to the deviation of the traffic light. The method proposed in the embodiment of the present application for updating the template is as follows: first, using the length and width of the first area in the first traffic image, determine the center of the signal light in the second traffic image in the second traffic image The location is the third area in the center of the area.

Then, determine the third information of the third area; wherein the third information indicates the grayscale feature and gradient feature of each pixel in the third area, and the grayscale of each pixel and other pixels in the third area Grayscale relational features. Next, multiply and sum the above template and the third information and the corresponding weights respectively to obtain fourth information, and update the above template based on the fourth information, and the updated template can be used for the next frame of traffic images Tracking of traffic lights.

For example, assuming that the center coordinates of the traffic lights in the second traffic image are (i _y , j _y ), the third information M ^* is:

λ为预设常数，

σ为预设标准差参数，☉为点乘运算，F(·)为傅里叶变换，F^-1(·)为傅里叶反变换，X^*为所述第一特征参数，X^*′为X^*的转置，w为所述第三区域的长，h为第三区域的宽。此处第三区域的长和宽与第一区域、第二区域的长、宽一致。in,

λ is a preset constant,

σ is the preset standard deviation parameter, ☉ is the dot product operation, F( ) is the Fourier transform, F ^-1 ( ) is the inverse Fourier transform, X ^* is the first characteristic parameter, X ^* ' is the transpose of X ^* , w is the length of the third region, and h is the width of the third region. Here, the length and width of the third area are consistent with the length and width of the first area and the second area.

Then the updated first characteristic parameter is: X=(1-β)*X+β*X ^* ;

The updated first information is: M=(1-β)*M+β*M ^* .

Further, based on the method for tracking traffic lights described in steps 101 to 104, the following is an example description of the tracking of traffic lights in a new traffic image, please refer to FIG. 3:

Assume that an electric police camera is set directly opposite the traffic light at the intersection of the valley. The electric police camera is used to track the traffic light and determine the status of the tracked traffic light, so as to determine the violation/violation of the vehicle for evidence collection. . The first information (template) of the first traffic image in the embodiment of the present application may be a correlation filter determined based on the first area, which is used to determine the point with the highest response as the target position after correlating with the new target, that is, the correlation filter The pixel corresponding to the maximum value is the center point of the target. Then, after acquiring a new traffic image, it is first necessary to determine whether the detection area framed by the deep learning model is correct. The determination method includes, but is not limited to, comparing the location information, state parameters, and standard location information and standard state information of the detection area. That is, whether the color of the traffic lights in the detection area is consistent with the standard status information, and whether the center coordinates of the traffic lights in the detection area, the aspect ratio of the detection area, and the standard location information are consistent. If it is inconsistent, re-extract the next frame of image for determination. If it is still inconsistent, it is determined that the deep learning model and/or standard position information need to be adjusted, that is, after the model parameters of the deep learning model are determined, the standard position information and standard state are re-determined. information; that is, steps 101 to 103 are executed. If they are consistent, perform initialization to extract the second information of the detection area (determine the correlation filter of the current tracker based on the shape parameters and feature parameters of the current tracker), and the template of the first area of the previous traffic image (that is, the first information) to correlate to determine the center position of the traffic light in the detection area, and then to determine whether the traffic light is offset. That is, based on the target tracking algorithm and the principle of signal correlation, the correlation filter of the first area is used as a template, and the information of this area and the correlation filter of the first area are used to obtain the correlation, and the maximum response position, that is, the maximum correlation value, can be determined. The corresponding pixel is the center point of the signal light in the new traffic image. Then, based on the maximum response position, the position of the rectangular frame, that is, the position parameter of the area where the signal light is located, can be updated. The new rectangular frame position can be used to obtain the traffic image information of the next frame, and accurately obtain the state change of the signal light, so as to avoid the traffic light overflowing the target area, resulting in the error or inability to judge the status of the traffic signal.

Based on the same inventive concept, an embodiment of the present application provides a traffic signal light tracking device, which corresponds to the aforementioned traffic signal light tracking method shown in FIG. Repeated places will not be repeated. Referring to Figure 4, the device includes:

Parameter unit 401: used to determine the position parameter and the first characteristic parameter of the first area where the traffic light is located in the first traffic image; wherein, the first area includes the coordinates of the center point of the traffic light in the first traffic image, and the first A feature parameter indicates grayscale features and gradient features of pixels in the first region.

Specifically, the circumscribed rectangle of the traffic signal light is used as a selection box, and the first area in the first traffic image is selected; standard location information of the first area is determined; wherein, the standard location information includes all the coordinates of the center point of the first area; extract a set number of the first traffic images, determine at least one first area in each of the first traffic images, and determine the position of the at least one first area Whether the information is consistent with the standard location information; if so, take the location information of the first area in any one of the first traffic images as the location parameter of the first area; The gray values and gradient values of all pixels in the first area in a traffic image are used to obtain the first characteristic parameters of the first area.

Information unit 402: used to determine first information of a first area in the first traffic image based on the first feature parameter; wherein the first information indicates that in the first traffic image, the first The grayscale feature and gradient feature of each pixel in the area, and the relationship feature between the grayscale of each pixel and the grayscale of other pixels.

Specifically, the first information M is:

λ为预设常数，

σ为预设标准差参数，☉为点乘运算，F(·)为傅里叶变换，F^-1(·)为傅里叶反变换，X为所述第一特征参数，X′为X的转置，w为所述第一区域的长，h为所述第一区域的宽，(i，j)为所述第一区域内的像素点坐标，(i_c，j_c)为所述第一区域内的中心像素点坐标。in,

λ is a preset constant,

σ is the preset standard deviation parameter, ☉ is the dot multiplication operation, F(·) is the Fourier transform, F ⁻¹ (·) is the inverse Fourier transform, X is the first characteristic parameter, and X′ is X The transpose of , w is the length of the first area, h is the width of the first area, (i, j) is the pixel coordinates in the first area, ( _{ic , j c )} is the The coordinates of the center pixel point in the first area.

Determining unit 403: for determining a second area in the second traffic image and second information of the second area based on the position parameter of the first area; wherein the second area includes the center point The second information indicates the grayscale feature and gradient feature of each pixel in the second region, and the relationship feature between the grayscale of each pixel and the grayscale of other pixels.

Central unit 404: used to use the first information as a template, determine the correlation between the template and the second information, and determine the location of the pixel corresponding to the maximum correlation value as the second traffic image The center position of the signal light in the middle; wherein, the correlation indicates the correlation between all the pixels in the first area in the first traffic image and all the pixels in the second area in the second traffic image.

The central unit is specifically configured to determine the correlation between the template and the second information based on the target tracking algorithm and the signal correlation principle; If the maximum value in the property is greater than the set threshold, it is determined that the traffic light tracking is successful.

The traffic signal tracking device further includes an update unit, which is specifically configured to use the length and width of the first area in the first traffic image to determine in the second traffic image the signal light in the second traffic image. The center position is the third area in the center of the area; the third information of the third area is determined; wherein the third information indicates the grayscale feature and gradient feature of each pixel in the third area, and each of the The relationship between the grayscale of a pixel point and the grayscale of other pixel points; the template and the template are respectively multiplied and summed with the corresponding weights to obtain fourth information, and the template is updated based on the fourth information for Tracking of traffic lights in the next frame of traffic image.

Based on the same inventive concept, an embodiment of the present application also provides a readable storage medium, including:

memory,

The memory is used for storing instructions which, when executed by the processor, cause the apparatus including the readable storage medium to complete the traffic signal tracking method as described above.

Those skilled in the art can clearly understand that, for the convenience and conciseness of the description, only the division of the above-mentioned functional modules is used for illustration. In practical applications, the above-mentioned functions can be allocated to different functional modules as required. The internal structure of the device is divided into different functional modules to complete all or part of the functions described above. For the specific working process of the system, apparatus and unit described above, reference may be made to the corresponding process in the foregoing method embodiments, and details are not described herein again.

In the several embodiments provided by the present invention, it should be understood that the disclosed apparatus and method may be implemented in other manners. For example, the device embodiments described above are only illustrative. For example, the division of the modules or units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components may be Incorporation may either be integrated into another system, or some features may be omitted, or not implemented. On the other hand, the shown or discussed mutual coupling or direct coupling or communication connection may be through some interfaces, indirect coupling or communication connection of devices or units, and may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution in this embodiment.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit. The above-mentioned integrated units may be implemented in the form of hardware, or may be implemented in the form of software functional units.

The integrated unit, if implemented in the form of a software functional unit and sold or used as an independent product, may be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the present application can be embodied in the form of software products in essence, or the parts that contribute to the prior art, or all or part of the technical solutions, and the computer software products are stored in a storage medium , including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor (processor) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage media include: Universal Serial Bus flash disk (Universal Serial Bus flash disk), mobile hard disk, Read-Only Memory (ROM), Random Access Memory (Random Access Memory, RAM), magnetic disk Or various media such as optical discs that can store program codes.

It will be apparent to those skilled in the art that various modifications and variations can be made in the present invention without departing from the spirit and scope of the invention. Thus, provided that these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims (11)

1. A method of tracking a traffic signal, comprising:

determining a position parameter and a first characteristic parameter of a first area where a traffic signal lamp is located in a first traffic image; the first area comprises coordinates of a central point of a traffic signal lamp in a first traffic image, and the first characteristic parameters indicate gray characteristics and gradient characteristics of pixel points in the first area;

determining first information of a first area in the first traffic image based on the first characteristic parameter; the first information indicates gray scale characteristics and gradient characteristics of each pixel point in the first area and relation characteristics of gray scale of each pixel point and gray scale of other pixel points in the first traffic image;

determining a second area in a second traffic image and second information of the second area based on the position parameter of the first area; the second region comprises the central point coordinate, and the second information indicates the gray feature and the gradient feature of each pixel point in the second region and the relation feature of the gray of each pixel point and the gray of other pixel points;

taking the first information as a template, determining the correlation between the template and the second information, and determining the position of a pixel point corresponding to the maximum correlation value as the central position of a signal lamp in the second traffic image; wherein the correlation indicates a correlation between all pixel points in a first region in the first traffic image and all pixel points in a second region in the second traffic image.

2. The method of claim 1, wherein said determining the relevance of the template to the second information comprises:

determining the correlation of the template and the second information based on a target tracking algorithm and a signal correlation principle;

and comparing the maximum value in the correlation with a set threshold value, and determining that the traffic signal lamp is successfully tracked when the maximum value in the correlation is greater than the set threshold value.

3. The method of claim 1 or 2, wherein the determining of the position parameter and the first characteristic parameter of the first area in the first traffic image in which the traffic signal lamp is located comprises:

taking a circumscribed rectangle of a traffic signal lamp as a selection frame, and selecting the first area in the first traffic image;

determining standard position information of the first area; wherein the standard position information includes center point coordinates of the first area;

extracting a set number of the first traffic images, determining at least one first area in each of the first traffic images, and determining whether position information of the at least one first area is consistent with the standard position information;

if so, taking the position information of the first area in any one of the first traffic images as the position parameter of the first area; and determining gray values and gradient values of all pixel points in the first area in any one first traffic image to obtain a first characteristic parameter of the first area.

4. The method of claim 3, wherein the first information M is:

wherein,

the lambda is a preset constant, and the temperature of the reaction kettle is controlled,

sigma is a preset standard deviation parameter, ☉ is a dot product operation, F (-) is Fourier transform, F^-1(. DEG) is inverse Fourier transform, X is the first characteristic parameter, X' is the transposition of X, w is the length of the first area, h is the width of the first area, (i, j) is the coordinates of pixel points in the first area_c，j_c) And the coordinates of the central pixel points in the first area are obtained.

5. The method of claim 4, wherein after determining that the position of the pixel corresponding to the maximum correlation value is the center position of the signal light in the second traffic image, the method comprises:

determining a third area taking the central position of a signal lamp in the second traffic image as the center of the area in the second traffic image by using the length and the width of the first area in the first traffic image;

determining third information of the third area; the third information indicates the gray characteristic and the gradient characteristic of each pixel point in the third area and the relation characteristic of the gray of each pixel point and the gray of other pixel points;

and multiplying and summing the template and the third information by corresponding weights respectively to obtain fourth information, and updating the template based on the fourth information to be used for tracking the signal lamp in the next frame of traffic image.

6. A tracking device for a traffic signal, comprising:

a parameter unit: the first traffic image processing method comprises the steps of determining a position parameter and a first characteristic parameter of a first area where a traffic signal lamp is located in a first traffic image; the first area comprises coordinates of a central point of a traffic signal lamp in a first traffic image, and the first characteristic parameters indicate gray characteristics and gradient characteristics of pixel points in the first area;

information unit: first information for determining a first area in the first traffic image based on the first characteristic parameter; the first information indicates gray scale characteristics and gradient characteristics of each pixel point in the first area and relation characteristics of gray scale of each pixel point and gray scale of other pixel points in the first traffic image;

a determination unit: the second area in the second traffic image and second information of the second area are determined based on the position parameter of the first area; the second region comprises the central point coordinate, and the second information indicates the gray feature and the gradient feature of each pixel point in the second region and the relation feature of the gray of each pixel point and the gray of other pixel points;

a central unit: the second traffic image processing module is used for determining the correlation between the template and the second information by taking the first information as the template, and determining the position of a pixel point corresponding to the maximum value of the correlation as the central position of a signal lamp in the second traffic image; wherein the correlation indicates a correlation between all pixel points in a first region in the first traffic image and all pixel points in a second region in the second traffic image.

7. The apparatus according to claim 6, characterized in that said central unit is specifically adapted to determine the correlation of said template with said second information based on a target tracking algorithm and a signal correlation principle; and comparing the maximum value in the correlation with a set threshold value, and determining that the traffic signal lamp is successfully tracked when the maximum value in the correlation is greater than the set threshold value.

8. The device according to claim 6 or 7, characterized in that the parameter unit is specifically configured to select the first area in the first traffic image using a circumscribed rectangle of a traffic light as a selection box; determining standard position information of the first area; wherein the standard position information includes center point coordinates of the first area; extracting a set number of the first traffic images, determining at least one first area in each of the first traffic images, and determining whether position information of the at least one first area is consistent with the standard position information; if so, taking the position information of the first area in any one of the first traffic images as the position parameter of the first area; and determining gray values and gradient values of all pixel points in the first area in any one first traffic image to obtain a first characteristic parameter of the first area.

9. The apparatus as claimed in claim 8, wherein the first information M is:

wherein,

the lambda is a preset constant, and the temperature of the reaction kettle is controlled,

sigma is a preset standard deviation parameter, ☉ is a dot product operation, F (-) is Fourier transform, F^-1(. cndot.) is an inverse Fourier transform, X is the first characteristic parameter, X' is the transpose of X, w is the length of the first region, h is the width of the first region, (i, j) are coordinates of pixel points within the first region_c，j_c) And the coordinates of the central pixel point in the first area are obtained.

10. The apparatus according to claim 9, further comprising an updating unit, specifically configured to determine a third area in the second traffic image with the center position of the signal lamp in the second traffic image as the area center, by using the length and the width of the first area in the first traffic image; determining third information of the third area; the third information indicates the gray feature and the gradient feature of each pixel point in the third area and the relationship feature of the gray of each pixel point and the gray of other pixel points; and multiplying and summing the template and the third information by corresponding weights respectively to obtain fourth information, and updating the template based on the fourth information to be used for tracking the signal lamp in the next frame of traffic image.

11. A readable storage medium, comprising,

a memory for storing a plurality of data files to be transmitted,

the memory is for storing instructions that, when executed by the processor, cause an apparatus comprising the readable storage medium to perform the method of any of claims 1-5.

Images