CN103279759A - A Convolutional Neural Network-Based Analysis Method for Vehicle Front Passability - Google Patents

Abstract

The invention discloses a vehicle front trafficability analysis method based on a convolutional neural network, which comprises the following steps of: firstly, a large number of real vehicle running environment images are collected through a camera arranged in front of a vehicle; preprocessing the image by using a Gamma correction function; and training the convolutional neural network. The invention adopts the Gamma correction method of nonlinear function superposition to preprocess the image, avoids the influence of the illumination with strong variation on the target identification and improves the image resolution. The invention adopts a geometric normalization method, and reduces the resolution difference caused by the distance between the recognition target and the camera. The convolutional neural network LeNet-5 adopted by the invention can extract implicit characteristics with category resolution capability, and the extraction process is simple; LeNet-5 combines local receptive field, weight sharing and sub-sampling, ensures robustness to simple geometric deformation, reduces training parameters of the network, and simplifies the network structure.

Description

A Convolutional Neural Network-Based Analysis Method for Vehicle Front Passability

technical field

The invention belongs to the technical field of safety assisted driving and intelligent transportation, relates to a vehicle front passability analysis method, in particular to a vehicle front passability analysis method based on a convolutional neural network by collecting video images in front of a vehicle through a camera.

Background technique

The trafficability analysis in front of the vehicle belongs to the field of intelligent transportation facing the environment perception outside the vehicle. It refers to the analysis of the driving safety of the detected environment based on advanced means such as sensor technology, computer technology or communication technology, and finds out the existing safety hazards. Alert and warn drivers or lay the groundwork for driverless vehicle navigation. At present, based on the video image information in front of the vehicle collected by the camera, the research on the trafficability analysis using the visual image understanding method mainly includes obstacle detection, pedestrian detection, vehicle detection, road detection, traffic sign detection, terrain and landform classification, etc.

The visual image understanding methods involved in the feasibility analysis can be divided into reconstruction-based methods and recognition-based methods. Among them, the reconstruction-based method is based on 3D or 2.5D reconstruction technology, and judges whether it is passable from a spatial perspective. It is difficult to avoid the inherent serious ambiguity, small reconstruction range, and poor real-time performance of 3D reconstruction. In the recognition-based image understanding methods, there are mainly algorithms based on modeling and template matching, general neural networks, support vector machines, self-supervised learning, methods based on statistical learning, etc. These methods need to extract explicit features of the target, extract The process is complicated, and it is easy to cause loss of important information and poor environmental adaptability.

For the structured road environment with strong illumination changes, if the original image is directly recognized, there will be a lot of interference information, and the process of explicit feature extraction is complicated, and the distance between the target and the camera will cause a difference in resolution. In addition, changes in illumination can affect image quality and reduce image resolution.

Contents of the invention

In order to solve the above-mentioned problems existing in the prior art, the present invention proposes a vehicle front passability analysis method based on convolutional neural network, which can extract the implicit features of the target, the extraction process is simple, and avoids reducing the resolution of the image , and can reduce the influence of light, which is suitable for structured road environments with strong light changes.

The technical scheme of the present invention is: a method for analyzing the traffic ahead of a vehicle based on a convolutional neural network, comprising the following steps:

A. Image acquisition

First, a large number of real vehicle driving environment images are collected by a camera installed in front of the vehicle, and the image has m×n pixels; then the lower three-fifths of the image are obtained by cropping as the region of interest; finally, the cropped The image is converted to a grayscale image.

B. Image preprocessing

B1. Construct a Gamma correction function by superposition of non-linear functions, and correct the grayscale image obtained in step A. The specific function formula is as follows:

G(x)＝1+f ₁ (x)+f ₂ (x)+f ₃ (x) (1)

f ₁ (x)=acos(πx/255) (2)

f ₂ (x)=(K(x)+b)cosβ+xsinα (3)

K(x)＝ρsin(4πx/255) (4)

α＝arctan(-2b/255) (5)

f ₃ (x)＝R(x)cos(3πx/255) (6)

R(x)＝c|2x/255-1| (7)

In the formula, x is the gray value of a certain pixel, G(x) represents the Gamma correction value corresponding to a certain gray value, a∈(0,1) is a weighting coefficient, and b represents f ₂ (x) The maximum variation range, ρ represents the amplitude of K(x), α represents the deflection angle of K(x), c represents the magnitude of R(x), and a+b+c<1 is satisfied.

The formula for calculating the gray value after Gamma correction is:

g(x)＝255(x/255) ^1/G(x) (8)

In the formula, g(x) represents the gray value of a certain pixel after Gamma correction.

After Gamma correction, the grayscale image P is obtained.

B2. For the grayscale image P, change the grayscale value of some pixels, the specific change method is as follows:

Select the pixels with a gray value of 0 in the image area other than the vehicle and road boundaries in the image, change their gray value to 1, and select the gray value in the image area other than the vehicle and road boundaries in the image 255 pixels, change its grayscale value to 254; change the grayscale value of the pixel points in the vehicle area in the image to 0, and change the grayscale value of the pixels in the road boundary area to 255, and the image after changing the pixels is gray degree image Q. So far, the pixels of the grayscale image Q include three categories: the first category is pixels with a grayscale value of 0, which represent vehicles; the second category is pixels with a grayscale value of 255, which represent road boundaries; the third category is Except for the pixels whose gray value is 0 and 255, it represents the road surface. The above three types of pixels are assigned corresponding labels, that is, the label "0" is assigned to the first type of pixel, indicating "vehicle", and the label "1" is assigned to the second type of pixel, indicating "road boundary". The label "2" is assigned to the third type of pixel, representing "road". Finally, the label of each pixel in the grayscale image Q is assigned to the corresponding pixel in the grayscale image P.

B3. Normalize the size of the grayscale image P:

B31, along the image height direction, select different pixel rows at intervals, denoted by x, and obtain the pixel width and height of the corresponding target of different pixel rows x through actual sampling measurement;

B32. Taking the image area with a pixel height of 0 to 32 in the image as the reference image area, and taking the pixel width W and height H of the target to be recognized in the reference image area as the reference, that is, the horizontal and vertical cropping of the reference image area is set The scale coefficients are all 1; use W and H to remove the width and height of the target on the remaining pixel rows, respectively, to obtain two sets of ratios, which are represented by Y and Z respectively;

B33. Finally, fit the pixel row x to the two groups of ratios Y and Z to obtain two fitting curves, as shown below:

Y=k ₁ x+b ₁ (9)

Z=k ₂ x+b ₂ (10)

Among them, Y represents the horizontal cropping ratio coefficient of the image, Z represents the vertical cropping ratio coefficient of the image, x represents a certain pixel row of the image, k ₁ and k ₂ represent the slopes of the two fitting curves respectively, and b ₁ and b ₂ represent the slopes of the two fitting curves respectively. The intercept of the fitted curve.

B34. Set both the horizontal and vertical cropping ratios of the reference image area to 1, that is, crop the reference image area into image samples of 32×32 pixels. As x increases, the horizontal and vertical crop sizes obtained according to formulas (9) and (10) also increase accordingly. By cropping, a series of image samples of different sizes are obtained, and finally the cropped image samples are uniformly scaled to a 32×32 pixel image. The obtained 32×32 pixel image is used as the training sample of the convolutional neural network.

C. Training of Convolutional Neural Networks

The typical convolutional neural network LeNet-5 consists of 8 layers, the input layer is a 32×32 pixel image; the network layers C1, C3 and C5 represent the convolutional layer respectively, the network layers S2 and S4 are sub-sampling layers, and the network layer F5 is In the fully connected layer, the number of neurons in the output layer is the same as the number of target categories to be identified, which can be changed according to the actual application environment. A facet in each layer of the network represents a feature map, which is a collection of neurons whose weights are shared in each layer. Neurons in each layer are only connected to neurons in a local receptive field of the previous layer.

The general form of a convolutional layer is:

${\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}^{\mathrm{<span\; class="notranslate">\; l</span>}}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; (</span>\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; \&Element;</span>{\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}^{\mathrm{<span\; class="notranslate">\; l</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; *</span>{\mathrm{<span\; class="notranslate">\; k</span>}}_{\mathrm{<span\; class="notranslate">\; ij</span>}}^{\mathrm{<span\; class="notranslate">\; l</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; b</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}^{\mathrm{<span\; class="notranslate">\; l</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 11</span>}<span\; class="notranslate">\; )</span>$

where l ∈ {1, 2, 3, 4, 5, 6, 7, 8} represents the number of layers, k is the convolution kernel, M _j represents a selection of input feature maps, and b represents the bias.

The general form of the subsampling layer is:

${\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}^{\mathrm{<span\; class="notranslate">\; l</span>}}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; \&beta;</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}^{\mathrm{<span\; class="notranslate">\; l</span>}}\mathrm{<span\; class="notranslate">\; down</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}^{\mathrm{<span\; class="notranslate">\; l</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; b</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}^{\mathrm{<span\; class="notranslate">\; l</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 12</span>}<span\; class="notranslate">\; )</span>$

In the formula, down(·) represents the sub-sampling function, which generally sums an n×n area of the previous layer image, β represents the weight of the sub-sampling layer, and b represents the bias.

According to the actual application environment, the number of output neurons of LeNet-5 is adjusted, and then the image samples with a pixel size of 32×32 obtained in step B are used for training. Through training, when the error between the output value of the convolutional neural network and the expected value is within an acceptable range, the convolutional neural network that can be used for the analysis of the trafficability ahead of the vehicle is obtained.

Compared with prior art, effect and benefit of the present invention are:

1. The present invention adopts the Gamma correction method of non-linear function superposition to preprocess the image, which avoids the influence of strongly changing illumination on target recognition and improves the image resolution.

2. The present invention adopts a geometric normalization method, which reduces the difference in resolution caused by the distance between the recognition target and the camera.

3. The convolutional neural network LeNet-5 adopted in the present invention can extract implicit features with category resolution capabilities, and the extraction process is simple; LeNet-5 combines local receptive fields, weight sharing and sub-sampling to ensure the accuracy of simple geometric deformations. Robustness, and reduce the training parameters of the network, simplify the network structure; the number of neurons in the output layer of LeNet-5 can be adjusted according to the actual application environment, and the environment adaptability is strong.

Description of drawings

The present invention has 3 accompanying drawings, wherein:

Figure 1 Flowchart of the method for analyzing the traversability ahead of vehicles based on convolutional neural network

Figure 2 Convolutional neural network LeNet-5 structure diagram.

Figure 3 Sample set for convolutional neural network training.

Detailed ways

The present invention will be further described below in conjunction with the accompanying drawings. Figure 1 is a flow chart of a method for analyzing the front traversability of a vehicle based on a convolutional neural network. The present invention takes the structured environment of the expressway as an example, and divides the environment in front of the vehicle into vehicles, road boundaries and road surfaces.

The analysis process of the present invention includes: image acquisition, image preprocessing, and convolutional neural network training.

A. Image acquisition

A large number of real highway driving environment images (640×480 pixels) are collected by the camera installed in front of the vehicle, and then the lower three-fifths of the image are used as the region of interest (640×288 pixels) to reduce the follow-up workload ; Finally, convert the cropped image to a grayscale image.

B. Image preprocessing

In the first step, the Gamma correction method has certain advantages in reducing the influence of light. Generally, when the Gamma value is greater than 1, the highlight part of the image is compressed and the dark part is expanded; when the Gamma value is less than 1, the highlight part of the image is expanded and the dark part is compressed. Construct a Gamma correction function by superposition of non-linear functions, and correct the grayscale image obtained in step A. The function formula is as follows:

G(x)＝1+f ₁ (x)+f ₂ (x)+f ₃ (x) (1)

f ₁ (x)=acos(πx/255) (2)

f ₂ (x)=(K(x)+b)cosβ+xsinα (3)

K(x)＝ρsin(4πx/255) (4)

α＝arctan(-2b/255) (5)

f ₃ (x)＝R(x)cos(3πx/255) (6)

R(x)＝c|2x/255-1| (7)

In the formula, x is the gray value of a certain pixel, G(x) represents the Gamma correction value corresponding to a certain gray value, a∈(0,1) is a weighting coefficient, and b represents f ₂ (x) The maximum variation range, ρ represents the amplitude of K(x), α represents the deflection angle of K(x), c represents the magnitude of R(x), and a+b+c<1 is satisfied. In general, a<b≤c, a=0.2, b=0.3, c=0.3 are advisable.

The formula for calculating the gray value after Gamma correction is:

g(x)＝255(x/255) ^1/G(x) (8)

In the formula, g(x) represents the gray value of a certain pixel after Gamma correction.

After Gamma correction, the grayscale image P is obtained.

In the second step, for the grayscale image P that has been corrected by Gamma, select the pixels with a grayscale value of 0 in the image except for the vehicle and road boundaries, and change the grayscale value of these pixels to 1 by programming, and select the image In addition to the pixels with a gray value of 255 except for the vehicle and road boundaries, the gray value of these pixels is changed to 254 by programming; then the gray value of the pixel points in the vehicle area in the image is changed to 0 by programming, The gray value of the pixels in the road boundary area is changed to 255, and the gray image Q is obtained. So far, the pixels of the grayscale image Q include three categories: the first category is pixels with a grayscale value of 0, which represent vehicles; the second category is pixels with a grayscale value of 255, which represent road boundaries; the third category is Except for the pixels whose gray value is 0 and 255, it represents the road surface. By programming, assign corresponding labels to the above three types of pixels, that is, assign the label "0" to the first type of pixel, indicating "vehicle", and assign the label "1" to the second type of pixel, indicating "road boundary" , assign the label "2" to the third type of pixel, which means "road". Finally, the label of each pixel in the image Q is assigned to the corresponding pixel in the grayscale image P by programming.

The third step is to normalize the size of the grayscale image P:

In the image, the number of pixels occupied by the same target is greatly affected by its own distance from the camera, that is, the number of pixels occupied by any target is inversely proportional to its distance from the camera. The present invention proposes a new geometric normalization method, which normalizes the size of the grayscale image, avoids reducing the resolution of the image, and reduces the difference in resolution caused by the distance between the recognition target and the camera. First, in the height of the image, different pixel rows are selected at intervals, represented by vector x, x={15 32 60 65 68 72 75 82 85 87 92 96 100 108 111 113 124 130 138 143 150 160}, measured by actual sampling, Obtain the pixel width and height of the target corresponding to different pixel rows; secondly, take the image area with a pixel height of 0 to 32 in the image as a reference, and take the pixel width W and height H of the target in the reference image area as a benchmark, that is, the reference The horizontal and vertical cropping ratio coefficients of the image area are both 1; finally, use W and H to remove the width and height of the target on the remaining pixel rows, respectively, to obtain two sets of ratios, which are represented by Y and Z, and then the pixel rows are programmed to The x data is fitted with the Y and Z data separately to obtain two fitting curves, as shown below:

Y＝0.0312x-0.8339 (9)

Z=0.0360x-1.0590 (10)

Wherein, Y represents the image horizontal cropping ratio factor, Z represents the image vertical cropping ratio coefficient, and x represents a certain pixel row of the image.

The horizontal and vertical cropping ratios of the reference image area are both 1, that is, the reference image area is cropped into image samples of 32×32 pixels. As x increases, the horizontal and vertical crop sizes obtained according to formulas (9) and (10) also increase accordingly. Through cropping, a series of image samples of different sizes are obtained, and finally the cropped image samples are uniformly scaled to a 32×32 pixel image by programming. The obtained 32×32 pixel image is used as the training sample of the convolutional neural network.

C. Training of Convolutional Neural Networks

The present invention adopts the structure of a typical convolutional neural network LeNet-5 as shown in FIG. 2 . A typical convolutional neural network LeNet-5 consists of 8 layers, and the input image is 32×32 pixels; network layers C1, C3, and C5 represent convolutional layers, network layers S2, S4 are sub-sampling layers, and network layer F5 is a fully connected layer , the number of neurons in the output layer is the same as the number of target categories to be recognized, which can be changed according to the actual application environment. A facet in each layer of the network represents a feature map, which is a collection of neurons whose weights are shared in each layer. Neurons in each layer are only connected to neurons in a local receptive field of the previous layer (counting from the input layer).

The convolutional layer C1 consists of 6 feature maps with a size of 28×28. Each neuron of the feature map is connected to a 5×5 neighborhood of the input image. The convolutional layer C1 contains 156 trainable parameters and 122304 A trainable connection. The sub-sampling layer S2 is composed of 6 feature maps with a size of 14×14. Each neuron in the feature map is connected to a neighborhood with a size of 2×2 in the convolutional layer C1. The sub-sampling layer S2 has 12 possible neurons. training parameters and 5880 trainable connections. The convolutional layer C3 consists of 16 feature maps with a size of 10×10. Each neuron of the feature map is connected to a 5×5 neighborhood of the sub-sampling layer S2. The convolutional layer C3 contains 1516 trainable parameters. and 151600 trainable connections. The sub-sampling layer S4 is composed of 16 feature maps with a size of 5×5. Each neuron of the feature map is connected to a neighborhood with a size of 2×2 of the convolutional layer C3. The sub-sampling layer S4 contains 32 neurons. Trainable parameters and 2000 trainable connections. The convolutional layer C5 consists of 120 feature maps. Each neuron of the feature map is connected to the 5×5 neighborhood of all the feature maps of the sub-sampling layer S4. The convolutional layer C5 contains 48120 trainable parameters and 48120 A trainable connection. The network layer F6 is fully connected with the convolutional layer C5, and the fully connected layer F6 contains 10164 trainable parameters. The output layer is composed of radial basis function units.

The general form of a convolutional layer is:

${\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}^{\mathrm{<span\; class="notranslate">\; l</span>}}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; (</span>\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; \&Element;</span>{\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}^{\mathrm{<span\; class="notranslate">\; l</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; *</span>{\mathrm{<span\; class="notranslate">\; k</span>}}_{\mathrm{<span\; class="notranslate">\; ij</span>}}^{\mathrm{<span\; class="notranslate">\; l</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; b</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}^{\mathrm{<span\; class="notranslate">\; l</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 11</span>}<span\; class="notranslate">\; )</span>$

In the formula, l represents the number of layers, k is the convolution kernel, M _j represents a selection of input feature maps, and b represents the bias.

The general form of the subsampling layer is:

${\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}^{\mathrm{<span\; class="notranslate">\; l</span>}}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; \&beta;</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}^{\mathrm{<span\; class="notranslate">\; l</span>}}\mathrm{<span\; class="notranslate">\; down</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}^{\mathrm{<span\; class="notranslate">\; l</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; b</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}^{\mathrm{<span\; class="notranslate">\; l</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 12</span>}<span\; class="notranslate">\; )</span>$

In the formula, down(·) represents the sub-sampling function, which generally sums an n×n area of the previous layer image, β represents the weight of the sub-sampling layer, and b represents the bias.

The present invention takes the highway environment as an example. The environment in front of the vehicle is divided into three parts: the road surface, the vehicle, and the road boundary. Therefore, the number of output neurons of LeNe-5 should be set to 3. The output of 0 means that the recognition target is a vehicle, and the output of 1 means that the recognition target is a vehicle. The recognition target is the road boundary, and the output 2 represents the recognition target is the road surface. The size of the sample set for network training is 5000, and some samples are shown in Figure 3. The selection of the initial weight of the network is generated by random method. Using this known classification mode to train the convolutional network, the network has the ability to map between input and output pairs. Through training, if the error between the output value and the expected value of the convolutional neural network is within an acceptable range, the convolutional neural network that can be used for the analysis of the traffic ahead of the vehicle is obtained.

Claims (1)

1. a method for analyzing the traversability ahead of a vehicle based on a convolutional neural network, characterized in that: comprise the following steps: A. Image acquisition First, a large number of real vehicle driving environment images are collected by a camera installed in front of the vehicle, and the image has m×n pixels; then the lower three-fifths of the image are obtained by cropping as the region of interest; finally, the cropped The image is converted to a grayscale image; B. Image preprocessing B1. Construct a Gamma correction function by superposition of non-linear functions, and correct the grayscale image obtained in step A. The specific function formula is as follows: G(x)＝1+f ₁ (x)+f ₂ (x)+f ₃ (x) (1) f ₁ (x)=acos(πx/255) (2) f ₂ (x)=(K(x)+b)cosβ+xsinα (3) K(x)＝ρsin(4πx/255) (4) α＝arctan(-2b/255) (5) f ₃ (x)＝R(x)cos(3πx/255) (6) R(x)＝c|2x/255-1| (7) In the formula, x is the gray value of a certain pixel, G(x) represents the Gamma correction value corresponding to a certain gray value, a∈(0,1) is a weighting coefficient, and b represents f ₂ (x) The maximum variation range, ρ represents the amplitude of K(x), α represents the deflection angle of K(x), c represents the amplitude of R(x), and satisfies a+b+c<1; The formula for calculating the gray value after Gamma correction is: g(x)＝255(x/255) ^1/G(x) (8) In the formula, g(x) represents the gray value of a pixel after Gamma correction; After Gamma correction, the grayscale image P is obtained; B2, B2, for the grayscale image P, change the grayscale value of some pixels, the specific change method is as follows: Select the pixels with a gray value of 0 in the image area other than the vehicle and road boundaries in the image, change their gray value to 1, and select the gray value in the image area other than the vehicle and road boundaries in the image 255 pixels, change its grayscale value to 254; change the grayscale value of the pixel points in the vehicle area in the image to 0, and change the grayscale value of the pixels in the road boundary area to 255, and the image after changing the pixels is gray So far, the pixels of the grayscale image Q include three categories: the first category is pixels with a grayscale value of 0, which represent vehicles; the second category is pixels with a grayscale value of 255, which represent road boundaries; The third category is to remove the pixels with a gray value of 0 and 255, which represent the road surface; the above three types of pixels are assigned corresponding labels, that is, the label "0" is assigned to the first type of pixels, which means "vehicle" , assign the label "1" to the second type of pixel point, indicating "road boundary", assign the label "2" to the third type of pixel point, indicating "road surface"; finally assign the label of each pixel point in the grayscale image Q Assign to the corresponding pixel in the grayscale image P; B3. Normalize the size of the grayscale image P: B31, along the image height direction, select different pixel rows at intervals, denoted by x, and obtain the pixel width and height of the corresponding target of different pixel rows x through actual sampling measurement; B32. Taking the image area with a pixel height of 0 to 32 in the image as the reference image area, and taking the pixel width W and height H of the target to be recognized in the reference image area as the reference, that is, the horizontal and vertical cropping of the reference image area is set The scale coefficients are all 1; use W and H to remove the width and height of the target on the remaining pixel rows, respectively, to obtain two sets of ratios, which are represented by Y and Z respectively; B33. Finally, fit the pixel row x to the two groups of ratios Y and Z to obtain two fitting curves, as shown below: Y=k ₁ x+b ₁ (9) Z=k ₂ x+b ₂ (10) Among them, Y represents the horizontal cropping ratio coefficient of the image, Z represents the vertical cropping ratio coefficient of the image, x represents a certain pixel row of the image, k ₁ and k ₂ represent the slopes of the two fitting curves respectively, and b ₁ and b ₂ represent the slopes of the two fitting curves respectively. The intercept of the fitted curve; B34. Set both the horizontal and vertical cropping ratios of the reference image area to 1, that is, to crop the reference image area into an image sample of 32×32 pixels; as x increases, the horizontal And the vertical cropping size is also increased accordingly; by cropping, a series of image samples of different sizes are obtained, and finally the cropped image samples are uniformly scaled to a 32×32 pixel image; the obtained 32×32 pixel image is used as a volume The training samples of the product neural network; C. Training of Convolutional Neural Networks The typical convolutional neural network LeNet-5 consists of 8 layers, the input layer is a 32×32 pixel image; the network layers C1, C3 and C5 represent the convolutional layer respectively, the network layers S2 and S4 are sub-sampling layers, and the network layer F5 is In the fully connected layer, the number of neurons in the output layer is the same as the number of target categories to be recognized, which can be changed according to the actual application environment; a surface of each network layer represents a feature map, which is shared by weights in each layer A collection of neurons; the neurons of each layer are only connected to the neurons of a local receptive field of the previous layer; The general form of a convolutional layer is: ${\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}^{\mathrm{<span\; class="notranslate">\; l</span>}}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; (</span>\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; \&Element;</span>{\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}^{\mathrm{<span\; class="notranslate">\; l</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; *</span>{\mathrm{<span\; class="notranslate">\; k</span>}}_{\mathrm{<span\; class="notranslate">\; ij</span>}}^{\mathrm{<span\; class="notranslate">\; l</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; b</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}^{\mathrm{<span\; class="notranslate">\; l</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 11</span>}<span\; class="notranslate">\; )</span>$ In the formula, l∈{1,2,3,4,5,6,7,8} represents the number of layers, k is the convolution kernel, M _j represents a selection of input feature maps, and b represents the bias; The general form of the subsampling layer is: ${\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}^{\mathrm{<span\; class="notranslate">\; l</span>}}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; \&beta;</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}^{\mathrm{<span\; class="notranslate">\; l</span>}}\mathrm{<span\; class="notranslate">\; down</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}^{\mathrm{<span\; class="notranslate">\; l</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; b</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}^{\mathrm{<span\; class="notranslate">\; l</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 12</span>}<span\; class="notranslate">\; )</span>$ In the formula, down( ) represents the sub-sampling function, which generally sums an n×n area of the previous layer image, β represents the weight of the sub-sampling layer, and b represents the bias; According to the actual application environment, the number of output neurons of LeNet-5 is adjusted, and then the image samples with a pixel size of 32×32 obtained in step B are used for training; through training, when the output value of the convolutional neural network and the expected value If the error is within an acceptable range, a convolutional neural network can be obtained that can be used for the analysis of the vehicle's front passability.

Images