CN114037969A

CN114037969A - Automatic driving lane information detection method based on radar point cloud and image fusion

Info

Publication number: CN114037969A
Application number: CN202111319426.XA
Authority: CN
Inventors: 张昌杰; 娄玉强; 刘凯
Original assignee: Jingzhou Zhida Electric Vehicle Co ltd
Current assignee: Jingzhou Zhida Electric Vehicle Co ltd
Priority date: 2021-11-09
Filing date: 2021-11-09
Publication date: 2022-02-11

Abstract

An automatic driving lane information detection method based on radar point cloud and image fusion relates to the technical field of automatic driving perception. According to the method, more accurate driving semantic information including a central line, a course angle, a stop line and the like is calculated through the lane line information detected in real time. The method has an exception handling function, can be used for handling the abnormal conditions of lane line identification errors, lane line loss and the like, and can prevent unknown errors from occurring in the running process of the vehicle. By deploying the method, the unmanned vehicle can sense semantic information of surrounding road surfaces in real time, assist the vehicle in driving tasks, and have good robustness and detection success rate.

Description

Automatic driving lane information detection method based on radar point cloud and image fusion

Technical Field

The invention relates to the technical field of automatic driving perception, in particular to an automatic driving lane information detection method based on radar point cloud and image fusion.

Background

The traditional lane line detection method is mainly based on an image method to detect lane lines and is divided into a traditional method and a deep learning method, and a hardware system depending on the traditional method is mainly a camera and a corresponding computing unit. In addition, in the laser radar-based lane line detection system, a laser radar sensor sends and receives laser pulses to form a radar point cloud image, the early radar point cloud-based lane line detection method is to separate lane line points and non-lane line points in point cloud by setting a point cloud reflectivity threshold, and a dependent hardware system mainly comprises a laser radar and a corresponding calculation unit.

In the traditional lane line detection system, the normal operation of a lane line detection algorithm based on images depends on good illumination conditions, and the accuracy of an algorithm model is obviously reduced in the case of complex road environments such as the loss of a lane line shielded by a vehicle and the like, so that the requirement of an unmanned driving task is difficult to meet; in the radar point cloud-based lane line detection system, false detection and missed detection are easily caused due to the discontinuity of radar point clouds, and meanwhile, models obtained by training point cloud data acquired in different models are difficult to match with other radars, so that the condition that the requirement of lane line detection cannot be met by a single lane line detection method is caused.

Disclosure of Invention

Aiming at the defects in the prior art, the invention aims to provide an automatic driving lane line detection method based on radar point cloud and image data fusion.

The technical scheme adopted by the invention is as follows: an automatic driving lane information detection method based on radar point cloud and image fusion is technically characterized by comprising the following steps:

acquiring data acquired by a laser radar, a camera and inertial navigation, and processing the acquired data;

acquiring a lane line detection result output by a lane line detection model and a segmentation result output by a lane mark segmentation model, and distributing a detection segmentation result obtained after the model runs;

generating a driving high-precision map through sensor data acquired in a test scene;

judging the road condition by combining high-precision map information and lane marks, and calculating driving semantic information through lane line information; and calculating to obtain the speed and steering parameters of the vehicle by using the road condition judgment result and the driving semantic information, and controlling the bottom layer of the vehicle.

In the scheme, the image adopts Gaussian filtering to preprocess input data, and the acquired sensor data is synchronized by using the timestamp.

In the above solution, the method for generating a high-precision map includes: the method comprises the steps of collecting sensor data of each road section in a segmented mode, using the data collected by different laser radars as a sequence, preprocessing the data, then carrying out lane identification segmentation on a plurality of sequences to obtain road surface information, superposing inertial navigation pose matrixes to obtain global point cloud with lane identification information, and finally obtaining a high-precision map of the sequences.

In the above solution, the lane mark segmentation model specifically includes: and segmenting point cloud data by adopting a U-Net network improved by taking an IOU (interaction-over-unity) as a loss function of the network to obtain a lane mark segmentation recognition result.

The invention has the beneficial effects that: according to the automatic driving lane information detection method based on radar point cloud and image fusion, more accurate driving semantic information including a central line, a course angle, a stop line and the like is calculated through lane line information detected in real time. The method has an exception handling function, can be used for handling the abnormal conditions of lane line identification errors, lane line loss and the like, and can prevent unknown errors from occurring in the running process of the vehicle. By deploying the method, the unmanned vehicle can sense semantic information of surrounding road surfaces in real time, assist vehicles in a sexual driving task, and have good robustness and detection success rate.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings needed in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings without creative efforts.

FIG. 1 is a block diagram of a software architecture according to an embodiment of the present invention;

FIG. 2 is a system data circuit diagram according to an embodiment of the present invention;

FIG. 3 is a schematic diagram of a lane marking detection algorithm in an embodiment of the present invention;

FIG. 4 is a high precision map module architecture diagram according to an embodiment of the present invention;

Detailed Description

The above objects, features and advantages of the present invention will become more apparent from the following detailed description of the present invention with reference to the accompanying drawings, which are illustrated in fig. 1 to 4, and the accompanying drawings.

The method for detecting the information of the automatic driving lane based on the radar point cloud and the image fusion is realized by the system architecture shown in fig. 1 and is connected by the data line shown in fig. 2. The system comprises a sensor group, a core calculation unit and an embedded calculation unit, wherein the sensor group is used for collecting and processing data and consists of a laser radar, a camera and an inertial navigation device, the core calculation unit and the embedded calculation unit are used for performing data calculation, and the 5G route, the sensors in the sensor group and the sensors in the sensor group are connected through a HUB (head-up-down bus). The embedded computing unit in this implementation is used to preprocess data. The core computing unit is used for operating the model. This embodiment may employ multiple lidar systems, each connected to the system via a HUB.

The automatic driving lane information detection method based on radar point cloud and image fusion in the embodiment can realize lane line detection, lane identification recognition and real-time construction of a driving high-precision map, is executed in a core computing unit, and specifically comprises the following steps:

step 1, sensor data acquisition and processing.

The method comprises the steps of firstly starting a data receiving node of a laser radar, a camera and an inertial navigation sensor, and receiving data acquired by each sensor, specifically point cloud data acquired by the laser radar, image data acquired by the camera and pose data acquired by the inertial navigation sensor. And sending the data of each sensor to a corresponding data preprocessing module, carrying out Gaussian filtering on the image data and adjusting the brightness saturation, carrying out noise reduction on the laser point cloud data, then synchronizing the data of the multiple sensors based on the time stamps, and simultaneously establishing software nodes to release the synchronized data of the multiple sensors.

And 2, detecting lane lines and identifying lane marks.

(1) And detecting the lane line. And the lane line detection model adopts the fused point cloud and image characteristics, extracts the characteristics through a Hourglass depth network, and then carries out lane line detection on the fused characteristics in an output branch. The specific flow is that a core computing unit firstly initializes a Hourglass network model, and then a subscription synchronous data node prepares to receive synchronized multi-sensor data. When the node receives the data, the calculation unit operates the lane line detection model and issues a detection segmentation result obtained after the model is operated on the ROS.

(2) And (5) lane mark segmentation and recognition. And adopting a lane line segmentation recognition model facing a high-precision map. And (4) selecting a point cloud range, taking the laser radar as a center, and intercepting the point cloud range within a certain range (20m by 10m) to obtain a point cloud aerial view serving as model input. The segmentation model adopts an improved U-Net network, the network model mainly comprises an encoder layer and a decoder layer, the encoder layer uses a convolution kernel of 7 x 7 to replace a conventional convolution kernel of 3 x 3, so that each convolution layer contains information with a large range, each encoder layer in the network executes convolution operation twice and maximum value pooling operation once, and after the convolution operation is finished, the encoder layer stores convolution output results and simultaneously transmits the output to the next encoder layer. In the decoder layer, each decoding layer performs up-sampling on the input feature diagram through deconvolution operation, then the deconvolution result is combined with the convolution result stored in the encoder layer, convolution operation is performed to obtain a new feature diagram, and the new feature diagram is sent to the next decoder layer. And finally, performing convolution operation on the output of the decoder layer by using a convolution kernel with the size of 1 x 1, and then obtaining a final segmentation result through the softmax layer. Compared with a U-Net model, the improved model adopts an IOU (interaction-over-Intersection) as a loss function of a network, and can have a better effect in a lane marking task. The IOU loss function is defined as follows:

therein, Loss_IOUAnd for the loss function, A is a set of lane marking points predicted by the network, and B is a set of lane marking points in the segmentation label.

Similar to lane line detection, the actual segmentation process is that a core computing unit firstly initializes a network model, and then subscribes a synchronous data node to prepare to receive synchronized multi-sensor data. When the node receives the data, the core computing unit operates the lane identification segmentation model and issues a segmentation result obtained after the model operates on software.

And 3, acquiring a high-precision map.

As shown in fig. 4, first, each road segment is acquired in segments, and the data of the lidar acquisition sensor is recorded, and the sequence is divided into a sequence 1, a sequence 2, and a sequence n, where each sequence represents data acquired by one lidar, that is, the sequence 1 is data acquired by a first lidar, the sequence 2 is data acquired by a second lidar, and so on. And then, processing each frame of data by using a preprocessing algorithm, wherein the processing comprises operations such as filtering, point cloud denoising and the like. And obtaining road surface information by using a plane fitting algorithm for the processed data, and projecting the point cloud data onto the plane. And the high-precision map module performs lane mark segmentation on each point cloud aerial view projection by using a lane mark segmentation model to acquire road surface information, and superimposes pose matrixes acquired by inertial navigation to obtain global point clouds with lane mark information. Because some errors can be generated by the segmentation algorithm, the pavement semantic information of the high-precision map cannot be guaranteed to be accurate, and therefore, the errors in the segmentation result are manually repaired through manual calibration by people, and the high-precision map corresponding to the sequence is obtained. The high-precision map can be obtained by performing the operation on the plurality of sequences and then projecting the result of each sequence to the world coordinate system through the pose acquired by inertial navigation.

And 4, judging and controlling the vehicle according to the lane line and the lane mark detection and recognition result.

When receiving lane line detection and segmentation data, the decision control module judges the road condition according to lane mark segmentation results by combining a high-precision map, and calculates deeper driving semantic information according to lane line information in a front view field. And finally, the decision control module calculates the speed and the steering parameters of the vehicle on the basis and sends the speed and the steering parameters to the bottom layer of the vehicle through a CAN bus for control.

The above description is only for the specific embodiments of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present invention, and all the changes or substitutions should be covered within the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the appended claims.

Claims

1. an automatic driving lane information detection method based on radar point cloud and image fusion, is characterized in that, comprises the following steps:

Acquire data acquired by lidar, camera and inertial navigation, and process the acquired data;

Obtain the lane line detection results output by the lane line detection model and the segmentation results output by the lane marking segmentation model, and publish the detection and segmentation results obtained after the model runs;

Generate a high-precision driving map through the sensor data collected in the test scene;

Combined with high-precision map information and lane markings, the road conditions are judged, and the driving semantic information is calculated through the lane line information; the speed and steering parameters of the vehicle are calculated by using the road condition judgment results and driving semantic information, and the bottom layer of the vehicle is controlled.

2. The automatic driving lane information detection method based on radar point cloud and image fusion according to claim 1, wherein the image adopts Gaussian filtering to preprocess the input data, and the acquired sensor data is synchronized by time stamp .

3. The automatic driving lane information detection method based on radar point cloud and image fusion according to claim 1, characterized in that, the method for generating a high-precision map is: collecting sensor data of each road section in sections and according to different The data collected by the lidar is used as a sequence, the above data is preprocessed, and then the lane markings are segmented for multiple sequences to obtain road surface information, and the inertial navigation pose matrix is superimposed to obtain a global point cloud with lane marking information. Obtain a high-resolution map of the sequence.

4. the automatic driving lane information detection method based on radar point cloud and image fusion according to claim 1, is characterized in that, described lane mark segmentation model, specifically refers to: adopt IOU as the loss function improvement of network The U-Net network segmented the point cloud data to obtain the segmentation and recognition results of the lane markings.