CN111476999A - Over-the-horizon sensing system for intelligent networked vehicles based on vehicle-road multi-sensor collaboration - Google Patents

Abstract

The invention discloses an intelligent networked vehicle over-the-horizon perception system based on vehicle-road multi-sensor collaboration, including four subsystems: a roadside subsystem, a vehicle-mounted subsystem, a communication subsystem and a CPU subsystem. The system uses a variety of sensors to collect the data of the surrounding environment of the vehicle and the remote road information respectively, and matches the information of the surrounding environment of the vehicle with the remote road information, so that the vehicle can realize the fine perception of the surrounding environment and at the same time, the target outside the field of view can be detected. Perceive, obtain road traffic conditions ahead, obtain weather conditions, road conditions ahead and other information, so that intelligent networked vehicles can realize beyond-the-horizon perception in a higher viewing angle and a wider range.

Description

Over-the-horizon sensing system for intelligent networked vehicles based on vehicle-road multi-sensor collaboration

technical field

The invention relates to the technical field of intelligent networked transportation, and more particularly to an intelligent networked vehicle over-the-horizon perception system based on vehicle-road multi-sensor collaboration.

Background technique

At present, with the continuous breakthroughs in smart car-related technologies, intelligence has gradually become an important development direction and symbol of automotive technology. Vehicle environment perception technology is one of the key technologies for the development of cars to intelligence, and it is also the basis of smart cars. In particular, driverless technology and active safety technology, which have been repeatedly mentioned in recent years, rely on the detection and perception of the surrounding environment by vehicles. Vehicle perception technology is of great help in reducing traffic accidents, and can also improve transportation efficiency and vehicle capacity per unit time on the road by increasing the average speed of vehicles on the road and shortening the distance between vehicles. At the same time, it can also make the vehicle run more smoothly, greatly reduce the probability of sudden braking, save fuel consumption and reduce exhaust emissions.

The traditional vehicle environment perception technology mainly has two ways to perceive the external environment: visual perception and distance perception, to sense whether there are obstacles around the vehicle and the precise distance from the vehicle to the obstacle. In the traditional technology, the sensing range of the vehicle is limited to the performance of the on-board sensor, and the maximum sensing range is usually about 200 meters. At the same time, when the vehicle travels at a high speed, the perception range is smaller, and there are problems such as obstacles affecting the perception range, blocking the field of vision, etc., and the environmental perception range is limited and easily disturbed, and the ideal application effect has not been achieved.

Therefore, how to provide a vehicle over-the-horizon perception system with a large environmental perception range and strong anti-interference ability is an urgent problem to be solved by those skilled in the art.

SUMMARY OF THE INVENTION

In view of this, the present invention provides an intelligent networked vehicle over-the-horizon perception system based on vehicle-road multi-sensor collaboration, so as to solve the problem of insufficient sensing distance of current intelligent vehicles, realize vehicle over-the-horizon perception, and improve intelligent networked vehicles. Autonomous driving safety.

In order to achieve the above object, the present invention adopts the following technical solutions:

An over-the-horizon sensing system for intelligent networked vehicles based on vehicle-road multi-sensor collaboration, the system includes:

a roadside subsystem that collects environmental data around the road in real time;

an on-board subsystem, the on-board subsystem perceives information about the surrounding environment of the vehicle and locates the vehicle in real time;

a communication subsystem, which is used for receiving the environmental data around the road collected by the roadside subsystem and the vehicle surrounding environment information and positioning data collected by the vehicle-mounted subsystem, and uploading the data;

The CPU subsystem, which receives the data uploaded by the communication subsystem, performs information fusion on the environmental data around the road, the environmental information around the vehicle and the positioning data, and perceives the environmental information outside the vehicle's field of view.

Further, the roadside subsystem includes a road information acquisition module, a meteorological data acquisition module and a roadside data processing module, and the road information acquisition module and the meteorological data acquisition module are both connected in communication with the roadside data processing module;

The road information collection module collects the environmental information around the road in real time, the meteorological data collection module collects the meteorological data of the driving environment in real time, and the roadside data processing module integrates and analyzes the environmental information around the road and the meteorological data of the driving environment. and processing, and reporting the processed roadside data to the CPU subsystem through the communication subsystem.

Further, the road information collection module includes a roadside camera, a microwave radar and a UWB positioning device, the roadside camera is used to collect and identify objects on the road, and the millimeter-wave radar is used to collect the position and speed of the objects. The UWB positioning device is used to assist in locating the position of the target, and the data collected by the roadside camera, the millimeter-wave radar and the UWB positioning device are all sent to the roadside data processing module.

Specifically, the roadside camera is used to collect video information, identify objects such as pedestrians, vehicles, obstacles, traffic lights, and traffic signs on the road, and obtain information such as the type, shape, and location of the objects.

Further, the meteorological data collection module includes a weather detector and a photoreceptor, the weather detector is used to detect the wind speed and weather information of the driving environment, and the photoreceptor is used to collect the light intensity information of the driving environment.

Further, the on-board subsystem includes on-board sensors, GPS positioning equipment and on-board data processing modules;

The vehicle-mounted sensor is used to detect information related to the driving environment of the vehicle, and the GPS positioning device is used to locate the vehicle in real time; both the vehicle-mounted sensor and the GPS positioning device are electrically connected to the vehicle-mounted data processing module, so The vehicle-mounted data processing module is used to integrate, analyze and process the collected driving environment-related information and vehicle positioning data.

Further, the vehicle-mounted sensor includes a vehicle-mounted camera, a millimeter-wave radar, and a lidar, the vehicle-mounted camera is used to collect the target object shape data in the vehicle driving environment, and the millimeter-wave radar detects the position of the target object through reflected microwaves. With the speed information, the lidar obtains 3D data by receiving the returned laser cloud points to reconstruct the surrounding environment of the vehicle in 3D.

Further, the vehicle-mounted data processing module performs integrated analysis and processing on the collected driving environment-related information and vehicle positioning data, specifically including:

Establish a unified coordinate system, and convert the target shape data obtained by the vehicle camera, the position and speed detected by the millimeter-wave radar, and the three-dimensional data obtained by the lidar into a unified coordinate system;

Match the data collected by the vehicle camera, millimeter wave radar and lidar, match the same target identified by multiple sensors in a unified coordinate system, and accurately calibrate the position information of the target to obtain accurate environmental perception information. ;

Unify the image format information obtained by the vehicle camera, the data format information of the millimeter wave radar and the image format information obtained by the lidar, and calibrate the data information in the image information;

Redundant correction is performed on the information obtained by different sensors, and error information of any sensor is corrected.

Further, the communication mode adopted by the communication subsystem includes 5G, DSRC and WIFI communication. The subsystem realizes the real-time communication between the roadside subsystem and the vehicle subsystem, and sends the data to the CPU subsystem.

Specifically, the CPU subsystem uses deep learning to fuse the received multi-source heterogeneous information, and matches the information of the surrounding environment of the vehicle with the remote road information, so that the vehicle can realize the fine perception of the surrounding environment and the visual field. It can perceive external targets, obtain road traffic conditions ahead, and obtain information such as weather conditions and road conditions ahead, so that intelligent networked vehicles can realize beyond-the-horizon perception in a higher viewing angle and a wider range.

As can be seen from the above technical solutions, compared with the prior art, the present disclosure provides an intelligent networked vehicle over-the-horizon perception system based on vehicle-road multi-sensor collaboration, which collects the surrounding environment of the vehicle by using a variety of sensors. Data and remote road information, match the surrounding environment information of the vehicle with the remote road information, so that the vehicle can perceive the objects outside the field of vision while realizing the fine perception of the surrounding environment, obtain the road traffic conditions ahead, obtain the weather conditions, Information such as road conditions ahead, so that intelligent networked vehicles can realize beyond-the-horizon perception in a higher viewing angle and a wider range.

Description of drawings

In order to explain the embodiments of the present invention or the technical solutions in the prior art more clearly, the following briefly introduces the accompanying drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only It is an embodiment of the present invention. For those of ordinary skill in the art, other drawings can also be obtained according to the provided drawings without creative work.

1 is a schematic diagram of the structure of a vehicle-road multi-sensor collaboration based intelligent networked vehicle over-the-horizon perception system provided by the present invention;

FIG. 2 is a schematic structural diagram of a roadside subsystem in an embodiment of the present invention;

3 is a schematic diagram of the structure of the vehicle-mounted subsystem in the embodiment of the present invention;

4 is a schematic diagram of a data processing flow of an intelligent networked vehicle over-the-horizon perception system based on vehicle-road multi-sensor collaboration in an embodiment of the present invention;

FIG. 5 is a schematic diagram of a system implementing over-the-horizon perception of an intelligent connected vehicle for remote road congestion information in an embodiment of the present invention;

FIG. 6 is a schematic diagram of realizing beyond-the-horizon perception of an intelligent network-connected vehicle for out-of-sight target information in an embodiment of the present invention.

Detailed ways

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

Referring to FIG. 1, an embodiment of the present invention discloses an intelligent networked vehicle over-the-horizon perception system based on vehicle-road multi-sensor collaboration, the system includes:

Roadside subsystem 1, roadside subsystem 1 collects environmental data around the road in real time;

On-board subsystem 2, on-board subsystem 2 perceives the information of the surrounding environment of the vehicle, and locates the vehicle in real time;

Communication subsystem 3, the communication subsystem 3 is used to receive the environmental data around the road collected by the roadside subsystem 1 and the vehicle surrounding environment information and positioning data collected by the vehicle-mounted subsystem 2, and upload the data;

The CPU subsystem 4, the CPU subsystem 4 receives the data uploaded by the communication subsystem 3, performs information fusion on the environmental data around the road, the environmental information around the vehicle and the positioning data, and perceives the environmental information outside the vehicle's field of view.

In a specific embodiment, referring to FIG. 2, the roadside subsystem 1 includes a road information collection module 11, a meteorological data collection module 12, and a roadside data processing module 13. The road information collection module 11 and the weather data collection module 12 are both It is connected in communication with the roadside data processing module 13, the road information collection module 11 collects the environmental information around the road in real time, the meteorological data collection module 12 collects the meteorological data of the driving environment in real time, and the roadside data processing module 13 compares the environmental information around the road with the driving environment. The environmental meteorological data is integrated, analyzed and processed, and the processed roadside data is reported to the CPU subsystem 4 through the communication subsystem 3 .

In a specific embodiment, the road information collection module 11 includes a roadside camera 111 , a millimeter wave radar 112 and a UWB positioning device 113 , the roadside camera 111 is used to collect and identify objects on the road, and the millimeter wave radar 112 is used to collect The position and speed information of the target object, the UWB positioning device 113 is used to assist in locating the position of the target object, and the data collected by the roadside camera 111 , the millimeter wave radar 112 and the UWB positioning device 113 are all sent to the roadside data processing module 13 .

In this embodiment, the roadside subsystem includes a fixed type and a mobile type. For stationary systems, the sensor is located on the pole above the road; for mobile systems, the sensor is located on the pole at the top of the movable stand. The installation of the roadside system needs to be different according to the terrain and traffic requirements of the road. The method of coexistence of fixed and mobile is basically adopted. According to the sensing range of the sensor, a reasonable design should be carried out to ensure full coverage and perception of the road section.

Specifically, the roadside camera 111 is used to collect video information, identify objects such as pedestrians, vehicles, obstacles, traffic lights, and traffic signs on the road, and obtain information such as the type, shape, and location of the objects.

In a specific embodiment, the weather data collection module 12 includes a weather detector 121 and a photoreceptor 122. The weather detector 121 is used to detect the wind speed and weather information of the driving environment, and the photoreceptor 122 is used to collect the light intensity information of the driving environment. .

In a specific embodiment, referring to FIG. 3 , the on-board subsystem 2 includes on-board sensors 21 , GPS positioning equipment 22 and on-board data processing module 23 ;

The vehicle-mounted sensor 21 is used to detect the information related to the driving environment of the vehicle, and the GPS positioning device 22 is used to perform real-time positioning of the vehicle; It integrates, analyzes and processes the collected driving environment-related information and vehicle positioning data.

In a specific embodiment, the vehicle-mounted sensor 21 includes a vehicle-mounted camera 211, a millimeter-wave radar 212, and a lidar 213. The vehicle-mounted camera 211 is used to collect target shape data in the vehicle driving environment, and the millimeter-wave radar 212 uses reflected microwaves to Detecting the position and speed information of the target, the lidar 213 reconstructs the surrounding environment of the vehicle in 3D by receiving the returned laser cloud points to obtain 3D data.

In this embodiment, after each sensor collects environmental information, the in-vehicle subsystem identifies and preprocesses the data through its in-vehicle data processing module, and uses deep learning, SLAM methods and other methods to perform information fusion to achieve a detailed understanding of the surrounding environment of the vehicle. Perception, map building, and vehicle localization capabilities.

In a specific embodiment, the vehicle-mounted data processing module 23 performs integrated analysis and processing on the collected driving environment-related information and vehicle positioning data, specifically including:

Establish a unified coordinate system, and convert the target shape data obtained by the vehicle camera, the position and speed detected by the millimeter-wave radar, and the three-dimensional data obtained by the lidar into a unified coordinate system;

Match the data collected by the vehicle camera, millimeter wave radar and lidar, match the same target identified by multiple sensors in a unified coordinate system, and accurately calibrate the position information of the target to obtain accurate environmental perception information. ;

Unify the image format information obtained by the vehicle camera, the data format information of the millimeter wave radar and the image format information obtained by the lidar, and calibrate the data information in the image information;

Redundant correction is performed on the information obtained by different sensors, and error information of any sensor is corrected.

In a specific embodiment, the communication methods adopted by the communication subsystem 3 include 5G, DSRC, and WIFI communication. The subsystem realizes the real-time communication between the roadside subsystem 1 and the vehicle subsystem 2, and sends the data to the CPU subsystem 4.

Specifically, the CPU subsystem 4 uses deep learning to fuse the received multi-source heterogeneous information, and matches the information of the surrounding environment of the vehicle with the remote road information, so that the vehicle can realize the fine perception of the surrounding environment and the outside view. The target perceives, obtains the traffic conditions of the road ahead, and obtains information such as weather conditions and road conditions ahead, so that the intelligent networked vehicle can realize beyond-the-horizon perception at a higher viewing angle and a wider range.

The UWB (Ultra Wide Band) mentioned in this embodiment is also called an ultra-wideband technology, and the ultra-wideband technology is a wireless carrier communication technology using a frequency bandwidth above 1 GHz. It does not use a sine carrier, but uses nanosecond non-sinusoidal narrow pulses to transmit data, so it occupies a large spectrum range. Although wireless communication is used, its data transmission rate can reach hundreds of megabits per second or more. .

To sum up, the over-the-horizon sensing system for intelligent connected vehicles based on vehicle-road multi-sensor collaboration provided by the embodiments of the present invention has the following advantages compared with the prior art:

The system uses a variety of sensors to collect the data of the surrounding environment of the vehicle and the remote road information respectively, and matches the information of the surrounding environment of the vehicle with the remote road information, so that the vehicle can realize the fine perception of the surrounding environment and at the same time, the target outside the field of vision can be detected. Perceive, obtain road traffic conditions ahead, obtain weather conditions, road conditions ahead, and other information, so that intelligent networked vehicles can achieve beyond-the-horizon perception in a higher viewing angle and a wider range.

The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same and similar parts between the various embodiments can be referred to each other. As for the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant part can be referred to the description of the method.

The above description of the disclosed embodiments enables any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims (9)

1. an intelligent networked vehicle over-the-horizon perception system based on vehicle-road multi-sensor collaboration, is characterized in that, comprises: a roadside subsystem that collects environmental data around the road in real time; an on-board subsystem, which senses information about the surrounding environment of the vehicle and locates the vehicle in real time; a communication subsystem, which is used for receiving the environmental data around the road collected by the roadside subsystem and the vehicle surrounding environment information and positioning data collected by the vehicle-mounted subsystem, and uploading the data; The CPU subsystem, which receives the data uploaded by the communication subsystem, performs information fusion on the environmental data around the road, the environmental information around the vehicle and the positioning data, and perceives the environmental information outside the vehicle's field of vision. 2. The intelligent networked vehicle over-the-horizon perception system based on vehicle-road multi-sensor collaboration according to claim 1, wherein the roadside subsystem comprises a road information collection module, a meteorological data collection module, and a roadside data collection module. side data processing module; The road information collection module and the meteorological data collection module are both connected in communication with the roadside data processing module, the road information collection module collects the environmental information around the road in real time, and the meteorological data collection module collects the meteorological data of the driving environment in real time , the roadside data processing module integrates, analyzes and processes the environmental information around the road and the meteorological data of the driving environment, and reports the processed roadside data to the CPU subsystem through the communication subsystem. 3. A vehicle-road multi-sensor collaboration based intelligent networked vehicle over-the-horizon perception system according to claim 2, wherein the road information collection module comprises a roadside camera, a millimeter-wave radar and a UWB positioning device , the roadside camera is used to collect and identify the state information of the target on the road, the millimeter wave radar is used to collect the position and speed information of the target, the UWB positioning device is used to assist in locating the position of the target, the The data collected by the roadside camera, the millimeter wave radar and the UWB positioning device are all sent to the roadside data processing module. 4. The intelligent networked vehicle over-the-horizon perception system based on vehicle-road multi-sensor collaboration according to claim 2, wherein the meteorological data acquisition module comprises a meteorological detector and a photoreceptor, and the meteorological detector is used for The wind speed and weather information of the driving environment are detected, and the photoreceptor is used to collect the light intensity information of the driving environment. 5. A vehicle-road multi-sensor collaboration based intelligent network-connected vehicle over-the-horizon perception system according to claim 2, wherein the roadside data processing module combines the environmental information around the road and the weather of the driving environment. Data is integrated, analyzed and processed, including: Establish a unified coordinate system, and convert the status information of the objects on the road obtained by the roadside cameras and the position and speed information of the objects detected by the millimeter-wave radar into the unified coordinate system; Identify and preprocess the converted data, and use deep learning and Kalman filtering algorithms for information fusion; Accurately detect and identify the target, measure the dynamic target speed and position in real time, and locate the vehicle. 6. The intelligent network-connected vehicle over-the-horizon perception system based on vehicle-road multi-sensor collaboration according to claim 1, wherein the on-board subsystem comprises on-board sensors, GPS positioning equipment and on-board data processing modules; The vehicle-mounted sensor is used to detect information related to the driving environment of the vehicle, and the GPS positioning device is used to locate the vehicle in real time; both the vehicle-mounted sensor and the GPS positioning device are electrically connected to the vehicle-mounted data processing module, so The vehicle-mounted data processing module is used to integrate, analyze and process the collected driving environment-related information and vehicle positioning data. 7. The intelligent networked vehicle over-the-horizon perception system based on vehicle-road multi-sensor collaboration according to claim 6, wherein the on-board sensors comprise on-board cameras, millimeter-wave radars and lidars, and the on-board sensors The camera is used to collect the shape data of the target object in the driving environment of the vehicle. The millimeter-wave radar detects the position and speed information of the target object through the reflected microwave. 3D reconstruction to obtain 3D data. 8 . The intelligent networked vehicle over-the-horizon perception system based on vehicle-road multi-sensor collaboration according to claim 7 , wherein the on-board data processing module is used for the collected driving environment related information and vehicle positioning data. 9 . Perform integrated analysis and processing, including: Establish a unified coordinate system, and convert the target shape data obtained by the vehicle camera, the position and speed detected by the millimeter-wave radar, and the three-dimensional data obtained by the lidar into a unified coordinate system; Match the data collected by the vehicle camera, millimeter wave radar and lidar, match the same target identified by multiple sensors in a unified coordinate system, and accurately calibrate the position information of the target to obtain accurate environmental perception information. ; Unify the image format information obtained by the vehicle camera, the data format information of the millimeter wave radar and the image format information obtained by the lidar, and calibrate the data information in the image information; Redundant correction is performed on the information obtained by different sensors, and error information of any sensor is corrected. 9 . The over-the-horizon sensing system for intelligent connected vehicles based on vehicle-road multi-sensor collaboration according to claim 1 , wherein the communication methods adopted by the communication subsystem include 5G, DSRC and WIFI communication. 10 .

Images