CN202022205U - Multisource information fusion device for safe driving assistance - Google Patents

Abstract

A multisource information fusion device for safe driving assistance consists of a vehicle-mounted processor, a radar, a video sensor, a navigator, a magnetic sensor, a program memory, a data memory, a hard disk, a video encoder, a voice decoder, an external clock chip, a first signal conditioning circuit and a second signal conditioning circuit, wherein the first signal conditioning circuit is composed of a high-speed data acquisition card and an analog-to-digital converter; the second signal conditioning circuit consists of a PCI (peripheral component interconnect) bridge interface chip and a CAN (controller area network) interface chip; the device can simultaneously acquire signals from the radar, the video sensor, the navigator and the magnetic sensor; multisource information analysis software is used to perform information fusion and decision making; and a decision result is prompted to a driver through a voice and image form, thereby overcoming the defects that the reliability is poor, the error is large, and information dead zones are available when a unisource sensor is used for acquiring information.

Description

Multi-source information fusion device for safe driving assistance

technical field

The utility model belongs to the field of vehicle safety auxiliary driving and relates to a multi-source information fusion device.

Background technique

With the development of the economy, the number of automobiles has increased rapidly, and traffic safety has become a problem that people pay more and more attention to. The number of people and property losses caused by traffic accidents worldwide every year is staggering. Facing the severe traffic safety situation, many countries are actively carrying out research on safety assisted driving.

In the past, safety assisted driving used the information fed back by single-source sensors to provide early warning signals or reminder signals for drivers. GPS positioning information is used to pass through dangerous road sections that are difficult to identify, such as road sections that are completely covered by snow, but single-source information has large limitations, large errors, and poor reliability. The ability of GPS positioning is very poor, and GPS positioning information is greatly affected by the weather, which is prone to deviation. Therefore, it is necessary to develop a multi-source information fusion device to solve the above problems.

Utility model content

The purpose of this utility model is to provide a multi-source information fusion device for safety assisted driving, which can simultaneously collect radar, video sensor, navigator, and magnetic sensor data, and perform information fusion and decision-making through multi-source information analysis software. The form of the image prompts the decision result to the driver, thereby overcoming the disadvantages of poor reliability of information obtained by single-source sensors, large errors and information blind spots.

In order to achieve the above object, the utility model adopts the following technical solutions:

A multi-source information fusion device for safety assisted driving, the device consists of a vehicle processor, radar, video sensor, navigator, magnetic sensor, program memory, data memory, hard disk, video encoder, voice decoder, external clock chip , the first signal conditioning circuit, the second signal conditioning circuit; it is characterized in that:

The video sensor consists of a camera and a video capture card with a 1394 interface. The output of the camera is connected to the input of the video capture card, and the output of the video capture card is connected to the No. 1 parallel peripheral interface of the on-board processor. The No. 2 parallel peripheral interface is connected with the input end of the video encoder; the output end of the video encoder is connected with the display; the audio interface of the vehicle-mounted processor is connected with the input end of the speech decoder, and the output end of the speech decoder is connected with the loudspeaker; The external clock chip is connected to the SPI interface of the vehicle processor; the first signal conditioning circuit is composed of a high-speed data acquisition card and an analog-to-digital converter; the magnetic sensor is connected to the high-speed data acquisition card, and the analog signal of the high-speed data acquisition card and the analog-digital converter The input end is connected, and the digital quantity output end of the analog-to-digital converter is connected with the I ² C bus interface of the vehicle-mounted processor; the second signal conditioning circuit is composed of a PCI bridge interface chip and a CAN interface chip; the radar is connected with the PCI bridge interface chip, and the PCI The bridge interface chip is connected to the CAN interface chip, and the data port, address port, and control port of the CAN interface chip are respectively connected to the data port, address port, and control port of the on-board processor through the address bus, data bus, and control bus. Connect with the address bus driver, the address bus driver is connected with the address port of the vehicle-mounted processor through the address bus, the data port of the hard disk is connected with the data bus driver, the data bus driver is connected with the data port of the vehicle-mounted processor through the data bus, the address bus driver, The control port of the data bus driver is connected to the control port of the vehicle-mounted processor through the control bus, and the data port, address port, and control port of the program memory are respectively connected to the data port and address port of the vehicle-mounted processor through the address bus, data bus, and control bus. , control port connection, the data port, address port, and control port of the data memory are respectively connected to the data port, address port, and control port of the on-board processor through the address bus, data bus, and control bus. Serial connection to the on-board processor.

The utility model has the following positive and beneficial effects:

This device uses the video sensor and radar to obtain the longitudinal information of the vehicle, that is, the information of the adjacent vehicle in front. The radar has a high accuracy in distance measurement. The video sensor can capture the outline of the object in front and analyze it with the image recognition software. The type of vehicle can be identified, and the advantages of the two complement each other, so that the type and distance of the vehicle in front can be well judged.

This device obtains the lateral position information of the vehicle by means of GPS/GIS navigator and magnetic sensor, that is, whether the vehicle is in the lane or not. The lateral position information of the vehicle is also very important for safe driving assistance. For steep and narrow snow-covered road sections and continuous sharp-turning road sections, magnetic track spikes can be laid on the road surface in advance, and magnetic sensors can be used to detect the position of the vehicle relative to the centerline of the lane, combined with GPS/GIS navigator, the two complement each other to reduce deviation and ensure The vehicle does not deviate from the lane.

This device has a wide range of application prospects, and can be used in the field of automatic driving, assisted driving, safe driving or other specific occasions. For example, it can provide accurate prompt information and early warning information for the driver. Increase driving safety.

Description of drawings

Fig. 1 is a schematic circuit diagram of the utility model.

Detailed ways

Please refer to Fig. 1, the utility model is a multi-source information fusion device for safety assisted driving, the device is composed of vehicle-mounted processor ADSP-BF561, radar, video sensor, navigator, magnetic sensor, program memory FLASH, data memory SDRAM , hard disk ATA-IDE, voice decoder, external clock chip.

The video sensor consists of a CCD camera and a video capture card with a 1394 interface. The output of the CCD camera is connected to the input of the video capture card, and the output of the video capture card is connected to the No. 1 parallel peripheral interface PPI1 of the vehicle processor ADSP-BF561 Connection, the video sensor uses the IEEE 1394 protocol as the output, the signal is stable, the image quality is high, and the color reproduction is good.

The second parallel peripheral interface PPI2 of the on-board processor ADSP-BF561 is connected to the input end of the video encoder ADV7171; the output end of the video encoder ADV7171 is connected to the display; the display device selects a color TFT-LCD module.

The audio interface SPORT of the on-board processor ADSP-BF561 is connected to the input end of the voice decoder AD1836, and the output end of the voice decoder AD1836 is connected to the speaker;

The external clock chip PCF2123 is connected to the SPI interface of the on-board processor; the first signal conditioning circuit is composed of a high-speed data acquisition card and an analog-to-digital converter, the magnetic sensor is connected to the high-speed data acquisition card, and the simulation of the high-speed data acquisition card and the analog-to-digital converter connected to the quantity input terminal, and the digital quantity output terminal of the analog-to-digital converter is connected to the I ² C bus interface of the vehicle-mounted processor.

The second signal conditioning circuit is composed of PCI bridge interface chip PCI9052 and CAN interface chip SJA1000, the radar is connected with the PCI bridge interface chip, the PCI bridge interface chip PCI9052 is connected with the CAN interface chip SJA1000, the data port, address port and control port of the CAN interface chip The data port, address port, and control port of the vehicle-mounted processor ADSP-BF561 are respectively connected through the address bus, data bus, and control bus.

The address port of the hard disk ATA-IDE is connected with the address bus driver (the model is 74HC245), and the address bus driver (the model is 74HC245) is connected with the address port of the vehicle processor ADSP-BF561 through the address bus.

The data port of the hard disk ATA-IDE is connected with the data bus driver (the model is SN74LVTH16245), and the data bus driver (the model is SN74LVTH16245) is connected with the data port of the vehicle processor ADSP-BF561 through the data bus.

The control ports of the address bus driver and the data bus driver are connected to the control port of the vehicle-mounted processor through the control bus.

The data port, address port, and control port of the program memory FLASH (model S29GL128P11TFI010) are respectively connected to the data port, address port, and control port of the on-board processor through the address bus, data bus, and control bus.

The data port, address port, and control port of the data memory SDRAM (model is MT48LC32M16A2P) are respectively connected to the data port, address port, and control port of the on-board processor through the address bus, data bus, and control bus.

The navigator is connected with the serial port UART of the vehicle processor through the RS232 level conversion chip MAX3232. The navigator adopts a commercially available product, and its interior is integrated with a GPS receiver and a GIS system. GPS is the abbreviation of the English Global Positioning System, and the Chinese abbreviation is the Global Positioning System. GIS is the abbreviation of English Geographic Information System, and the Chinese abbreviation is Geographic Information System. GIS information is pre-stored in the navigator. Its main working principle is to receive the GPS positioning signal through the GPS receiver, and then calculate the real-time longitude, latitude, speed and other positioning information of the vehicle through the internal processor of the navigator, and then store the positioning information into the database through the data interface. GIS information is stored, and the navigator transmits the vehicle's accurate position, speed, direction and surrounding detailed geographic environment information to the on-board processor ADSP-BF561 in real time.

The magnetic sensor is connected to the input port of the high-speed data acquisition card, the output port of the high-speed data acquisition card is connected to the analog signal input port of the analog-to-digital converter, and the digital signal output port of the analog-to-digital converter is connected to the I ² C bus interface of the vehicle processor connect.

The on-board processor ADSP-BF561 uses its parallel peripheral interface PPI1 to collect video data. The PPI1 has a DMA function and can perform high-speed data transmission without kernel intervention. After the transmission is completed, it can automatically send a DMA interrupt to the kernel.

The on-board processor ADSP-BF561 is a high-performance product launched by American Analog Devices. Its main frequency can reach up to 750MHz. Dual 16-channel DMA controllers provide extremely high data bandwidth and support line start and field start synchronization packets. decoding.

The on-board processor ADSP-BF561 has rich peripheral interfaces, including 4 128MB SDRAM interfaces, 4 1MB asynchronous memory interfaces, 3 timer/counters, 1 serial UART, 1 SPI interface, 2 synchronous memory interfaces, two Audio interface SPORT, 48 programmable general-purpose I/O interfaces.

The vehicle-mounted processor ADSP-BF561 provides two 16-bit parallel peripheral interfaces (PPI1 and PPI2). In the utility model, the No. 1 parallel peripheral interface PPI1 is used to connect with the video sensor, and the No. 2 parallel peripheral interface PPI2 is used to for connecting to a display. Each Parallel Peripheral Interface includes a dedicated clock pin, up to 3 frame sync pins and up to 16 data pins, in ITU-R656 mode, PPI1 or PPI2 provides half-duplex of 8 or 10-bit video data Work, two-way transmission.

The audio interface SPORT provided by the on-board processor ADSP-BF561 has two sending and receiving channels, supports I ² C data format, can process 8 stereo signals, supports 32-bit word width, and the data rate can reach 100Mb/s. This device uses SPORT realizes voice function.

The on-board processor ADSP-BF561 has a SPI-compatible port, which enables the on-board processor to communicate with multiple SPI-compatible devices. SPI (Serial Peripheral Interface--Serial Peripheral Interface) is a synchronous serial peripheral interface , using 3 pins to transmit data, including 2 data pins (master output-slave input MOSI and master input-slave output MISO) and a clock pin (serial clock SCK).

The on-board processor ADSP-BF561 has 1 SPI chip select input pin SPISS, allowing other SPI devices to select the on-board processor ADSP-BF561, and 7 SPI chip select output pins SPISEL7-1, enabling ADSP-BF561 to select other SPI equipment.

Due to the limited internal storage space of the on-board processor ADSP-BF561, an external FlASH must be connected to store the program. The program memory FLASH of this device selects the memory S29GL128P11TFI010 that can be rewritable by electricity and protects against power failure. The device is 8Mbit, and the data width can be configured as 8 bits or 16 bits. FLASH is used for system boot program loading and parameter saving; after each power-on or system reset, the system automatically downloads the program in the program memory FLASH to ADSP-BF561, and then starts to execute the program.

The video signal collected by the video sensor enters the vehicle-mounted processor ADSP-BF561 from the PPI1 interface, and is recognized by the image recognition software inside the vehicle-mounted processor, and the type of the vehicle in front is judged according to the outline of the vehicle in front. The amount of data is large, and the on-board processor ADSP-BF561 has limited memory, so the external SDRAM of the system is used as a data buffer. In terms of device selection, considering that the internal storage controller of the on-board processor ADSP-BF561 supports a maximum capacity of 128MB of single-chip SDRAM, the data memory SDRAM uses Hynix's MT48LC32M16A2P. This device is a synchronous high-speed dynamic memory that can meet For data buffering, its 16-bit data width can meet storage requirements. The clock CLK of MT48LC32M16A2P is provided by the system clock of ADSP-BF561: SDRAM is the frame memory of the system, which is used to store the original image collected and the intermediate result of algorithm processing.

Different connection devices are used between the four information sources of radar, video, navigator and magnetic sensor and the on-board processor. The radar uses a Can card based on the Can protocol, the video uses an interface based on the IEEE 1394 protocol, the navigator uses a serial port based on the RS232 protocol, and the magnetic sensor uses an A/D conversion card. In order to realize clock synchronization, a combination of interrupt mode, system clock and external clock is used. All hardware devices are stably fixed in the car and controlled by the on-board processor. The hardware composition of each module is shown in Figure 1.

The video sensor uses the German BASLER video sensor scA780-54fc, which has automatic exposure time, automatic gain and automatic white balance functions, and can adapt to changes in external light; progressive scan color CCD, resolution ≥ 780*580; pixel size 8.3um *8.3um; the maximum frame rate is not less than 55 frames per second; the signal output is 1394 signals; it has a synchronization function, and the exposure can be controlled by programming.

The radar uses the German UMRR-09xx-23-29ACC, the maximum detection range is 160 meters, when there is no relative moving target, the minimum detection range is 0.9 meters; when there is a relative moving target, the minimum detection range is 0.5 meters; azimuth : ≥±20°, the cycle interval is 40ms, the system works in the temperature range of -40°C to 85°C, and can work stably for a long time.

The data line of the radar is connected to the PCI bridge interface chip PCI9052, and connected to the vehicle processor through the SJA1000 chip. The SJA1000 chip is the CAN bus controller of Philips, and it supports the CAN2.0 protocol.

PCI bus is an advanced high-performance 32/64-bit local bus introduced by Intel Corporation. It can support multiple sets of peripheral devices at the same time. 132MB/s).

There are generally two schemes for realizing the PCI interface: using programmable logic devices and dedicated interface devices. The utility model adopts the PCI bus dedicated interface chip PCI9052 of PLX Company as the PCI interface chip in the CAN card, which is responsible for data communication with the vehicle-mounted processor.

PCI9052 is a PCI bus slave mode interface chip developed by PLX Company. It has low power consumption and complies with the PCI2.1 specification. Because PCI9052 can start the reading and writing of the local bus, the CAN card no longer needs a microcontroller, and the CAN communication controller can be used. The CAN card of this device adopts SJA1000, SJA1000 supports BasicCAN and PeliCAN mode, has FIFO, supports hot plug and other functions, not only can realize CAN bus interface function, but also the chip can output programmable CLKOUT signal according to the frequency of the crystal oscillator, the signal Just can be used as the bus frequency of the local bus of PCI9052, which saves devices and facilitates the design. The bus frequency of the CAN bus can be 12MHz, 16MHz or 24MHz, and the PCI9052 automatically realizes the access synchronization between the local bus and the PCI bus.

Because the hardware resources of PCI devices in the computer are dynamically allocated by the system, PCI configuration design is required. PCI9052 provides 5 local address spaces, choose one of them as the address space of SJA1000, and distribute 32 8-bit addresses. Set the corresponding initialization at the same time, the register PCIBAR2 in the PCI configuration register is set to 0XFFFFFFE0, and the values of other registers can be set to 0. Set the range of the local address space to 0X00000000~0X00000020. PCI9052 provides 2 local interrupt sources, SJA1000 provides level-triggered interrupt signals, and the interrupt trigger mode of PCI9052 is set as level-triggered. Use CS0 of PCI9052 as the chip selection signal of SJA1000. The initial value of the PCI9052 register is provided by FLASH, read after the PCI9052 is powered on.

The above-mentioned circuit is used for data acquisition of the radar frequency signal, and realizes the interface with the on-board processor, so as to facilitate the post-processing of the collected signal. The CAN card samples and quantizes the analog video signal sent by the radar receiver. After the range is merged, it is aligned relative to the main pulse, and then adds frame header information and transmits it to the driver through DMA.

The data acquisition accuracy of the radar is 12bits, and the data sampling frequency is 10MHz. From the perspective of signal processing, 10MHz speed and 12-bit accuracy are sufficient to ensure radar signal processing and the requirements for moving target detection.

The magnetic sensor adopts the existing mature technology. The data of the magnetic sensor is connected to Advantech’s PCI-1747 high-speed data acquisition card through the data line, and is transmitted to the on-board processor through the I ² C interface by using the analog-to-digital converter MAX1169. The analog-to-digital converter MAX1169 is a product of MAXIM Company. It is a 16-bit approximate continuous analog-to-digital converter. There is a 4MHz clock signal on the chip, which is compatible with the high-speed I ² C interface. PCI-1747 is a high-resolution multi-channel counting analog input PCI bus card. Its sampling rate can reach 250KS/s, PCI-1747 provides 64 single-ended, 32 differential analog inputs or combined inputs. It also has a 1K sample FIFO buffer for analog input data.

MAX3232 is a communication interface chip produced by TI Company in the United States that complies with the EIA/TIA-232 protocol, including two transmitters, two receivers and dual charging circuits. Featuring low power consumption, high data rates, and a proprietary low-dropout transmit output stage, it utilizes internal dual charge pumps to provide true RS232 performance from a +3.0V to +5.5V supply. When working on +3.3V power supply, the charge pump only needs four small capacitors of 0.1μF. The MAX3232 is guaranteed to maintain RS232 output levels at a data rate of 250kbps.

This device requires synchronous sampling and fusion of various information sources, so the real-time requirements for time are high, and the clock still runs when the system is powered off, so a clock chip with power-down detection function and additional battery power supply is selected. PCF2123.

PCF2123 is a low-power real-time clock chip produced by NXP Semiconductors, which has an SPI bus interface. The performance of the PC2123 real-time clock includes: a freely programmable alarm and timing function, which can generate a wake-up signal on the interrupt pin for the designer to choose. A programmable bias register enables fine-tuning of clock and frequency adjustments. Second, minute, hour, day, week, month and year registers are encoded in BCD code format, which can be easily converted to decimal numbers for display or printing. Data is transferred serially over an SPI bus at a maximum data rate of 6.25 megabits per second (Mbits).

The accuracy of the real time clock is guaranteed by the quartz crystal used. Each CMOS-based real-time clock/calendar uses a low-power 32.768kHz quartz oscillator to provide clock and calendar functions. The calendar function can record the year, month, day and week. The error of the quartz crystal itself and the influence of the surrounding environment on its timing accuracy can be easily corrected by adjusting the calibration register on the chip. This real-time clock does not require any other external components. The PCF2123 is a CMOS real-time clock and calendar optimized for low power consumption. It is a time manager that accurately measures time and periodically wakes up the sleeping main controller.

AD1836 is a high-performance voice decoder newly launched by ADI, which is suitable for digital audio system. It uses 5V power supply, and the input and output level of the digital interface is LVTTL level. In order to reduce signal interference, the input and output of analog signals are all in the form of differential. The system clock is 12.288MHz, the data sampling rate is up to 96kHz, and the number of sampling bits is the highest. It is 24 bits. By configuring the internal function registers, the corresponding functions can be selected, such as: clock, working mode, sampling rate, and number of sampling bits.

Considering the overall safety of the vehicle, the device simultaneously performs horizontal information fusion of the vehicle and longitudinal information fusion of the vehicle. Use radar and video sensors to carry out longitudinal information fusion of vehicles, according to the information obtained by radar and video sensors, fuse according to redundant fusion strategies, identify the type and distance of the vehicle in front, and use complementary fusion strategies to obtain this information from the navigator car location information.

For dangerous road sections that are difficult to identify, such as steep and narrow road sections completely covered by snow, and continuous sharp-turning road sections, magnetic track nails can be laid on the road surface in advance. When the vehicle passes through these road sections, the magnetic sensor is used to locate the vehicle laterally , so as to ensure that the vehicle does not deviate from the lane line, the horizontal fusion of the magnetic sensor and the navigator can inform the information of the curve ahead and the information of the dangerous road ahead, and can also be used as a supplement to the expansion of the vertical vision; from the perspective of the vehicle's horizontal information fusion , the use of magnetic sensors to position the vehicle laterally is also a supplement to the overall safety of the vehicle.

Longitudinal information fusion and horizontal information fusion make decisions through multi-source information analysis software, and the decision results are provided to the driver in the form of voice and images, thus providing guarantee for safe assisted driving.

This device has a wide range of application prospects, and can be used in the field of automatic driving, assisted driving, safe driving or other specific occasions. For example, it can provide accurate prompt information and early warning information for the driver. Increase driving safety.

Claims (1)

1. A multi-source information fusion device for safety assisted driving, which consists of a vehicle-mounted processor, radar, video sensor, navigator, magnetic sensor, program memory, data memory, hard disk, video encoder, voice decoder, external Composed of a clock chip, a first signal conditioning circuit, and a second signal conditioning circuit; its features are: The video sensor consists of a camera and a video capture card with a 1394 interface. The output of the camera is connected to the input of the video capture card, and the output of the video capture card is connected to the No. 1 parallel peripheral interface of the on-board processor. The No. 2 parallel peripheral interface is connected with the input end of the video encoder; the output end of the video encoder is connected with the display; the audio interface of the vehicle-mounted processor is connected with the input end of the speech decoder, and the output end of the speech decoder is connected with the loudspeaker; The external clock chip is connected to the SPI interface of the vehicle processor; the first signal conditioning circuit is composed of a high-speed data acquisition card and an analog-to-digital converter; the magnetic sensor is connected to the high-speed data acquisition card, and the analog signal of the high-speed data acquisition card and the analog-digital converter The input end is connected, and the digital quantity output end of the analog-to-digital converter is connected with the I ² C bus interface of the vehicle-mounted processor; the second signal conditioning circuit is composed of a PCI bridge interface chip and a CAN interface chip; the radar is connected with the PCI bridge interface chip, and the PCI The bridge interface chip is connected to the CAN interface chip, and the data port, address port, and control port of the CAN interface chip are respectively connected to the data port, address port, and control port of the on-board processor through the address bus, data bus, and control bus. Connect with the address bus driver, the address bus driver is connected with the address port of the vehicle-mounted processor through the address bus, the data port of the hard disk is connected with the data bus driver, the data bus driver is connected with the data port of the vehicle-mounted processor through the data bus, the address bus driver, The control port of the data bus driver is connected to the control port of the vehicle-mounted processor through the control bus, and the data port, address port, and control port of the program memory are respectively connected to the data port and address port of the vehicle-mounted processor through the address bus, data bus, and control bus. , control port connection, the data port, address port, and control port of the data memory are respectively connected to the data port, address port, and control port of the on-board processor through the address bus, data bus, and control bus. Serial connection to the on-board processor.

Images