CN119722732A

CN119722732A - Obstacle tracking method, device and vehicle

Info

Publication number: CN119722732A
Application number: CN202311285199.2A
Authority: CN
Inventors: 牛宝龙
Original assignee: Xiaomi Automobile Technology Co Ltd
Current assignee: Xiaomi Automobile Technology Co Ltd
Priority date: 2023-09-28
Filing date: 2023-09-28
Publication date: 2025-03-28

Abstract

The present disclosure relates to an obstacle tracking method, device and electronic device, wherein the method comprises: obtaining 3D target frame information of each obstacle around a vehicle at a current time point, and 3D detection frame information of each obstacle around the vehicle at each historical time point; for each obstacle, inputting the 3D detection frame information of the obstacle at each historical time point into a tracking filter of the obstacle, and determining the 3D prediction frame information of the obstacle at the current time point; determining the obstacle to which each 3D target frame information belongs at the current time point according to each 3D target frame information at the current time point and the 3D prediction frame information of each obstacle, for updating the tracking filter of each obstacle, and determining the updated 3D prediction frame information of the obstacle at the current time point, and then tracking the obstacle, so as to realize 3D obstacle detection in multiple shooting device scenarios, improve obstacle detection efficiency, and then improve obstacle tracking efficiency.

Description

Obstacle tracking method and device and vehicle

Technical Field

The disclosure relates to the technical field of automatic driving, and in particular relates to a method and a device for tracking an obstacle and a vehicle.

Background

The existing obstacle tracking method mainly comprises the steps of obtaining peripheral images of a vehicle at each time point, which are obtained by shooting by a single shooting device, carrying out target detection processing on the peripheral images at each time point, determining 2D detection frame information of the obstacle at each time point, and carrying out matching processing on the 2D detection frame information of the obstacle at the adjacent time point to determine whether the same obstacle corresponds to the same obstacle.

The scheme is suitable for 2D obstacle detection based on single shooting equipment, and is difficult to be suitable for 3D obstacle detection under a scene of a plurality of shooting equipment.

Disclosure of Invention

The disclosure provides an obstacle tracking method, an obstacle tracking device and a vehicle.

According to a first aspect of an embodiment of the present disclosure, there is provided an obstacle tracking method, including obtaining 3D target frame information of a periphery of a vehicle at a current time point and 3D detection frame information of obstacles around the vehicle at a historical time point, determining the 3D target frame information according to a plurality of peripheral images of the vehicle at the current time point, inputting the 3D detection frame information of the obstacles at the historical time point into a tracking filter of the obstacles for each obstacle, determining 3D prediction frame information of the obstacles at the current time point, determining the obstacles to which the 3D target frame information belongs according to the 3D target frame information and the 3D prediction frame information of the obstacles at the current time point, updating the tracking filter of the obstacles according to the 3D prediction frame information of the obstacles at the current time point and the obstacles to which the 3D target frame information belongs, obtaining updated tracking filters of the obstacles, and processing the updated tracking filters of the obstacles for each obstacle according to the 3D prediction frame information of the obstacles at the current time point and the previous time point, and processing the updated tracking frame information of the obstacles.

In one embodiment of the disclosure, acquiring each piece of 3D target frame information of a vehicle periphery at a current time point comprises acquiring a plurality of periphery images of the vehicle at the current time point, acquiring an obstacle detection model by a plurality of photographing devices at the vehicle, inputting the periphery images into the obstacle detection model, acquiring a plurality of pieces of 3D candidate frame information and confidence degrees corresponding to the 3D candidate frame information, and filtering the plurality of pieces of 3D candidate frame information in combination with the confidence degrees to obtain each piece of 3D target frame information at the current time point.

In one embodiment of the disclosure, the obstacle detection model is obtained by combining a plurality of surrounding images of the vehicle at a historical time point and obstacle information of the periphery of the vehicle in a vehicle body coordinate system at the historical time point, the plurality of surrounding images are input into the obstacle detection model to obtain a plurality of 3D candidate frame information and confidence degrees corresponding to the 3D candidate frame information, the obstacle detection model is input with the plurality of surrounding images to obtain a plurality of 3D candidate frame information and corresponding confidence degrees in the vehicle body coordinate system, and coordinate conversion processing is performed by combining the position information of the vehicle in the world coordinate system at the current time point and the plurality of 3D candidate frame information in the vehicle body coordinate system to obtain a plurality of 3D candidate frame information in the world coordinate system.

In one embodiment of the disclosure, filtering the plurality of 3D candidate frame information in combination with the confidence coefficient to obtain each 3D target frame information at the current time point, wherein the filtering includes performing ground projection processing on each 3D candidate frame information to obtain ground 2D candidate frame information corresponding to each 3D candidate frame information, sorting each ground 2D candidate frame information in descending order according to the confidence coefficient corresponding to each 3D candidate frame information to obtain a sorting result, extracting ground 2D candidate frame information with the highest confidence coefficient from the sorting result, deleting ground 2D candidate frame information with a cross-over ratio with the ground 2D candidate frame information being greater than or equal to a first cross-over ratio threshold in the sorting result, re-executing the steps until the ground 2D candidate frame information processing in the sorting result is completed, and taking the 3D candidate frame information corresponding to the extracted ground 2D candidate frame information as each 3D target frame information at the current time point.

In one embodiment of the disclosure, the 3D target frame information and the 3D detection frame information include an obstacle category, the determining, according to the 3D target frame information of each 3D target frame at the current time point and the 3D prediction frame information of each obstacle, an obstacle to which each 3D target frame information belongs at the current time point includes acquiring, for each obstacle, 3D target frame information to be compared in each 3D target frame information, wherein the obstacle category in the 3D target frame information to be compared is consistent with the obstacle category, determining a matching degree between the 3D prediction frame information of the obstacle and each 3D target frame information to be compared, and determining an obstacle to which the 3D target frame information to be compared corresponding to the largest matching degree in a plurality of matching degrees belongs as the obstacle.

In one embodiment of the disclosure, the 3D target frame information and the 3D target frame information further comprise position information, the determining of the matching degree between the 3D predicted frame information of the obstacle and each 3D target frame information to be compared comprises performing ground projection processing on the 3D predicted frame information of the obstacle and each 3D target frame information to be compared to obtain ground 2D predicted frame information corresponding to the 3D predicted frame information of the obstacle and ground 2D target frame information corresponding to each 3D target frame information to be compared, determining distance data between the 3D predicted frame information of the obstacle and the 3D target frame information to be compared according to the position information in the 3D target frame information to be compared and the position information in the 3D predicted frame information of the obstacle, and determining of the matching degree between the 3D predicted frame information of the obstacle and the ground 2D target frame information to be compared according to the ground 2D predicted frame information of the obstacle and the ground 2D predicted frame information of the obstacle, and determining of the matching degree between the 3D predicted frame information of the obstacle and the ground 2D target frame information to be compared according to the ground 2D predicted frame information of the obstacle and the ground predicted frame information to be compared.

In one embodiment of the disclosure, the method for determining the matching degree between the 3D predicted frame information of the obstacle and each 3D target frame information to be compared further comprises determining image 2D predicted frame information corresponding to the 3D predicted frame information of the obstacle and each image 2D target frame information corresponding to the 3D target frame information to be compared by combining the position information of the vehicle in the world coordinate system at the current time point and the position information of a plurality of shooting devices on the vehicle relative to the vehicle, determining image areas corresponding to each image 2D target frame information and image areas corresponding to the image 2D predicted frame information by combining a plurality of peripheral images of the vehicle at the current time point, determining cross-correlation data between the image 2D target frame information and the image 2D predicted frame information, and the image similarity data between the image areas corresponding to the image 2D target frame information and the image areas corresponding to the image 2D predicted frame information, determining the cross-correlation data between the image areas corresponding to the image 2D target frame information and the image 2D predicted frame information according to the image 2D target frame information, and the cross-correlation data between the image 2D target frame information and the image 2D predicted frame information, and the image area corresponding to the image 2D target frame information, and the image area matching data.

In one embodiment of the disclosure, the determining the image 2D predicted frame information corresponding to the 3D predicted frame information of the obstacle and the image 2D target frame information corresponding to the 3D target frame information of each of the to-be-compared objects by combining the position information of the vehicle in the world coordinate system at the current time point and the position information of the plurality of photographing devices in the world coordinate system of the vehicle relative to the vehicle, determining the position information of the plurality of photographing devices in the world coordinate system at the current time point, determining the photographing devices corresponding to the 3D target frame information of each of the to-be-compared objects and the photographing devices corresponding to the 3D predicted frame information of the obstacle according to the position information of the plurality of photographing devices, determining the coordinate conversion relation between the image coordinate system of each photographing device and the coordinate system of each of the to-be-compared object, and determining the coordinate conversion relation between the image coordinate system of each photographing device and the world coordinate system of each of the 3D predicted frame information of the obstacle according to the position information of each of the photographing devices, and determining the 3D predicted frame information of each of the to-be-to-be-compared object.

In one embodiment of the present disclosure, the method further comprises determining updated 3D prediction frame information of the obstacle at the current point in time as 3D detection frame information of the obstacle at the current point in time.

According to a second aspect of the embodiments of the present disclosure, there is further provided an obstacle tracking device, which includes an acquisition module configured to acquire 3D target frame information of a periphery of a vehicle at a current time point and 3D detection frame information of a periphery of the vehicle at each historical time point, the 3D target frame information being determined according to a plurality of peripheral images of the vehicle at the current time point, a first determination module configured to input 3D detection frame information of the obstacle at each historical time point into a tracking filter of the obstacle for each obstacle, determine 3D prediction frame information of the obstacle at the current time point, a second determination module configured to determine an obstacle of each 3D target frame information at the current time point according to each 3D target frame information at the current time point and 3D prediction frame information of each obstacle, an update module configured to update the 3D prediction frame information of each obstacle at the current time point and the obstacle to which each 3D target frame information belongs, update the filter for each obstacle to obtain a current time point, and update the 3D prediction frame information of each obstacle for each obstacle before the current time point is input to the tracking filter.

According to a third aspect of embodiments of the present disclosure there is also provided a vehicle comprising a processor, a memory for storing instructions executable by the processor, wherein the processor is configured to implement the steps of the obstacle tracking method as described above.

According to a fourth aspect of embodiments of the present disclosure, there is also provided a non-transitory computer-readable storage medium, which when executed by a processor, causes the processor to perform the obstacle tracking method as described above.

The technical scheme provided by the embodiment of the disclosure at least brings the following beneficial effects:

The method comprises the steps of obtaining 3D target frame information of the periphery of a vehicle at a current time point and 3D detection frame information of obstacles around the vehicle at historical time points, determining the 3D target frame information according to a plurality of peripheral images of the vehicle at the current time point, inputting the 3D detection frame information of the obstacles at the historical time points into tracking filters of the obstacles for each obstacle, determining the 3D prediction frame information of the obstacles at the current time point, determining the obstacles which the 3D target frame information of the obstacles belongs to according to the 3D target frame information of the obstacles at the current time point and the 3D prediction frame information of the obstacles, updating the tracking filters of the obstacles according to the 3D prediction frame information of the obstacles at the current time point and the obstacles which the 3D target frame information of the obstacles belong to, obtaining updated tracking filters of the obstacles at the current time point and the previous 3D detection frame information of the obstacles at the historical time points, inputting the updated tracking filters of the obstacles, obtaining the updated 3D prediction frame information of the obstacles at the current time point, and further improving the 3D prediction frame information of the obstacles, and further improving the 3D detection efficiency of the obstacles.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the disclosure and together with the description, serve to explain the principles of the disclosure and do not constitute an undue limitation on the disclosure.

FIG. 1 is a flow chart of a method of obstacle tracking according to one embodiment of the present disclosure;

FIG. 2 is a flow chart of an obstacle tracking method according to another embodiment of the disclosure;

FIG. 3 is a flow chart of an obstacle tracking method according to another embodiment of the disclosure;

FIG. 4 is a schematic diagram of an obstacle tracking device according to one embodiment of the disclosure;

Fig. 5 is a block diagram of a vehicle according to an exemplary embodiment of the present disclosure.

Detailed Description

In order to enable those skilled in the art to better understand the technical solutions of the present disclosure, the technical solutions of the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings.

It should be noted that the terms "first," "second," and the like in the description and claims of the present disclosure and in the foregoing figures are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments of the disclosure described herein may be capable of operation in sequences other than those illustrated or described herein. The implementations described in the following exemplary examples are not representative of all implementations consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with some aspects of the present disclosure as detailed in the accompanying claims.

Fig. 1 is a flowchart of an obstacle tracking method according to one embodiment of the present disclosure. It should be noted that, the obstacle tracking method of the present embodiment may be applied to an obstacle tracking device, and the device may be configured in an electronic apparatus, so that the electronic apparatus may perform an obstacle tracking function.

The electronic device may be any device with computing capability, for example, may be a personal computer (Personal Computer, abbreviated as PC), a mobile terminal, a server, a controller in a vehicle, and the mobile terminal may be a hardware device with various operating systems, touch screens and/or display screens, such as a vehicle-mounted device, a mobile phone, a tablet computer, a personal digital assistant, and a wearable device. In the following embodiments, an execution body is described as an example of an electronic device.

As shown in fig. 1, the method comprises the steps of:

And 101, acquiring 3D target frame information of the periphery of the vehicle at the current time point and 3D detection frame information of the obstacles of the periphery of the vehicle at each historical time point, wherein the 3D target frame information is determined according to a plurality of peripheral images of the vehicle at the current time point.

In the embodiment of the present disclosure, a plurality of photographing apparatuses may be provided on a vehicle, and positions, photographing angle ranges, and the like of the plurality of photographing apparatuses on the vehicle may be different. Wherein, a plurality of shooting devices can acquire the surrounding image of the vehicle in real time.

In the embodiment of the disclosure, the information of each 3D target frame around the vehicle at the current time point may be determined by combining the electronic device with the position information of the vehicle at the current time point, the position information of a plurality of photographing devices on the vehicle, the photographing angle range of the photographing devices, and the information of each 2D detection frame in a plurality of surrounding images.

In the embodiment of the disclosure, at least one of position information, size information, heading angle information, and category information may be included in the 3D object frame information. The position information may be represented by 3D coordinate information of each corner of the 3D target frame, and the size information may refer to a length, a width and a height of the 3D target frame. The heading angle may be angle information of the 3D target frame with respect to the vehicle. The type information, i.e., the type of obstacle, may be, for example, a person, a vehicle, a static object, or the like, and may be set according to actual needs.

The 3D detection frame information may include at least one of position information, size information, heading angle information, and category information.

Step 102, inputting 3D detection frame information of the obstacle at each historical time point into a tracking filter of the obstacle for each obstacle, and determining 3D prediction frame information of the obstacle at the current time point.

In the disclosed embodiments, a tracking filter may be provided for each obstacle. The number of the tracking filters of the obstacle may be one or more, for example, a 2D frame filter, a 3D frame speed filter, a 3D frame acceleration filter, a 3D frame size filter, and the like.

In one example, the input of the 2D frame filter may be the ground coordinate information, the length and the width of one corner in the 3D detection frame information of the obstacle at each historical time point, and the output of the 2D frame filter may be the ground coordinate information, the length and the width of one corner in the 3D prediction frame information of the obstacle at the current time point. In another example, the input of the 2D frame filter may be the ground coordinate information, the length, and the width of the center point of the 3D detection frame information of the obstacle at each historical point in time, and the output of the 2D frame filter may be the ground coordinate information, the length, and the width of the center point of the 3D prediction frame information of the obstacle at the current point in time.

The input of the 3D frame speed filter may be position information and heading angle information in 3D detection frame information of the obstacle at each historical time point, and the output of the 3D frame speed filter may be position information, heading angle information and speed information in 3D prediction frame information of the obstacle at the current time point.

The input of the 3D frame acceleration filter may be position information and heading angle information in 3D detection frame information of the obstacle at each historical time point, and the output of the 3D frame acceleration filter may be position information, heading angle information, speed information and acceleration information in 3D prediction frame information of the obstacle at the current time point.

The input of the 3D frame size filter may be size information in 3D detection frame information of the obstacle at each historical time point, and the output of the 3D frame size filter may be size information in 3D prediction frame information of the obstacle at the current time point.

Under the condition that the number of the tracking filters of the obstacle is multiple, the 3D prediction frame information of the obstacle at the current time point can be comprehensively determined by combining the output results of the tracking filters.

Step 103, determining the obstacle to which each piece of 3D target frame information belongs at the current time point according to each piece of 3D target frame information at the current time point and the 3D prediction frame information of each obstacle.

In the embodiment of the disclosure, the electronic device may determine, according to the 3D target frame information and the 3D prediction frame information of each obstacle at the current time point, a matching degree between each 3D target frame information and each 3D prediction frame information, and determine, in combination with the matching degree, an obstacle to which each 3D target frame information belongs at the current time point.

And 104, updating the tracking filter of each obstacle according to the 3D prediction frame information of each obstacle at the current time point and the obstacle to which each 3D target frame information belongs to, so as to obtain the updated tracking filter of each obstacle.

In the embodiment of the present disclosure, the process of executing step 104 by the electronic device may be, for example, determining, for each obstacle, difference information between 3D prediction frame information of the obstacle and 3D target frame information pertaining to the obstacle, and updating a tracking filter of the obstacle according to the difference information to obtain an updated tracking filter of the obstacle.

Step 105, for each obstacle, inputting 3D detection frame information of the obstacle at the current time point and previous historical time points into an updated tracking filter of the obstacle to obtain updated 3D prediction frame information of the obstacle at the current time point, and using the updated 3D prediction frame information of the obstacle to track the obstacle.

In the embodiment of the disclosure, the tracking filter of the obstacle may be a kalman filter, and the input of the kalman filter may relate to a plurality of time points and the output may relate to a plurality of time points. That is, after the 3D detection frame information of the obstacle at the current time point and the previous historical time points are input into the updated kalman filter, the 3D prediction frame information of the obstacle at the current time point and a plurality of future time points can be predicted, so that the updated 3D prediction frame information of the obstacle at the current time point is obtained.

In the embodiment of the present disclosure, after step 105, the electronic device may further perform a process of determining updated 3D prediction frame information of the obstacle at the current point in time as 3D detection frame information of the obstacle at the current point in time. Therefore, the electronic equipment can acquire the 3D detection frame information of the obstacle at each time point, and track the obstacle.

According to the obstacle tracking method, 3D target frame information of the periphery of a vehicle at a current time point and 3D detection frame information of the periphery of the vehicle at historical time points are obtained through acquisition, 3D target frame information is obtained according to multiple peripheral images of the vehicle at the current time point, 3D detection frame information of the obstacles at the historical time points is input into a tracking filter of the obstacles for each obstacle, 3D prediction frame information of the obstacles at the current time point is determined, the obstacles which the 3D target frame information of the obstacles belong to are determined according to the 3D target frame information of the obstacles at the current time point and the 3D prediction frame information of the obstacles, 3D prediction frame information of the obstacles at the current time point and the obstacles which the 3D target frame information of the obstacles belong to are updated according to the 3D prediction frame information of the obstacles at the current time point, updated tracking filters of the obstacles are obtained for each obstacle, 3D detection frame information of the obstacles at the current time point and the previous historical time points is input into the 3D detection frame information of the obstacles, the updated tracking filters of the obstacles are updated for each obstacle, and the 3D detection frame information of the obstacles at the current time point is updated, the updated for the 3D detection frame information of the obstacles is obtained, and the 3D detection efficiency of the obstacles is improved is achieved.

Fig. 2 is a flowchart of an obstacle tracking method according to another embodiment of the present disclosure. It should be noted that, the obstacle tracking method of the present embodiment may be applied to an obstacle tracking device, and the device may be configured in an electronic apparatus, so that the electronic apparatus may perform an obstacle tracking function.

As shown in fig. 2, the method comprises the steps of:

And step 201, acquiring a plurality of surrounding images of the vehicle at the current time point, wherein the plurality of surrounding images are shot by a plurality of shooting devices on the vehicle at the current time point.

Step 202, obtaining an obstacle detection model.

In the embodiment of the disclosure, the obstacle detection model may be trained by combining surrounding images at a plurality of historical time points of the vehicle and obstacle information around the vehicle in advance. The obstacle information includes, for example, position information, size information, heading angle information, category information, and the like of a detection frame in which the obstacle is located. The obstacle information around the vehicle may be obstacle information around the vehicle in the vehicle body coordinate system.

The obstacle detection model can be trained by electronic equipment or other equipment. The training process of the electronic device on the obstacle detection model may be, for example, acquiring a plurality of surrounding images of the vehicle at a historical time point and laser point cloud data at the historical time point, determining 3D detection frame information of the obstacle in a vehicle body coordinate system at the historical time point by combining the laser point cloud data at the historical time point, establishing correspondence between the 3D detection frame information of the obstacle and the plurality of surrounding images of the vehicle at the historical time point to obtain training data, and training the initial obstacle detection model by adopting the training data to obtain a trained obstacle detection model.

Step 203, inputting the plurality of surrounding images into the obstacle detection model, and obtaining a plurality of pieces of 3D candidate frame information and the confidence corresponding to the 3D candidate frame information.

In the embodiment of the disclosure, the obstacle detection model is trained by combining a plurality of surrounding images of the vehicle at historical time points and obstacle information of the periphery of the vehicle in a vehicle body coordinate system at the historical time points. Correspondingly, the electronic device may perform the process of step 203 by, for example, inputting a plurality of peripheral images into the obstacle detection model, obtaining a plurality of pieces of 3D candidate frame information in the vehicle body coordinate system and corresponding confidence levels, and performing coordinate conversion processing in combination with the position information of the vehicle in the world coordinate system at the current time point and the plurality of pieces of 3D candidate frame information in the vehicle body coordinate system to obtain a plurality of pieces of 3D candidate frame information in the world coordinate system.

Because the vehicle body coordinate systems at different time points may be different in the running process of the vehicle, a plurality of pieces of 3D candidate frame information in the vehicle body coordinate systems need to be converted into the world coordinate system for processing, so that the plurality of pieces of 3D candidate frame information are ensured to be positioned in the same coordinate system, and the accuracy of the follow-up tracking processing is further improved.

And 204, filtering the plurality of 3D candidate frame information by combining the confidence coefficient to obtain each 3D target frame information at the current time point.

In the embodiment of the present disclosure, the electronic device performs the step 204, for example, may perform ground projection processing on each piece of 3D candidate frame information to obtain ground 2D candidate frame information corresponding to each piece of 3D candidate frame information, sort each piece of ground 2D candidate frame information in descending order according to the confidence level corresponding to each piece of 3D candidate frame information to obtain a sorting result, extract the ground 2D candidate frame information with the greatest confidence level from the sorting result, delete the ground 2D candidate frame information with the highest confidence level in the sorting result and the intersection ratio with the ground 2D candidate frame information being greater than or equal to the first intersection ratio threshold, re-perform the above steps until the ground 2D candidate frame information in the sorting result is processed, and use the 3D candidate frame information corresponding to the extracted ground 2D candidate frame information as each piece of 3D target frame information at the current time point.

The ground projection processing of the 3D candidate frame information may refer to removing the height information or the Z-axis coordinate information in the 3D candidate frame information.

The determining process of the intersection ratio of the two pieces of ground 2D candidate frame information may be, for example, obtaining an intersection of the areas occupied by the two pieces of ground 2D candidate frame information, obtaining a union of the areas occupied by the two pieces of ground 2D candidate frame information, and determining the ratio of the intersection area to the union area as the intersection ratio of the two pieces of ground 2D candidate frame information.

The filtering processing is performed on the plurality of 3D candidate frame information, repeated 3D candidate frame information in the plurality of 3D candidate frame information can be removed, only one piece of 3D candidate frame information is reserved for each obstacle, and subsequent matching processing is facilitated.

Step 205, acquiring 3D detection frame information of each obstacle around the vehicle at each historical time point.

Step 206, inputting 3D detection frame information of the obstacle at each historical time point into a tracking filter of the obstacle for each obstacle, and determining 3D prediction frame information of the obstacle at the current time point.

Step 207, determining the obstacle to which each piece of 3D target frame information belongs at the current time point according to each piece of 3D target frame information at the current time point and the 3D predicted frame information of each piece of obstacle.

And step 208, updating the tracking filter of each obstacle according to the 3D prediction frame information of each obstacle at the current time point and the obstacle to which each 3D target frame information belongs to, so as to obtain the updated tracking filter of each obstacle.

Step 209, inputting 3D detection frame information of the obstacle at the current time point and previous historical time points for each obstacle, and obtaining updated 3D prediction frame information of the obstacle at the current time point by using an updated tracking filter of the obstacle.

It should be noted that, the descriptions of steps 206 to 209 may refer to the detailed descriptions of steps 102 to 105 in the embodiment shown in fig. 1, and will not be described in detail here.

In the obstacle tracking method of the embodiment of the disclosure, a plurality of surrounding images of a vehicle at a current time point are acquired by acquiring the plurality of surrounding images, a plurality of photographing devices of the vehicle photograph the surrounding images at the current time point to acquire an obstacle detection model, the plurality of surrounding images are input into the obstacle detection model to acquire a plurality of 3D candidate frame information and confidence levels corresponding to the 3D candidate frame information, the plurality of 3D candidate frame information is filtered by combining the confidence levels to acquire 3D target frame information of each obstacle around the vehicle at the current time point, 3D detection frame information of each obstacle around the vehicle at each historical time point is acquired, 3D detection frame information of each obstacle at each historical time point is input into a tracking filter of the obstacle to determine 3D prediction frame information of the obstacle at the current time point, the obstacle belongs to each 3D target frame information at the current time point is determined according to the 3D target frame information of each obstacle at the current time point and the 3D prediction frame information of each obstacle, the 3D target frame information of each obstacle at the current time point is updated according to the 3D prediction frame information of each obstacle at the current time point and the 3D prediction frame information of each obstacle, the current time point is updated is determined based on the 3D detection frame information of each obstacle at each obstacle detection frame of each obstacle at the current time point, and each obstacle is updated, further improving the obstacle tracking efficiency.

Fig. 3 is a flowchart of an obstacle tracking method according to another embodiment of the present disclosure. It should be noted that, the obstacle tracking method of the present embodiment may be applied to an obstacle tracking device, and the device may be configured in an electronic apparatus, so that the electronic apparatus may perform an obstacle tracking function.

As shown in fig. 3, the method comprises the steps of:

Step 301, acquiring 3D target frame information of the periphery of the vehicle at the current time point and 3D detection frame information of the periphery of the vehicle at each historical time point, wherein the 3D target frame information is determined according to a plurality of peripheral images of the vehicle at the current time point, and the 3D target frame information and the 3D detection frame information comprise the types of the obstacles.

Step 302, inputting 3D detection frame information of the obstacle at each historical time point into a tracking filter of the obstacle for each obstacle, and determining 3D prediction frame information of the obstacle at the current time point.

Step 303, for each obstacle, acquiring 3D target frame information to be compared in each piece of 3D target frame information, wherein the category of the obstacle in the 3D target frame information to be compared is consistent with the category of the obstacle.

Step 304, determining the matching degree between the 3D predicted frame information of the obstacle and each 3D target frame information to be compared.

In the embodiment of the disclosure, the 3D target frame information and the 3D prediction frame information further include location information. Correspondingly, the electronic device performs the process of step 304, for example, performing ground projection processing on the 3D predicted frame information of the obstacle and each piece of 3D target frame information to be compared to obtain ground 2D predicted frame information corresponding to the 3D predicted frame information of the obstacle and ground 2D target frame information corresponding to each piece of 3D target frame information to be compared, determining distance data between the 3D predicted frame information of the obstacle and the 3D target frame information to be compared according to position information in the 3D target frame information to be compared and position information in the 3D predicted frame information of the obstacle for each piece of 3D target frame information to be compared, determining ground cross-correlation data between the 3D predicted frame information of the obstacle and the 3D target frame information to be compared according to the ground 2D predicted frame information corresponding to the 3D target frame information to be compared and the ground 2D predicted frame information corresponding to the 3D predicted frame information of the obstacle, and determining the matching degree between the 3D predicted frame information of the obstacle and the 3D target frame information to be compared according to the distance data and the ground cross-correlation data.

The distance data between the 3D predicted frame information of the obstacle and the 3D target frame information to be compared can be obtained after distance calculation according to the 3D coordinate information of one corner in the 3D predicted frame information of the obstacle and the 3D coordinate information of the corresponding corner in the 3D target frame information to be compared, or can be obtained after distance calculation according to the 3D coordinate information of the center point of the 3D predicted frame information of the obstacle and the 3D coordinate information of the center point of the 3D target frame information to be compared.

The determining process of the ground intersection data between the 3D prediction frame information of the obstacle and the 3D target frame information to be compared may be, for example, obtaining an intersection between a occupied area of the 3D prediction frame information and a occupied area of the 3D target frame information to be compared, obtaining a union between the occupied area of the 3D prediction frame information and the occupied area of the 3D target frame information to be compared, and determining a ratio of the intersection to the union as the ground intersection data.

In the embodiment of the disclosure, the process of determining the matching degree between the 3D predicted frame information of the obstacle and the 3D target frame information to be compared by the electronic device according to the distance data and the ground intersection ratio data may be, for example, determining the distance matching degree according to the distance data and a preset distance threshold, determining the ground intersection ratio matching degree according to the ground intersection ratio data and a second intersection ratio threshold, and determining the matching degree between the 3D predicted frame information of the obstacle and the 3D target frame information to be compared by combining the distance matching degree and the ground intersection ratio matching degree.

The distance matching degree is determined to be zero when the distance data is larger than or equal to a preset distance threshold value, and is determined to be larger when the distance data is smaller than the preset distance threshold value.

The ground intersection ratio matching degree is determined to be zero when the ground intersection ratio data is smaller than or equal to the second intersection ratio threshold value, and is determined according to the ground intersection ratio data when the ground intersection ratio data is larger than the second intersection ratio threshold value, wherein the larger the ground intersection ratio data is, the larger the ground intersection ratio matching degree is.

The electronic equipment can determine the weight corresponding to the distance matching degree and the weight corresponding to the ground intersection ratio matching degree, conduct weighted summation processing on the distance matching degree and the ground intersection ratio matching degree according to the weight to obtain a weighted summation result, and take the weighted summation result as the matching degree between 3D prediction frame information of the obstacle and 3D target frame information to be compared.

In the embodiment of the disclosure, the electronic device may further perform the process of determining image 2D predicted frame information corresponding to 3D predicted frame information of an obstacle and image 2D target frame information corresponding to each piece of 3D target frame information to be compared in combination with position information of a vehicle in a world coordinate system at a current time point and position information of a plurality of photographing devices on the vehicle relative to the vehicle, determining image areas corresponding to each piece of image 2D target frame information and image areas corresponding to the image 2D predicted frame information in combination with a plurality of surrounding images of the vehicle at the current time point, determining intersection ratio data between the image 2D target frame information and the image 2D predicted frame information and image similarity data between the image areas corresponding to the image 2D target frame information and the image areas corresponding to the image 2D predicted frame information for each piece of image 2D target frame information. Correspondingly, the electronic device can determine the matching degree between the 3D predicted frame information of the obstacle and the 3D target frame information to be compared according to the distance data, the ground intersection ratio data, the intersection ratio data and the image similarity data.

The image similarity data may be calculated according to a feature similarity calculation, a same pixel number calculation, a structural similarity (Structural Similarity, SSIM), or the like.

Step 305, determining an obstacle to which the 3D target frame information to be compared, corresponding to the largest matching degree of the plurality of matching degrees, is the obstacle.

And 306, updating the tracking filter of each obstacle according to the 3D prediction frame information of each obstacle at the current time point and the obstacle to which each 3D target frame information belongs to, so as to obtain the updated tracking filter of each obstacle.

Step 307, for each obstacle, inputting 3D detection frame information of the obstacle at the current time point and previous historical time points into an updated tracking filter of the obstacle to obtain updated 3D prediction frame information of the obstacle at the current time point, and using the updated 3D prediction frame information of the obstacle to track the obstacle.

In the obstacle tracking method of the embodiment of the disclosure, 3D detection frame information of each obstacle around a vehicle at each historical time point is obtained by obtaining each 3D target frame information around the vehicle at the current time point; the method comprises the steps of obtaining 3D target frame information according to a plurality of surrounding images of a vehicle at a current time point, determining 3D target frame information according to a plurality of surrounding images of the vehicle at the current time point, obtaining 3D target frame information to be compared in the 3D target frame information according to each obstacle, determining matching degree between 3D predicted frame information of the obstacle and each 3D target frame information to be compared, determining the obstacle to be compared, which corresponds to the largest matching degree in the matching degree, of the 3D target frame information, as the obstacle, inputting 3D predicted frame information of each obstacle at each historical time point according to the 3D predicted frame information of each obstacle at the current time point and the obstacle to which each 3D target frame information belongs, updating the tracking filter of each obstacle to obtain updated tracking filter of each obstacle, determining matching degree between the 3D predicted frame information of the obstacle and each 3D target frame information to be compared according to each obstacle to the current time point, and the current time point to which each obstacle belongs, and the current time point to be used for updating the 3D predicted frame information of each obstacle can be combined, therefore, the obstacle corresponding to the 3D target frame information can be accurately determined, the obstacle tracking efficiency is further improved, and the accurate tracking processing of the obstacle is realized.

Fig. 4 is a schematic structural view of an obstacle tracking device according to an embodiment of the disclosure.

As shown in fig. 4, the obstacle tracking device may include an acquisition module 401, a first determination module 402, a second determination module 403, an update module 404, and a third determination module 405.

The system comprises an acquisition module 401, a first determination module 402, a second determination module 403, an updating module 404, and a third determination module, wherein the acquisition module 401 is used for acquiring 3D target frame information of the periphery of a vehicle at a current time point and 3D prediction frame information of each obstacle at each historical time point, the 3D target frame information is determined according to a plurality of peripheral images of the vehicle at the current time point, the 3D detection frame information of the obstacle at each historical time point is input into a tracking filter of the obstacle, the 3D prediction frame information of the obstacle at the current time point is determined, the second determination module 403 is used for determining the obstacle to which each 3D target frame information belongs according to each 3D target frame information at the current time point and the 3D prediction frame information of each obstacle at each historical time point, the updating module 404 is used for updating the tracking filter of each obstacle according to the 3D prediction frame information of each obstacle at the current time point and the obstacle to which each 3D target frame information belongs, the updated filter of each obstacle is obtained after updating the tracking filter of each obstacle is obtained, and the current time point is input into the 3D prediction frame information of each obstacle.

In one embodiment of the disclosure, the obtaining module 401 is specifically configured to obtain a plurality of peripheral images of the vehicle at the current time point, obtain a plurality of peripheral images by a plurality of photographing devices on the vehicle at the current time point, obtain an obstacle detection model, input the plurality of peripheral images into the obstacle detection model, obtain a plurality of 3D candidate frame information and a confidence level corresponding to the 3D candidate frame information, and filter the plurality of 3D candidate frame information in combination with the confidence level to obtain each 3D target frame information at the current time point.

In one embodiment of the disclosure, the obstacle detection model is obtained by combining a plurality of peripheral images of the vehicle at a historical time point and obstacle information of the periphery of the vehicle in a vehicle body coordinate system at the historical time point, the obtaining module 401 is specifically further configured to input the plurality of peripheral images into the obstacle detection model to obtain a plurality of 3D candidate frame information and corresponding confidence degrees in the vehicle body coordinate system, and coordinate conversion processing is performed by combining the position information of the vehicle in the world coordinate system at the current time point and the plurality of 3D candidate frame information in the vehicle body coordinate system to obtain a plurality of 3D candidate frame information in the world coordinate system.

In an embodiment of the present disclosure, the obtaining module 401 is specifically further configured to perform ground projection processing on each piece of 3D candidate frame information to obtain ground 2D candidate frame information corresponding to each piece of 3D candidate frame information, sort each piece of ground 2D candidate frame information in descending order according to the confidence level corresponding to each piece of 3D candidate frame information to obtain a sorting result, extract the ground 2D candidate frame information with the greatest confidence level from the sorting result, delete the ground 2D candidate frame information with the highest confidence level in the sorting result, where the intersection ratio with the ground 2D candidate frame information is greater than or equal to the first intersection ratio threshold, re-execute the above steps until the ground 2D candidate frame information in the sorting result is processed, and use the 3D candidate frame information corresponding to the extracted ground 2D candidate frame information as each piece of 3D target frame information at the current time point.

In one embodiment of the disclosure, the 3D target frame information and the 3D detection frame information include an obstacle category, and the second determining module 403 is specifically configured to obtain, for each obstacle, 3D target frame information to be compared in each piece of 3D target frame information, where the obstacle category in the 3D target frame information to be compared is consistent with the category of the obstacle, determine a matching degree between the 3D prediction frame information of the obstacle and each piece of 3D target frame information to be compared, and determine an obstacle to which the 3D target frame information to be compared corresponding to the largest matching degree in the plurality of matching degrees belongs as the obstacle.

In one embodiment of the disclosure, the 3D target frame information and the 3D prediction frame information further include position information, the second determining module 403 is specifically further configured to perform ground projection processing on the 3D prediction frame information and each 3D target frame information to be compared of the obstacle to obtain ground 2D prediction frame information corresponding to the 3D prediction frame information of the obstacle and ground 2D prediction frame information corresponding to each 3D target frame information to be compared, determine, for each 3D target frame information to be compared, distance data between the 3D prediction frame information of the obstacle and the 3D target frame information to be compared according to the position information in the 3D target frame information to be compared and the position information in the 3D prediction frame information of the obstacle, determine ground 2D target frame information corresponding to the 3D target frame information to be compared and ground 2D prediction frame information corresponding to the 3D prediction frame information of the obstacle, and cross-match the distance data between the 3D prediction frame information of the obstacle and the 3D target frame information to be compared according to the ground 2D target frame information to be compared and the distance data between the 3D prediction frame information of the obstacle and the 3D target frame information to be compared.

In one embodiment of the disclosure, the second determining module 403 is further specifically configured to determine, in combination with the position information of the vehicle in the world coordinate system at the current time point and the position information of the plurality of photographing devices on the vehicle relative to the vehicle, image 2D predicted frame information corresponding to the 3D predicted frame information of the obstacle and image 2D target frame information corresponding to each piece of 3D target frame information to be compared, determine, in combination with the plurality of surrounding images of the vehicle at the current time point, an image area corresponding to each piece of image 2D target frame information and an image area corresponding to the image 2D predicted frame information, and determine, for each piece of image 2D target frame information, intersection ratio data between the image 2D target frame information and the image 2D predicted frame information and image similarity data between the image area corresponding to the image 2D target frame information and the image area corresponding to the image 2D predicted frame information, and the corresponding second determining module 403 is specifically configured to determine the intersection ratio data between the image 2D predicted frame information and the image predicted frame information according to the distance data, the intersection ratio data, and the image similarity data between the image 2D predicted frame information and the image predicted frame information.

In one embodiment of the present disclosure, the second determining module 403 is specifically further configured to determine, in combination with the position information of the vehicle in the world coordinate system at the current time point and the position information of the plurality of photographing devices on the vehicle relative to the vehicle, the position information of the plurality of photographing devices in the world coordinate system at the current time point, determine, according to the position information of the plurality of photographing devices, photographing devices corresponding to the 3D target frame information to be compared and photographing devices corresponding to the 3D predicted frame information of the obstacle, determine, according to the position information of the photographing devices, a coordinate conversion relationship between an image coordinate system and the world coordinate system of each photographing device, determine, in combination with the coordinate conversion relationship between the image coordinate system and the world coordinate system of each photographing device, image 2D predicted frame information corresponding to the 3D predicted frame information of the obstacle, and image 2D target frame information corresponding to each 3D target frame information to be compared.

In one embodiment of the disclosure, the apparatus further comprises a fourth determining module configured to determine updated 3D prediction frame information of the obstacle at the current time point as 3D detection frame information of the obstacle at the current time point.

According to the obstacle tracking device, 3D target frame information of the periphery of a vehicle at a current time point and 3D detection frame information of the periphery of the vehicle at historical time points are obtained through acquisition, 3D target frame information is obtained according to determination of a plurality of peripheral images of the vehicle at the current time point, 3D detection frame information of the obstacles at the historical time points is input into a tracking filter of the obstacles for each obstacle, 3D prediction frame information of the obstacles at the current time point is determined, the obstacles which the 3D target frame information of the obstacles at the current time point belongs to are determined according to the 3D target frame information of the obstacles at the current time point and the 3D prediction frame information of the obstacles, 3D prediction frame information of the obstacles at the current time point and the obstacles which the 3D target frame information of the obstacles belong to are updated according to the 3D prediction frame information of the obstacles at the current time point, updated tracking filters of the obstacles are obtained, 3D detection frame information of the obstacles at the current time point and the previous historical time points is input into the 3D detection frame information of the obstacles for each obstacle, and the 3D detection frame information of the obstacles at the current time point is updated, the updated tracking filter of the obstacles is obtained, and the 3D detection efficiency of the obstacles can be improved.

According to a third aspect of embodiments of the present disclosure, there is also provided an electronic device comprising a processor, a memory for storing processor-executable instructions, wherein the processor is configured to implement the obstacle-tracking method as described above.

In order to implement the above-described embodiments, the present disclosure also proposes a storage medium.

Wherein the instructions in the storage medium, when executed by the processor, enable the processor to perform the obstacle tracking method as described above.

To achieve the above embodiments, the present disclosure also provides a computer program product.

Wherein the computer program product, when executed by a processor of an electronic device, enables the electronic device to perform the method as above.

Fig. 5 is a block diagram of a vehicle 500 according to an exemplary embodiment of the present disclosure. For example, the vehicle 500 may be a hybrid vehicle, or may be a non-hybrid vehicle, an electric vehicle, a fuel cell vehicle, or other type of vehicle. The vehicle 500 may be an autonomous vehicle, a semi-autonomous vehicle, or a non-autonomous vehicle.

Referring to fig. 5, a vehicle 500 may include various subsystems, such as an infotainment system 510, a perception system 520, a decision control system 530, a drive system 540, and a computing platform 550. Vehicle 500 may also include more or fewer subsystems, and each subsystem may include multiple components. In addition, interconnections between each subsystem and between each component of the vehicle 500 may be achieved by wired or wireless means.

In some embodiments, the infotainment system 510 may include a communication system, an entertainment system, a navigation system, and the like.

The sensing system 520 may include several sensors for sensing information of the environment surrounding the vehicle 500. For example, sensing system 520 may include a global positioning system (which may be a GPS system, or may be a beidou system or other positioning system), an inertial measurement unit (inertial measurement unit, IMU), a lidar, millimeter wave radar, an ultrasonic radar, and a camera device.

Decision control system 530 may include a computing system, a vehicle controller, a steering system, a throttle, and a braking system.

The drive system 540 may include components that provide powered movement of the vehicle 500. In one embodiment, the drive system 540 may include an engine, an energy source, a transmission, and wheels. The engine may be one or a combination of an internal combustion engine, an electric motor, an air compression engine. The engine is capable of converting energy provided by the energy source into mechanical energy.

Some or all of the functions of the vehicle 500 are controlled by the computing platform 550. The computing platform 550 may include at least one processor 551 and memory 552, and the processor 551 may execute instructions 553 stored in the memory 552.

The processor 551 may be any conventional processor, such as a commercially available CPU. The processor may also include, for example, an image processor (Graphic Process Unit, GPU), a field programmable gate array (Field Programmable GATE ARRAY, FPGA), a System On Chip (SOC), an Application SPECIFIC INTEGRATED Circuit (ASIC), or a combination thereof.

The memory 552 may be implemented by any type or combination of volatile or nonvolatile memory devices such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disk.

In addition to instructions 553, memory 552 may store data such as road maps, route information, vehicle position, direction, speed, and the like. The data stored by memory 552 may be used by computing platform 550.

In an embodiment of the present disclosure, the processor 551 may execute instructions 553 to complete all or part of the steps of the obstacle-tracking method described above.

Furthermore, the word "exemplary" is used herein to mean serving as an example, instance, illustration. Any aspect or design described herein as "exemplary" is not necessarily to be construed as advantageous over other aspects or designs. Rather, the use of the word exemplary is intended to present concepts in a concrete fashion. As used herein, the term "or" is intended to mean an inclusive "or" rather than an exclusive "or". That is, unless specified otherwise, or clear from context, "X application a or B" is intended to mean any one of the natural inclusive permutations. That is, if X application A, X application B, or both X applications A and B, "X application A or B" is satisfied under any of the foregoing instances. In addition, the articles "a" and "an" as used in this application and the appended claims are generally understood to mean "one or more" unless specified otherwise or clear from context to be directed to a singular form.

Also, although the disclosure has been shown and described with respect to one or more implementations, equivalent alterations and modifications will occur to others skilled in the art upon the reading and understanding of this specification and the annexed drawings. The present disclosure includes all such modifications and alterations and is limited only by the scope of the claims. In particular regard to the various functions performed by the above described components (e.g., elements, resources, etc.), the terms used to describe such components are intended to correspond, unless otherwise indicated, to any component which performs the specified function of the described component (which is functionally equivalent), even though not structurally equivalent to the disclosed structure. In addition, while a particular feature of the disclosure may have been disclosed with respect to only one of several implementations, such feature may be combined with one or more other features of the other implementations as may be desired and advantageous for any given or particular application. Furthermore, to the extent that the terms "includes," including, "" has, "" having, "or variants thereof are used in either the detailed description or the claims, such terms are intended to be inclusive in a manner similar to the term" comprising.

Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This application is intended to cover any adaptations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.

It is to be understood that the present disclosure is not limited to the precise arrangements and instrumentalities shown in the drawings, and that various modifications and changes may be effected without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1.A method of obstacle tracking, the method comprising:

Acquiring 3D target frame information of each obstacle around a vehicle at a current time point and 3D detection frame information of each obstacle around the vehicle at each historical time point, wherein the 3D target frame information is determined according to a plurality of surrounding images of the vehicle at the current time point;

Inputting 3D detection frame information of the obstacle at each historical time point into a tracking filter of the obstacle for each obstacle, and determining 3D prediction frame information of the obstacle at the current time point;

Determining an obstacle to which each piece of 3D target frame information belongs at the current time point according to each piece of 3D target frame information at the current time point and 3D prediction frame information of each piece of obstacle;

Updating the tracking filter of each obstacle according to the 3D prediction frame information of each obstacle at the current time point and the obstacle to which each 3D target frame information belongs to obtain an updated tracking filter of each obstacle;

and inputting 3D detection frame information of the obstacle at the current time point and each previous historical time point into an updated tracking filter of the obstacle for each obstacle to obtain updated 3D prediction frame information of the obstacle at the current time point, and performing tracking processing on the obstacle.

2. The method according to claim 1, wherein acquiring respective 3D object frame information of the vehicle periphery at the current point in time includes:

Acquiring a plurality of peripheral images of the vehicle at the current time point, wherein the plurality of peripheral images are shot by a plurality of shooting devices on the vehicle at the current time point;

Obtaining an obstacle detection model;

inputting a plurality of peripheral images into the obstacle detection model, and acquiring a plurality of pieces of 3D candidate frame information and confidence degrees corresponding to the 3D candidate frame information;

And combining the confidence coefficient, and filtering the plurality of 3D candidate frame information to obtain each piece of 3D target frame information at the current time point.

3. The method according to claim 2, wherein the obstacle detection model is trained in combination with a plurality of surrounding images of the vehicle at historical points in time and obstacle information of the periphery of the vehicle in a vehicle body coordinate system at the historical points in time;

Inputting the plurality of surrounding images into the obstacle detection model, obtaining a plurality of pieces of 3D candidate frame information, and a confidence level corresponding to the 3D candidate frame information, including:

inputting a plurality of peripheral images into the obstacle detection model, and acquiring a plurality of 3D candidate frame information and corresponding confidence coefficients in a vehicle body coordinate system;

and carrying out coordinate conversion processing by combining the position information of the vehicle in the world coordinate system at the current time point and the plurality of 3D candidate frame information under the vehicle body coordinate system to obtain the plurality of 3D candidate frame information in the world coordinate system.

4. The method according to claim 2, wherein the filtering the plurality of 3D candidate frame information in combination with the confidence level to obtain each 3D target frame information at the current time point includes:

Performing ground projection processing on each piece of 3D candidate frame information to obtain ground 2D candidate frame information corresponding to each piece of 3D candidate frame information;

according to the confidence coefficient corresponding to each 3D candidate frame information, carrying out descending order sorting on each ground 2D candidate frame information to obtain a sorting result;

The ground 2D candidate frame information with the highest confidence coefficient is taken out from the sorting result, and the ground 2D candidate frame information with the intersection ratio with the ground 2D candidate frame information larger than or equal to a first intersection ratio threshold value in the sorting result is deleted;

re-executing the steps until the ground 2D candidate frame information processing in the sequencing result is completed;

and taking the 3D candidate frame information corresponding to the extracted ground 2D candidate frame information as each 3D target frame information at the current time point.

5. The method according to claim 1, wherein the 3D target frame information and the 3D detection frame information include an obstacle category, and wherein the determining, according to each piece of 3D target frame information at the current time point and the 3D prediction frame information of each obstacle, the obstacle to which each piece of 3D target frame information at the current time point belongs includes:

aiming at each obstacle, acquiring 3D target frame information to be compared in each piece of 3D target frame information, wherein the category of the obstacle in the 3D target frame information to be compared is consistent with the category of the obstacle;

determining the matching degree between the 3D predicted frame information of the obstacle and each 3D target frame information to be compared;

And determining an obstacle to which the 3D target frame information to be compared, corresponding to the largest matching degree in the plurality of matching degrees, is the obstacle.

6. The method of claim 5, wherein the 3D object frame information and the 3D prediction frame information further include location information, and wherein the determining a degree of matching between the 3D prediction frame information of the obstacle and each 3D object frame information to be compared includes:

Performing ground projection processing on the 3D predicted frame information of the obstacle and the 3D target frame information to be compared to obtain ground 2D predicted frame information corresponding to the 3D predicted frame information of the obstacle and ground 2D target frame information corresponding to the 3D target frame information to be compared;

determining distance data between 3D predicted frame information of the obstacle and 3D target frame information to be compared according to the position information in the 3D target frame information to be compared and the position information in the 3D predicted frame information of the obstacle aiming at each 3D target frame information to be compared;

According to the ground 2D target frame information corresponding to the 3D target frame information to be compared and the ground 2D predicted frame information corresponding to the 3D predicted frame information of the obstacle, determining ground intersection comparison data between the 3D predicted frame information of the obstacle and the 3D target frame information to be compared;

And determining the matching degree between the 3D predicted frame information of the obstacle and the 3D target frame information to be compared according to the distance data and the ground intersection ratio data.

7. The method of claim 6, wherein determining a degree of matching between the 3D predicted frame information of the obstacle and each 3D target frame information to be compared further comprises:

Combining the position information of the vehicle in the world coordinate system at the current time point and the position information of a plurality of shooting devices on the vehicle relative to the vehicle, and determining image 2D predicted frame information corresponding to the 3D predicted frame information of the obstacle and image 2D target frame information corresponding to each piece of 3D target frame information to be compared;

combining a plurality of surrounding images of the vehicle at the current time point, determining an image area corresponding to each image 2D target frame information and an image area corresponding to the image 2D prediction frame information;

For each image 2D target frame information, determining cross ratio data between the image 2D target frame information and the image 2D prediction frame information, and image similarity data between an image area corresponding to the image 2D target frame information and an image area corresponding to the image 2D prediction frame information;

The step of determining the matching degree between the 3D predicted frame information of the obstacle and the 3D target frame information to be compared according to the distance data and the ground intersection ratio data, comprises the following steps:

and determining the matching degree between the 3D predicted frame information of the obstacle and the 3D target frame information to be compared according to the distance data, the ground intersection ratio data, the intersection ratio data and the image similarity data.

8. The method according to claim 7, wherein the determining the image 2D predicted frame information corresponding to the 3D predicted frame information of the obstacle and the image 2D target frame information corresponding to each 3D target frame information to be compared in combination with the position information of the vehicle in the world coordinate system at the current point in time and the position information of the plurality of photographing devices on the vehicle with respect to the vehicle includes:

determining the position information of a plurality of shooting devices in the world coordinate system at the current time point by combining the position information of the vehicle in the world coordinate system at the current time point and the position information of a plurality of shooting devices on the vehicle relative to the vehicle;

According to the position information of the shooting devices, shooting devices corresponding to the 3D target frame information to be compared and shooting devices corresponding to the 3D prediction frame information of the obstacle are determined;

determining a coordinate conversion relation between an image coordinate system and a world coordinate system of each shooting device according to the position information of each shooting device;

And determining image 2D predicted frame information corresponding to the 3D predicted frame information of the obstacle and image 2D target frame information corresponding to the 3D target frame information to be compared by combining the coordinate conversion relation between the image coordinate system and the world coordinate system of each shooting device.

9. The method according to claim 1, wherein the method further comprises:

And determining the updated 3D prediction frame information of the obstacle at the current time point as the 3D detection frame information of the obstacle at the current time point.

10. An obstacle tracking device, the device comprising:

the system comprises an acquisition module, a detection module and a display module, wherein the acquisition module is used for acquiring all 3D target frame information of the periphery of a vehicle at a current time point and 3D detection frame information of all barriers of the periphery of the vehicle at each historical time point, wherein the 3D target frame information is determined according to a plurality of peripheral images of the vehicle at the current time point;

A first determining module, configured to input 3D detection frame information of the obstacle at each historical time point into a tracking filter of the obstacle for each obstacle, and determine 3D prediction frame information of the obstacle at the current time point;

a second determining module, configured to determine, according to each piece of 3D target frame information at the current time point and the 3D prediction frame information of each piece of obstacle, the obstacle to which each piece of 3D target frame information at the current time point belongs;

the updating module is used for updating the tracking filter of each obstacle according to the 3D prediction frame information of each obstacle at the current time point and the obstacle to which each 3D target frame information belongs to so as to obtain an updated tracking filter of each obstacle;

And a third determining module, configured to input, for each obstacle, 3D detection frame information of the obstacle at the current time point and previous historical time points, to an updated tracking filter of the obstacle, to obtain updated 3D prediction frame information of the obstacle at the current time point, and to perform tracking processing on the obstacle.

11. A vehicle, characterized by comprising:

A processor;

a memory for storing the processor-executable instructions;

wherein the processor is configured to:

The step of implementing the obstacle tracking method as claimed in any one of claims 1 to 9.

12. A non-transitory computer readable storage medium, which when executed by a processor, causes the processor to perform the obstacle tracking method of any one of claims 1 to 9.