JP2022076876A - Position estimation device, position estimation method, and position estimation program - Google Patents

Abstract

To provide a position estimation device, etc. capable of estimating a position with a reduced processing load.SOLUTION: A position estimation device 100 estimates a position of a vehicle equipped with a LiDAR 11 for detecting an environment point cloud that is a point cloud of singularities of a surrounding environment of the vehicle by detecting reflected light for light irradiation, and an outside camera 12 for capturing an image of the surrounding environment. The position estimation device 100 includes a landmark detection unit 110 for detecting a landmark in the surrounding environment from an image captured by the camera. The position estimation device 100 includes an extraction unit 120 for extracting a point cloud within an estimation range, which is a range where the landmark is estimated to be present, from an environment point cloud as a point cloud within the range. The position estimation device 100 includes a specification unit 130 for specifying the landmark point cloud, which is a point cloud corresponding to the landmark, from a within-range point cloud. The position estimation device 100 includes a position determination unit 140 for matching the landmark point cloud with a point cloud map to determine the estimated position of the vehicle.SELECTED DRAWING: Figure 2

Description

The disclosure herein relates to techniques for estimating the position of a vehicle.

Patent Document 1 discloses a system that provides map information to a vehicle. In this system, a high-precision map based on the information detected by the vehicle sensor is generated by the server. The vehicle acquires a high-precision map from the server and estimates the current position based on the scan data of LiDAR (Light Detection and Ranging / Laser Imaging Detection and Ranging) detected while driving and the high-precision map. As a method of estimating the current position based on the scan data and the high-precision map, there is scan matching that matches the point cloud of the scan data with the point cloud included in the high-precision map.

U.S. Patent Application Publication No. 2018/018806

Since the position estimation by the above-mentioned scan matching matches a large number of points with each other, the processing load tends to be large.

It is an object of the disclosure to provide a position estimation device, a position estimation method, and a position estimation program capable of position estimation with a reduced processing load.

The plurality of embodiments disclosed herein employ different technical means to achieve their respective objectives. Further, the scope of claims and the reference numerals in parentheses described in this section are examples showing the correspondence with the specific means described in the embodiment described later as one embodiment, and limit the technical scope. is not.

One of the disclosed position estimation devices is an optical sensor (11) that detects an environment point cloud, which is a point cloud of feature points of the surrounding environment of the vehicle (A) by detecting reflected light for light irradiation, and an ambient environment. It is a position estimation device that estimates the position of the vehicle equipped with the camera (12) that captures the image.
A detection unit (110) that detects a landmark (L) in the surrounding environment from an image captured by a camera,
An extraction unit (120) that extracts a point cloud within the estimated range (Re), which is the range where the landmark is estimated to exist, from the environmental point cloud as a point cloud within the range.
The specific part (130) that identifies the landmark point cloud, which is the point cloud corresponding to the landmark, from the point cloud within the range, and
A position-fixing unit (140) that matches the landmark point cloud with the point cloud map to determine the estimated vehicle position, and
To prepare for.

One of the disclosed position estimation methods is an optical sensor (11) that detects an environment point cloud, which is a point cloud of feature points of the surrounding environment of the vehicle (A) by detecting reflected light for light irradiation, and an ambient environment. It is a position estimation method executed by a processor (102) in order to estimate the position of a vehicle equipped with a camera (12) for imaging a light beam.
The detection process (S10, S20) that detects the landmark (L) in the surrounding environment from the image captured by the camera,
An extraction process (S30, S40) that extracts a point cloud within the estimated range (Re), which is the range where the landmark is estimated to exist, from the environmental point cloud as a point cloud within the range.
A specific process (S50) for identifying a landmark point cloud, which is a point cloud corresponding to a landmark, from within the range point cloud, and
The position-fixing process (S60), which determines the estimated vehicle position by matching the landmark point cloud with the point cloud map,
including.

One of the disclosed position estimation programs is an optical sensor (11) that detects an environment point cloud, which is a point cloud of feature points of the surrounding environment of the vehicle (A) by detecting reflected light for light irradiation, and an ambient environment. A position estimation program including an instruction to be executed by a processor (102) in order to estimate the position of a vehicle equipped with a camera (12) for capturing an image.
The order is
A detection process (S10, S20) for detecting a landmark (L) in the surrounding environment from an image captured by a camera.
An extraction process (S30, S40) that extracts a point cloud within the estimated range (Re), which is the range where the landmark is estimated to exist, from the environmental point cloud as a point cloud within the range.
A specific process (S50) for identifying a landmark point cloud, which is a point cloud corresponding to a landmark, from the range point cloud, and
The position-fixing process (S60), which matches the landmark point cloud with the point cloud map and determines the estimated vehicle position,
including.

According to these disclosures, the point cloud within the estimated range in which the landmark detected from the image captured by the camera is presumed to exist is extracted from the environmental point cloud by the optical sensor. Then, the landmark point cloud is specified from the point cloud within the range, and the estimated position of the vehicle is determined by matching the landmark point cloud with the point cloud map. Therefore, the environmental point cloud detected by the optical sensor is limited to the within-range point cloud based on the captured image, further limited to the within-range point cloud to the landmark point cloud, and then matched with the point cloud map. Used for. Therefore, although it is possible to reliably estimate the position by matching by securing the landmark point cloud, the number of environment point clouds used for matching itself can be suppressed. As described above, a position estimation device, a position estimation method, and a position estimation program capable of estimating a position with a reduced processing load can be provided.

It is a figure which shows the system including the position estimation apparatus. It is a block diagram which shows an example of the function which a position estimation apparatus has. It is a schematic diagram which shows an example of a defined estimation range. It is a flowchart which shows an example of the position estimation method executed by a position estimation apparatus.

(First Embodiment)
The position estimation device 100 of the first embodiment will be described with reference to FIGS. 1 to 4. The position estimation device 100 is an electronic control device mounted on the vehicle A. The position estimation device 100 is connected to another vehicle-mounted configuration via at least one of, for example, a LAN (Local Area Network), a wire harness, an internal bus, and the like. Other vehicle-mounted configurations include a peripheral monitoring sensor 10, an automated driving ECU 40, and a vehicle control ECU 50.

The peripheral monitoring sensor 10 is an autonomous sensor that monitors the surrounding environment of the vehicle A, and is a LiDAR (Light Detection and Ranging / Laser Imaging Detection and Ranging) 11 and an external camera 12 that captures a predetermined range including the front of the vehicle A. including. Further, the peripheral monitoring sensor 10 may include a millimeter wave radar, sonar, and the like.

The LiDAR 11 is an example of an optical sensor that detects an environment point cloud, which is a point cloud of the surrounding environment of the vehicle A, by detecting the reflected light for the light irradiation. The LiDAR 11 has a light emitting element, an image pickup element, and an image pickup circuit. The light emitting device is a semiconductor device that emits directional laser light, such as a laser diode. The light emitting element irradiates the laser beam toward the outside of the vehicle A in the form of an intermittent pulse beam. The image pickup device is a semiconductor device having high sensitivity to light, such as a SPAD (Single Photon Avalanche Diode). An incident surface dedicated to LiDAR 11 is arranged on the outer world side of the image pickup device. The image sensor is exposed by the light incident on the incident surface from the sensing region determined by the angle of view of the image sensor in the outside world.

The image pickup circuit is an integrated circuit that controls exposure and scanning of a plurality of pixels in the image pickup element and processes signals from the image pickup element to convert them into data. The image pickup circuit exposes the image pickup element by irradiating light from the light emitting element. In the reflected light mode, at least an object point in the sensing region in the surrounding environment becomes a reflection point of the laser beam. As a result, the laser light reflected at the reflection point (hereinafter referred to as reflected light) is incident on the image sensor through the incident surface. At this time, the image pickup circuit senses the reflected light by scanning a plurality of pixels of the image pickup element. The image pickup circuit converts half of the beam flight time from the irradiation timing to the detection of the reflected light in each pixel into a distance value from the image pickup element to the reflection point. The image pickup circuit associates the converted distance value with each pixel and converts it into data. As a result, the image pickup circuit acquires the environment point cloud data in which the three-dimensional distance information is associated with each point of the environment point cloud.

The external camera 12 captures a sensing region that at least partially overlaps with the sensing region of the LiDAR 11 in the surrounding environment. The external camera 12 has an image pickup device and an image pickup circuit. The image pickup device is, for example, a semiconductor device such as CMOS. An incident surface dedicated to the outside world camera 12 is arranged on the outside world side of the image pickup device. The image sensor is exposed by the light incident on the incident surface from the sensing region determined by the angle of view of the image sensor in the outside world.

The image pickup circuit is an integrated circuit that controls the exposure and scanning of a plurality of pixels in the image pickup element and processes the signal from the element to convert it into data. The image pickup circuit exposes the image pickup element under the control of the image pickup element. At this time, the object point in the sensing region becomes the reflection point of the external light. As a result, the external light reflected at the reflection point is incident on the image sensor through the incident surface. At this time, the image pickup circuit senses the reflected external light by scanning a plurality of pixels of the image pickup element. Here, in particular, the imaging circuit acquires an image captured according to the intensity of external light by converting the luminance value acquired for each of a plurality of pixels according to the intensity of the sensed external light into two-dimensional data as each pixel value. ..

In addition, the image pickup circuit is based on, for example, a point group image acquired from LiDAR11, an image captured from a camera, and the like, and by image analysis processing, an obstacle on a traveling path, a preceding vehicle, a parallel running vehicle, an oncoming vehicle, etc. Recognize the presence or absence of other vehicles and their position.

The automatic driving ECU 40 is an electronic control device that constantly or temporarily executes at least one of autonomous driving control and advanced driving support control as an automatic driving mode. The automatic driving ECU 40 acquires the estimated self-position of the vehicle A from the position estimation device 100, and generates a control instruction regarding acceleration / deceleration or steering of the vehicle A in the automatic driving mode based on the position.

The vehicle control ECU 50 is an electronic control device that controls acceleration / deceleration and steering of the vehicle A. The vehicle control ECU 50 includes a steering ECU that performs steering control, a power unit control ECU that performs acceleration / deceleration control, a brake ECU, and the like. The vehicle control ECU 50 acquires detection signals output from each sensor such as a steering angle sensor and a vehicle speed sensor mounted on the vehicle A, and controls each traveling of an electronically controlled throttle, a brake actuator, an EPS (Electric Power Steering) motor, and the like. Output a control signal to the device. The vehicle control ECU 50 controls each driving control device so as to realize autonomous driving or advanced driving support according to the control instruction by acquiring the control instruction of the vehicle A from the automatic driving ECU 40.

The position estimation device 100 includes at least one dedicated computer. The dedicated computer constituting the position estimation device 100 is, for example, a locator ECU that estimates the state quantity of the vehicle A including its own position. The position estimation device 100 includes at least one memory 101 and one processor 102 by including such a dedicated computer.

The memory 101 non-temporarily stores programs, data, and the like that can be read by a computer. The memory 101 is, for example, a non-transitory tangible storage medium of at least one type among semiconductor memories, magnetic media, optical media, and the like. The memory 101 stores at least a position estimation program described later as a program executed by the processor 102.

Further, the memory 101 stores a landmark map database (hereinafter, “DB”) 125 and a point cloud map DB 145 in a predetermined data storage area. Each map DB 125, 145 is a database that stores different types of map data.

The landmark map DB 125 stores information about the landmark L, which is a feature on and around the road. The landmark L includes, for example, road markings (travel lane markings, stop lines, etc.), road signs, structures, and the like. Structures include, for example, street lights and traffic lights. Further, the structure may include a pedestrian bridge, an elevated structure, a building of a specific facility, and the like.

The landmark map DB 125 is a vector map database that holds the position and shape of the landmark L as vector information. The landmark map DB 125 may include the attribute information of each landmark L. The map data of the landmark map DB 125 is generated based on, for example, an image captured by the outside world camera 12 collected by a surveying vehicle equipped with the outside world camera 12. The map data of the landmark map 125 may be updated as appropriate based on the images captured by the outside world camera 12 of the surveying vehicle or the general vehicle.

The point cloud map DB 145 stores map data of a three-dimensional point cloud composed of point clouds of road shapes and feature points of structures. The map data of the point cloud map DB 145 includes at least three-dimensional coordinate information about each point of the point cloud. The map data of the point cloud map DB 145 may include attribute information of each point. The map data of the point cloud map DB 145 is generated based on, for example, the point cloud data by the LiDAR 11 collected by the surveying vehicle equipped with the LiDAR 11. The map data of the point cloud map DB 145 may be appropriately updated based on the point cloud data from the LiDAR 11 of the surveying vehicle or the general vehicle.

The processor 102 includes, for example, at least one of a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a RISC (Reduced Instruction Set Computer) -CPU, and the like as a core. The processor 102 executes a plurality of instructions included in the position estimation program stored in the memory 101. As a result, the position estimation device 100 constructs a plurality of functional units for estimating the position of the vehicle A. The functional part can be paraphrased as a functional block. In this way, in the position estimation device 100, a plurality of functional units are constructed by causing the processor 102 to execute a plurality of instructions by the position estimation program stored in the memory 101. As shown in FIG. 2, the plurality of functional units constructed by the position estimation device 100 include a landmark detection unit 110, an extraction unit 120, a specific unit 130, and a position determination unit 140.

The landmark detection unit 110 acquires a captured image transmitted from the outside world camera 12. The landmark detection unit 110 detects the landmark L from the captured image by image recognition processing. The landmark L is, for example, at least one of a road marking, a road sign and a structure. The landmark detection unit 110 detects the relative position, shape and size of the landmark L by image recognition, and provides the information to the extraction unit 120 as detection information. If the landmark L cannot be detected from the captured image, the landmark detection unit 110 detects the landmark L from another captured image in the next control cycle.

The extraction unit 120 sets the position range in which the actual landmark L is estimated to exist as the estimation range Re based on the detection information of the landmark L by the landmark detection unit 110. The extraction unit 120 extracts the point cloud within the estimation range Re as the point cloud within the range from the environment point cloud detected by LiDAR 11.

More specifically, the extraction unit 120 sets the estimation range Re to a range centered on the position of the landmark L based on the detection information (camera detection position) (see FIG. 3). In FIG. 3, the circle with the hatched grid indicates the camera detection position as the representative position, and the circle with the hatched broken line indicates the actual position of the landmark L. The camera detection position is, for example, a representative position such as the position of the center of gravity of the landmark L. Alternatively, the camera detection position may be the contour position of the detected landmark L.

The extraction unit 120 determines the position of the estimated range Re based on the landmark map information stored in the landmark map DB. Specifically, the extraction unit 120 compares the position information included in the landmark map information with the camera detection position, and determines whether or not an object different from the landmark L is erroneously recognized as the landmark L. judge. If it is determined that the recognition is erroneous here, the landmark detection unit 110 repeats the detection process. If it is determined that there is no erroneous recognition, the position of the estimated range Re is determined as the position based on the camera detection position.

The extraction unit 120 determines the size of the estimated range Re at the determined position based on the distance from the vehicle A to the camera detection position (camera detection distance) and the area of the landmark L. Specifically, the extraction unit 120 sets the size of the estimated range Re in the imaging direction of the external camera 12 to be larger as the camera detection distance is larger. In the example shown in FIG. 3, it is assumed that the imaging direction substantially coincides with the traveling direction of the vehicle A. For example, the extraction unit 120 may adjust the size of the image pickup direction according to the camera detection distance based on the coefficient related to the relationship between the position accuracy in the image pickup direction of the external camera 12 and the camera detection distance. The correspondence between the camera detection distance and the magnitude of the estimation range Re in the imaging direction may be linear or non-linear.

Further, the extraction unit 120 reduces the size of the estimation range Re in the imaging direction as the area of the specific surface in the landmark L recognized from the captured image is larger. Here, the specific surface is the surface of the landmark L recognized from the captured image by image recognition. For example, when the landmark L is a signboard sign, the surface facing the outside world camera 12 side of the signboard is regarded as a specific surface. Further, when the landmark L is a road surface indication, the surface on the road surface of the road surface indication is regarded as a specific surface. The correspondence between the size of the area and the size of the estimated range Re in the imaging direction may be linear or non-linear.

The extraction unit 120 sets the size of the estimation range Re in the image pickup direction to be larger than the size in the direction orthogonal to the image pickup direction (orthogonal direction). For example, when the camera detection position is a representative position, the extraction unit 120 defines the estimation range Re in an elliptical shape centered on the camera position, with the imaging direction having a major axis and the orthogonal direction having a minor axis.

The extraction unit 120 acquires the environment point cloud data from LiDAR 11, and extracts the environment point cloud within the estimation range Re set as described above as the range inside point cloud. As a result, the environmental point cloud outside the estimation range Re is rejected. In the example shown in FIG. 3, the circles with hatched dots indicate the point cloud within the range. The ranged point cloud in FIG. 3 is merely an example, and more ranged point clouds may actually be extracted. The acquired environment point cloud data is provided with type information based on processing such as semantic segmentation by the LiDAR 11 or the extraction unit 120 or the like. Alternatively, the extraction unit 120 may assign type information only to the point cloud within the range. The extraction unit 120 provides the data of the extracted range point cloud to the specific unit 130.

The specifying unit 130 identifies the point cloud corresponding to the landmark L as the landmark point cloud from the extracted range point cloud. As an example, the identification unit 130 identifies the landmark point cloud based on the position information of the point cloud within the range and the reflection intensity information of the point cloud within the range. To describe in detail the identification based on the position information of the point cloud within the range, the specific unit 130 describes the position of the center of gravity of each candidate point cloud that is a candidate for the landmark point cloud in the point cloud within the range, and the landmark detected from the captured image. The degree of coincidence with the position of the center of gravity of L is calculated. The specific unit 130 tentatively determines the candidate point cloud having the highest degree of coincidence as the landmark point cloud. Here, the candidate point cloud means each group of the point cloud grouped based on the type information in the range point cloud.

To describe in detail the identification based on the reflection intensity information of the point cloud within the range, the identification unit 130 compares the reflection intensity estimated based on the detection information of the landmark L with the actual reflection intensity of each candidate point cloud. Specifically, the specific unit 130 defines a range of reflection intensity estimated for the landmark point cloud from the type of the landmark L detected from the captured image. In the following, the range may be referred to as a reflection intensity range. The correspondence between the type of the landmark L and the reflection intensity range is stored in advance in, for example, a memory 101 or the like. In addition to the type of the landmark L, the specific unit 130 may specify the reflection intensity range according to the distance to the landmark L.

The specific unit 130 compares the specified reflection intensity range with the reflection intensity of the candidate point cloud in the point cloud within the range, and determines the landmark point cloud. For example, the specific unit 130 may calculate the average value of the reflection intensity of each point in the candidate point group, and when the average value is within the reflection intensity range, the candidate point group may be determined as a landmark point group. ..

The specific unit 130 tentatively determines a candidate point cloud to be a landmark point cloud based on the reflection intensity, and determines whether or not to determine the candidate point cloud as a landmark point cloud based on the position information. May be good. Further, the identification unit 130 may specify the landmark point cloud after removing the point cloud of the moving object from the point cloud within the range.

The position determination unit 140 matches the specified landmark point cloud with the point cloud map data stored in the point cloud map DB. The positioning unit 140 performs matching according to a specified algorithm such as an ICP algorithm or an NDT algorithm. As a result, the position determination unit 140 identifies the corresponding point cloud corresponding to the landmark point cloud from the point cloud map data. The position determination unit 140 estimates the absolute position of the vehicle A on the point cloud map data based on the relative position of the vehicle A with respect to the landmark point cloud and the coordinate position information of the corresponding point cloud. Determined as (estimated self-position). The position determining unit 140 provides the determined estimated self-position to the automatic driving ECU 40.

Next, the flow of the position estimation method executed by the position estimation device 100 jointly by the functional blocks will be described below with reference to FIG. In the flow described later, "S" means a plurality of steps of the flow executed by a plurality of instructions included in the program.

First, in S10, the landmark detection unit 110 acquires an image captured by the outside world camera 12. Next, in S20, the landmark detection unit 110 performs image recognition processing on the captured image and determines whether or not the landmark L is present in the captured image. If it is determined that the image does not exist, the flow returns to S10, and the determination is performed again for another captured image.

On the other hand, if it is determined in S20 that the landmark L is present in the captured image, this flow shifts to S30. In S30, the extraction unit 120 sets the estimation range Re in which the landmark L is presumed to exist, based on the detection information of the landmark L from the captured image. Specifically, in S30, the extraction unit 120 determines the position of the estimated range Re based on the landmark map information, and images the estimated range Re based on the camera detection distance and the area of the specific surface of the landmark L. Specifies the size of the direction. At this time, the size of the estimation range Re in the direction orthogonal to the image pickup direction is defined to be smaller than the size in the image pickup direction. In the following S40, the extraction unit 120 extracts the point cloud within the range from the environment point cloud based on the set estimation range Re.

Further, in S50, the identification unit 130 identifies the landmark point cloud from the within-range point cloud based on the position information including the height information of the landmark L and the reflection intensity of the within-range point cloud. If the landmark point cloud cannot be identified from the range point cloud, this flow returns to S10. On the other hand, when the landmark point cloud is specified from the range point cloud in S50, this flow shifts to S60. Then, in S60, the position determination unit 140 matches the point cloud map data stored in the point cloud map DB 145 with the landmark point cloud, and determines the estimated self-position of the vehicle A.

The above-mentioned S10 and S20 are examples of the "detection process", S30 and S40 are examples of the "extraction process", S50 is an example of the "specific process", and S60 is an example of the "position determination process".

According to the above first embodiment, the point cloud within the estimated range Re where the landmark L detected from the image captured by the external camera 12 is presumed to exist is extracted from the environmental point cloud by LiDAR 11. Then, the landmark point cloud is specified from the point cloud within the range, and the estimated position of the vehicle A is determined by matching the landmark point cloud with the point cloud map. Therefore, the environmental point cloud detected by LiDAR11 is limited to the within-range point cloud based on the captured image, further limited to the within-range point cloud to the landmark point cloud, and then matched with the point cloud map. used. Therefore, although it is possible to reliably estimate the position by matching by securing the landmark point cloud, the number of environment point clouds used for matching itself can be suppressed. From the above, it is possible to estimate the position where the processing load is reduced.

Further, according to the first embodiment, the larger the estimated distance from the vehicle A to the landmark L, the larger the size of the estimated range Re in the imaging direction of the external camera 12 is defined. Therefore, for a distant landmark L in which the detection accuracy of the position in the imaging direction based on the captured image is lower, an estimated range Re having a larger magnitude in the imaging direction can be defined. Therefore, there is a high possibility that the actual landmark L is included in the estimation range Re, and the landmark point cloud can be extracted more reliably.

In addition, according to the first embodiment, the larger the area of the specific surface in the landmark L recognized based on the captured image, the smaller the size of the estimated range Re in the imaging direction is set. Therefore, an estimation range Re having a smaller size in the imaging direction can be defined for the large landmark L on the specific surface, which can increase the detection accuracy of the imaging direction position based on the captured image. Therefore, the number of point clouds included in the estimation range Re is reduced, and the processing load can be further reduced.

Further, according to the first embodiment, the size of the estimation range Re in the image pickup direction is set to be larger than the size in the direction orthogonal to the image pickup direction. Therefore, an estimated range Re having a larger magnitude in the imaging direction in which the detection accuracy of the position based on the captured image becomes lower can be defined. Therefore, there is a high possibility that the actual landmark L is included in the estimation range Re, and the landmark point cloud can be extracted more reliably.

Further, according to the first embodiment, the estimation range Re is determined based on the vector map including the position information of the landmark L. Therefore, when the false detection of the landmark L based on the captured image occurs, this can be corrected based on the position information of the landmark L.

In addition, according to the first embodiment, the landmark point cloud is specified based on the height information of the landmark L estimated based on the captured image. Therefore, the landmark point cloud can be identified more accurately.

Further, according to the first embodiment, the landmark point cloud is specified based on the reflection intensity range estimated for the landmark L and the reflection intensity of the point cloud within the range. Therefore, a point cloud whose reflection intensity is out of the estimated reflection intensity range can be excluded from the landmark point cloud. Therefore, false detection of the landmark point cloud can be avoided.

(Other embodiments)
The disclosure herein is not limited to the exemplary embodiments. Disclosures include exemplary embodiments and modifications by those skilled in the art based on them. For example, the disclosure is not limited to the parts and / or combinations of elements shown in the embodiments. Disclosure can be carried out in various combinations. Disclosures can have additional parts that can be added to the embodiments. Disclosures include those in which the parts and / or elements of the embodiment are omitted. Disclosures include the replacement or combination of parts and / or elements between one embodiment and another. The technical scope disclosed is not limited to the description of the embodiments. Some technical scopes disclosed are indicated by the description of the scope of claims and should be understood to include all modifications within the meaning and scope equivalent to the description of the scope of claims. ..

As a modification of the above-described embodiment, the position estimation device 100 may select the landmark L used for position estimation based on the accuracy of the camera detection position. For example, the extraction unit 120 may interrupt the extraction of the point cloud within the range for the landmark L in which the size of the estimation range Re exceeds the allowable range. The permissible range in this case may be appropriately changed based on the number of detected landmarks L and the like.

In the above-described embodiment, the self-position estimated by the position estimation device 100 is used for executing the automatic operation mode in the automatic operation ECU 40, but it may be used for executing other applications. For example, the self-position may be used to execute a route guidance display on an in-vehicle display.

In the above-described embodiment, the dedicated computer constituting the position estimation device 100 is assumed to be a locator ECU. Instead of this, the dedicated computer constituting the position estimation device 100 may be the automatic operation ECU 40. Alternatively, the dedicated computer constituting the position estimation device 100 may be an actuator ECU that individually controls the traveling actuator of the vehicle A. The dedicated computer constituting the position estimation device 100 may be a navigation ECU for navigating the traveling route of the vehicle A. The dedicated computer constituting the position estimation device 100 may be an HCU (HMI (Human Machine Interface) Control Unit) that controls the information presentation of the information presentation system of the vehicle A.

The position estimation device 100 may be a dedicated computer configured to include at least one of a digital circuit and an analog circuit as a processor 102. Here, particularly digital circuits include, for example, ASIC (Application Specific Integrated Circuit), FPGA (Field Programmable Gate Array), SOC (System on a Chip), PGA (Programmable Gate Array), CPLD (Complex Programmable Logic Device) and the like. Of these, at least one. Further, such a digital circuit may include a memory 101 in which a program is stored.

The position estimation device 100 may be provided by one computer, or a set of computer resources linked by a data communication device. For example, a part of the functions provided by the position estimation device 100 in the above-described embodiment may be realized by another ECU.

11 LiDAR (optical sensor), 12 external camera (camera), 100 position estimation device, 102 processor, 110 landmark detection unit (detection unit), 120 extraction unit, 130 specific unit, 140 position determination unit, A vehicle, L land Mark, Re Estimated range.

Claims (9)

An optical sensor (11) that detects an environment point cloud that is a point cloud of characteristic points of the surrounding environment of the vehicle (A) by detecting reflected light for light irradiation, and a camera (12) that captures the surrounding environment. It is a position estimation device that estimates the position of the mounted vehicle.
A detection unit (110) that detects a landmark (L) in the surrounding environment from an image captured by the camera.
An extraction unit (120) that extracts a point cloud within the estimation range (Re), which is a range in which the landmark is estimated to exist, from the environment point cloud as a point cloud within the range.
A specific portion (130) that identifies a landmark point cloud, which is a point cloud corresponding to the landmark, from the point cloud within the range, and
A position-determining unit (140) that determines the estimated position of the vehicle by matching the landmark point cloud with the point cloud map.
A position estimator equipped with. The position estimation device according to claim 1, wherein the extraction unit increases the size of the estimation range in the imaging direction of the camera as the estimated distance from the vehicle to the landmark increases. The position according to claim 1 or 2, wherein the larger the area of the specific surface in the landmark recognized based on the captured image, the smaller the size of the estimated range in the imaging direction of the camera. Estimator. The position estimation device according to any one of claims 1 to 3, wherein the extraction unit sets the size of the estimation range in the image pickup direction of the camera to be larger than the size in the direction orthogonal to the image pickup direction. .. The position estimation device according to any one of claims 1 to 4, wherein the extraction unit determines the position of the estimation range based on a vector map including the position information of the landmark. The position estimation device according to any one of claims 1 to 5, wherein the specific unit identifies the landmark point cloud based on the height information of the landmark estimated based on the captured image. The specific unit is any one of claims 1 to 6, which specifies the landmark point cloud based on the reflection intensity range estimated for the landmark and the reflection intensity of the point cloud within the range. The position estimation device described in. An optical sensor (11) that detects an environment point cloud that is a point cloud of characteristic points of the surrounding environment of the vehicle (A) by detecting reflected light for light irradiation, and a camera (12) that captures the surrounding environment. A position estimation method executed by a processor (102) for estimating the position of the mounted vehicle.
A detection process (S10, S20) for detecting a landmark (L) in the surrounding environment from an image captured by the camera.
An extraction process (S30, S40) for extracting a point cloud within the estimation range (Re), which is a range in which the landmark is estimated to exist, from the environment point cloud as a point cloud within the range.
A specific process (S50) for specifying a landmark point cloud, which is a point cloud corresponding to the landmark, from the point cloud within the range.
The position-fixing process (S60), which determines the estimated position of the vehicle by matching the landmark point cloud with the point cloud map,
Position estimation method including. An optical sensor (11) that detects an environment point cloud, which is a point cloud of characteristic points of the surrounding environment of the vehicle (A) by detecting reflected light for light irradiation, and a camera (12) that captures the surrounding environment. A position estimation program including an instruction to be executed by a processor (102) in order to estimate the position of the mounted vehicle.
The command is
A detection process (S10, S20) for detecting a landmark (L) in the surrounding environment from an image captured by the camera.
An extraction process (S30, S40) for extracting a point cloud within the estimation range (Re), which is a range in which the landmark is estimated to exist, from the environment point cloud as a point cloud within the range.
A specific process (S50) for specifying a landmark point cloud, which is a point cloud corresponding to the landmark, from the point cloud within the range.
A position-fixing process (S60) in which the landmark point cloud is matched with the point cloud map to determine the estimated position of the vehicle.
Position estimation program including.

Images