CN118225770B - Vehicle bottom centering checking method and vehicle bottom checking system based on intelligent vision - Google Patents

Vehicle bottom centering checking method and vehicle bottom checking system based on intelligent vision Download PDF

Info

Publication number
CN118225770B
CN118225770B CN202410635508.2A CN202410635508A CN118225770B CN 118225770 B CN118225770 B CN 118225770B CN 202410635508 A CN202410635508 A CN 202410635508A CN 118225770 B CN118225770 B CN 118225770B
Authority
CN
China
Prior art keywords
vehicle
license plate
robot
checking
midpoint
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202410635508.2A
Other languages
Chinese (zh)
Other versions
CN118225770A (en
Inventor
李晓凯
朱光强
李球
罗富章
彭锦文
胡伟
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Maxvision Technology Corp
Original Assignee
Maxvision Technology Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Maxvision Technology Corp filed Critical Maxvision Technology Corp
Priority to CN202410635508.2A priority Critical patent/CN118225770B/en
Publication of CN118225770A publication Critical patent/CN118225770A/en
Application granted granted Critical
Publication of CN118225770B publication Critical patent/CN118225770B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01M—TESTING STATIC OR DYNAMIC BALANCE OF MACHINES OR STRUCTURES; TESTING OF STRUCTURES OR APPARATUS, NOT OTHERWISE PROVIDED FOR
    • G01M17/00—Testing of vehicles
    • G01M17/007—Wheeled or endless-tracked vehicles
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84—Systems specially adapted for particular applications
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G06T3/4038—Image mosaicing, e.g. composing plane images from plane sub-images
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/80—Geometric correction
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/60—Analysis of geometric attributes
    • G06T7/66—Analysis of geometric attributes of image moments or centre of gravity
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
    • G06V10/462—Salient features, e.g. scale invariant feature transforms [SIFT]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2200/00—Indexing scheme for image data processing or generation, in general
    • G06T2200/32—Indexing scheme for image data processing or generation, in general involving image mosaicing
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30248—Vehicle exterior or interior
    • G06T2207/30252—Vehicle exterior; Vicinity of vehicle

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Immunology (AREA)
  • Biochemistry (AREA)
  • General Health & Medical Sciences (AREA)
  • Analytical Chemistry (AREA)
  • Pathology (AREA)
  • Chemical & Material Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Geometry (AREA)
  • Health & Medical Sciences (AREA)
  • Multimedia (AREA)
  • Length Measuring Devices By Optical Means (AREA)

Abstract

The application provides a vehicle bottom centering checking method and a vehicle bottom checking system based on intelligent vision, wherein the method comprises the following steps: the inspection robot acquires an external image of the vehicle in real time, and acquires a license plate positioning point and a front wheel midpoint from the external image of the vehicle; positioning a headstock positioning point according to the abscissa of the license plate positioning point and the ordinate of the midpoint of the front wheel; the inspection robot adjusts the direction to move to the head center line in real time based on the relative position of the head positioning point and the vision center line; the inspection robot acquires a vehicle bottom image in real time, and acquires a rear wheel midpoint from the vehicle bottom image; the checking robot adjusts the direction movement to the center line of the vehicle tail in real time based on the relative positions of the center point of the rear wheel and the center line of the visual field. Compared with the prior art, the application only depends on the vision of the inspection robot in the whole process, realizes centering correction, head centering navigation and tail centering navigation on the non-centered license plate, reduces cost and delay, and improves the adaptability, positioning accuracy and quality of the spliced images of the vehicle bottom.

Description

Vehicle bottom centering checking method and vehicle bottom checking system based on intelligent vision
Technical Field
The application belongs to the technical field of vehicle inspection, and particularly relates to an intelligent vision-based vehicle bottom centering inspection method and an intelligent vision-based vehicle bottom inspection system.
Background
Underbody inspection is a security measure aimed at preventing the escape of non-compliant items by hiding them in the underbody. In the prior art, a method for checking by using an under-vehicle checking robot is provided, wherein the under-vehicle checking robot is an electronic device capable of penetrating through an under-vehicle space and performing high-definition photographing on the under-vehicle by using a linear array camera, and continuous partial images acquired by the linear array camera are spliced to acquire a complete under-vehicle image.
However, because the parking positions of the vehicles are different, the vehicle bottom checking robot needs to navigate to the vehicle head through a manual remote control or a robot workstation, but the process has wireless network delay; and then the center point of the license plate is detected by the common camera to determine the center line of the vehicle, but when some license plates are not installed in the center, the vehicle bottom checking robot cannot enter the vehicle bottom from the center line of the vehicle. In addition, the vehicle bottom checking robot can realize obstacle avoidance through detection by a physical sensor after entering the vehicle bottom, but the vehicle bottom environment is complex, and the navigation process is easy to deviate from the center line of the vehicle. Therefore, the problem of poor quality such as malformation, target missing and the like exists in the whole vehicle bottom image finally obtained through image stitching.
Disclosure of Invention
The embodiment of the application aims to provide an intelligent vision-based vehicle bottom centering checking method and an intelligent vision-based vehicle bottom checking system, which are used for solving the technical problem of insufficient quality of a vehicle bottom spliced image in the checking process of the vehicle bottom in the prior art.
In order to achieve the above purpose, the application adopts the following technical scheme: provides a vehicle bottom centering checking method based on intelligent vision, the vehicle bottom centering checking method based on intelligent vision is based on a checking robot and comprises the following steps:
the inspection robot acquires an external image of the vehicle in real time, and acquires a license plate positioning point and a front wheel midpoint from the external image of the vehicle;
positioning a headstock positioning point according to the abscissa of the license plate positioning point and the ordinate of the midpoint of the front wheel;
The inspection robot adjusts the direction to move to the head center line in real time based on the relative position of the head positioning point and the vision center line;
the inspection robot acquires a vehicle bottom image in real time, and acquires a rear wheel midpoint from the vehicle bottom image;
the checking robot adjusts the direction movement to the center line of the vehicle tail in real time based on the relative positions of the center point of the rear wheel and the center line of the visual field.
Preferably, the method for acquiring license plate positioning points from the external image of the vehicle comprises the following steps:
detecting four license plate vertices from an external image of the vehicle;
outputting quadrangles formed by the vertexes of the four license plates;
And taking intersection points of the diagonal lines of the quadrangles as license plate positioning points.
Preferably, the method for acquiring license plate positioning points from the external image of the vehicle comprises the following steps:
detecting four license plate vertices from an external image of the vehicle;
outputting quadrangles formed by the vertexes of the four license plates;
Detecting all internal angle angles of the quadrangle, and determining the internal angle closest to the right angle;
and taking the midpoint of the opposite sides of the triangle closest to the right-angle inner angle as a license plate positioning point.
Preferably, the method for acquiring the midpoint of the front wheel from the external image of the vehicle comprises the steps of:
Detecting an external image of the vehicle through a front wheel key point detection model, and outputting 2 front wheel key points;
taking the midpoints of the 2 front wheel key points as front wheel midpoints;
the front wheel key points are three-party junction points of the front wheel, the vehicle chassis and the background area.
Preferably, after positioning the headstock positioning point according to the abscissa of the license plate positioning point and the ordinate of the midpoint of the front wheel, the method further comprises the steps of:
acquiring a rear wheel midpoint from an external image of the vehicle;
Acquiring a front wheel midpoint abscissa x f and a rear wheel midpoint abscissa x b;
Move x d coordinates to the right for the headstock anchor point, where x d=2（xf-xb).
Preferably, after positioning the headstock positioning point according to the abscissa of the license plate positioning point and the ordinate of the midpoint of the front wheel, the method further comprises the steps of:
acquiring a rear wheel midpoint from an external image of the vehicle;
Acquiring a distance L b between a pair of rear wheel key points, a distance L f between a pair of front wheel key points, a front wheel midpoint abscissa x f and a rear wheel midpoint abscissa x b;
The headstock anchor point is moved to the right by x d coordinates, where x d=（xf-xb）Lf/Lb.
Preferably, the method for judging whether the inspection robot moves to the center line of the headstock comprises the following steps:
stopping moving when a headstock positioning point is not detected from a continuous multi-frame image outside the vehicle;
acquiring a last frame of vehicle external image containing a headstock positioning point, wherein the vehicle external image is acquired by a depth camera;
acquiring the distance L c from the inspection robot to a license plate locating point in the external image of the last frame of vehicle according to the depth information;
Calculating the distance from a projection point of a license plate locating point on the ground to the inspection robot based on the distance L c S last;
Checking the straight blind travel path S of the robot, wherein S robot is a check robot body length.
Preferably, after the inspection robot moves to the center line of the headstock, the method further comprises the steps of:
detecting an included angle alpha between the normal vector direction of the head positioning point of the last frame of vehicle external image and the body of the inspection robot based on the depth information;
The inspection robot rotates in situ by an alpha degree, so that the normal vector direction of the body of the inspection robot and the positioning point of the headstock is parallel.
Preferably, the method for judging whether the inspection robot moves to the center line of the vehicle tail comprises the following steps:
and checking the straight blind travel path 2S of the robot when the midpoint of the rear wheel is not detected from the continuous multi-frame images of the bottom of the vehicle.
The application also provides a vehicle bottom checking system, which comprises a checking robot, a workstation and a background control terminal, wherein the checking robot is used for executing the vehicle bottom centering checking method based on intelligent vision, the workstation is used for performing logistic service on the checking robot and receiving monitoring data, uploading the monitoring data to the background control terminal, and the background control terminal is used for processing, analyzing and detecting the monitoring data.
Compared with the prior art, the vehicle bottom centering checking method based on intelligent vision only depends on the vision of the checking robot in the whole process, realizes centering correction, head centering navigation and tail centering navigation of the non-centered license plate, reduces cost and delay, and improves vehicle adaptability, positioning accuracy and vehicle bottom spliced image quality.
Compared with the prior art, the vehicle bottom checking system provided by the application has the advantages that the checking robot is used for executing the vehicle bottom centering checking method based on intelligent vision, and the whole process is only dependent on the vision of the checking robot, so that centering correction, head centering navigation and tail centering navigation of a non-centered license plate are autonomously realized, monitoring data can be uploaded through a data interface after each checking and navigation to a workstation in an offline state, and the stability and the adaptability are improved.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the embodiments or the description of the prior art will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a schematic flow chart of an intelligent vision-based vehicle bottom centering and checking method according to an embodiment of the present application;
FIG. 2 is a schematic illustration of an inspection robot navigation path line based on the method of FIG. 1;
FIG. 3 is a schematic diagram of a method for obtaining license plate anchor points based on the method of FIG. 1;
FIG. 4 is a schematic diagram of another method for obtaining license plate anchor points based on the method of FIG. 1;
FIG. 5 is a flow chart of training a wheel keypoint detection model based on DSNTNN frames in the method of FIG. 1;
FIG. 6 is a schematic diagram of correction of left displacement of the headstock positioning point based on the method of FIG. 1;
FIG. 7 is another schematic illustration of an inspection robot navigation path line based on the method of FIG. 1;
FIG. 8 is a schematic view of an exterior image of a vehicle when no movement of the headstock setpoint is detected over a succession of frames based on the method of FIG. 1;
FIG. 9 is a schematic view of a rear wheel midpoint acquired from a vehicle bottom image based on the method of FIG. 1;
Fig. 10 is a schematic diagram of an underbody inspection system according to an embodiment of the present application.
Detailed Description
In order to make the technical problems, technical schemes and beneficial effects to be solved more clear, the application is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the application.
It will be understood that when an element is referred to as being "mounted" or "disposed" on another element, it can be directly on the other element or be indirectly on the other element. When an element is referred to as being "connected to" another element, it can be directly connected to the other element or be indirectly connected to the other element.
It is to be understood that the terms "length," "width," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," and the like are merely for convenience in describing and simplifying the description based on the orientation or positional relationship shown in the drawings, and do not indicate or imply that the devices or elements referred to must have a particular orientation, be constructed and operated in a particular orientation, and thus are not to be construed as limiting the application.
Furthermore, the terms "first," "second," and the like, are used for descriptive purposes only and are not to be construed as indicating or implying a relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defining "a first" or "a second" may explicitly or implicitly include one or more such feature. In the description of the present application, the meaning of "a plurality" is two or more, unless explicitly defined otherwise.
Referring to fig. 1 to 2 together, a vehicle bottom centering inspection method based on intelligent vision according to an embodiment of the present application will now be described. The vehicle bottom centering checking method based on intelligent vision is based on a checking robot and comprises the following steps:
step S1, acquiring an external image of a vehicle in real time by a checking robot, and acquiring a license plate positioning point and a front wheel midpoint from the external image of the vehicle;
Step S2, positioning a headstock positioning point according to the abscissa of a license plate positioning point and the ordinate of a front wheel midpoint;
Step S3, the inspection robot adjusts the direction movement to the head center line in real time based on the relative positions of the head positioning points and the vision center line;
S4, acquiring a vehicle bottom image in real time by the inspection robot, and acquiring a rear wheel midpoint from the vehicle bottom image;
And S5, the checking robot adjusts the direction to move to the center line of the vehicle tail in real time based on the relative positions of the center point of the rear wheel and the center line of the visual field.
It will be appreciated that the position and orientation of the different vehicles will deviate after the vehicle arrives at the inspection area, and that some license plate positions of the vehicles are not centrally located.
Therefore, in step S1, the inspection robot acquires the vehicle exterior image in real time, and after detecting the vehicle target, acquires the license plate anchor point and the front wheel center point from the vehicle exterior image.
The prior license plate installation rule: the license plate should be deformation-free, perforated and substantially perpendicular to the ground; the front license plate must be installed in the middle or right-shifted position of the front lower part of the vehicle body, and the rear license plate must be installed in the middle or left-shifted position of the rear lower front part of the vehicle body. Therefore, the surface of the license plate is equivalent to the plane of the vehicle head, and the license plate positioning point is a point on the plane of the vehicle head.
The front wheel midpoint is the midpoint position of the left and right front wheels, and the front wheel midpoint is positioned on the vehicle head center line, so in the step S2, the vehicle head locating point is positioned according to the abscissa of the license plate locating point and the ordinate of the front wheel midpoint, and the vehicle head locating point is equivalent to the point obtained by correcting the license plate locating point relative to the vehicle head center line, namely the position of the license plate in the centered state, and the problem of positioning the non-centered license plate is solved.
In step S3, after determining the head positioning point, the inspection robot adjusts the direction in real time based on the relative position of the head positioning point and the view center line, for example, the head positioning point is located at the left side of the view center line, i.e., the inspection robot travels in the front left direction, for example, the head positioning point is located at the right side of the view center line, i.e., the inspection robot travels in the front right direction, for example, the head positioning point overlaps the view center line, i.e., the inspection robot travels straight, so that the inspection robot automatically navigates to the head center line.
Compared with the prior art, in the step S1 and the step S3, the workstation is not required to control navigation, communication with the workstation is not required, a physical sensor is not required to be arranged on the inspection robot, the whole process only depends on the vision of the inspection robot, the centering correction is carried out on the non-centering license plate, then the relative position of the head positioning point and the visual field center line is utilized to realize automatic navigation, network delay is avoided, and the navigation precision is improved.
In steps S4 to S5, after the inspection robot reaches the position of the vehicle head bottom, the inspection robot acquires the vehicle bottom image in real time, acquires the rear wheel midpoint from the vehicle bottom image, adjusts the direction to the vehicle tail centerline in real time based on the relative position of the rear wheel midpoint and the view centerline again, and completes the centering navigation again depending on the vision of the inspection robot.
Compared with the prior navigation mode using the physical sensor, the navigation method has the advantages that the physical sensor is greatly interfered due to the fact that the vehicle chassis is complex in environment and different in concave-convex, the walking path of the inspection robot can deviate from the center line of the vehicle tail, and the inspection robot is bent, so that in the moving process, the vehicle bottom panoramic image obtained by performing image splicing processing on the continuous vehicle bottom partial image obtained by the linear array camera on the inspection robot can be deformed, and foreign matters at the vehicle bottom can not be identified.
In steps S4 to S5 of the embodiment, the direction is adjusted in real time based on the relative position between the midpoint of the rear wheel and the center line of the field of view, so that the direction is not affected by the chassis of the vehicle, the inspection robot can also be adjusted in real time, the inspection robot can run straight at a constant speed, and the quality of image splicing processing is improved.
It should be noted that, the relative relationship between the front wheel and the rear wheel, and the relationship between the front and the rear of the vehicle are defined based on the relative positions of the vehicle and the inspection robot, and when the directions of the vehicle are opposite, the front wheel may also be referred to as the rear wheel, and the front of the vehicle may also be referred to as the rear of the vehicle. That is, the inspection robot of the present embodiment can start inspection from any one end of the vehicle.
Compared with the prior art, the vehicle bottom centering checking method based on intelligent vision only depends on the vision of the checking robot in the whole process, realizes centering correction, head centering navigation and tail centering navigation of the non-centered license plate, reduces cost and delay, and improves vehicle adaptability, positioning accuracy and vehicle bottom spliced image quality.
In another embodiment of the present application, in step S1, please refer to fig. 3 together, a method for obtaining a license plate positioning point from an external image of a vehicle includes the steps of:
detecting four license plate vertices from an external image of the vehicle;
outputting quadrangles formed by the vertexes of the four license plates;
And taking intersection points of the diagonal lines of the quadrangles as license plate positioning points.
It can be understood that the method is suitable for the situation that the deviation of the top points of four license plates is smaller, and compared with the prior art, the method has the advantage that the license plate locating point can be located more accurately by taking the intersection point of the diagonal lines of the license plate detection frame as the license plate locating point and the intersection point of the diagonal lines of the quadrangle as the license plate locating point. The embodiment trains a license plate key point detection model based on YOLOX frames, which is a target detection algorithm based on deep learning, and can rapidly and accurately detect various object key points in images.
In another embodiment of the present application, in step S1, please refer to fig. 4 together, a method for obtaining a license plate positioning point from an external image of a vehicle includes the steps of:
detecting four license plate vertices from an external image of the vehicle;
outputting quadrangles formed by the vertexes of the four license plates;
Detecting all internal angle angles of the quadrangle, and determining the internal angle closest to the right angle;
and taking the midpoint of the opposite sides of the triangle closest to the right-angle inner angle as a license plate positioning point.
It can be understood that in the detection process of four license plate vertices, a license plate vertex deviation is inevitably large, so that a final license plate positioning point is offset. In the rectangle, the offset of any one vertex will cause the change of the internal angle corresponding to the vertex and the adjacent vertex, so the geometric principle is combined with image processing to restore the center point of the rectangle in the embodiment. The method comprises the steps of determining the inner angle closest to a right angle by detecting all inner angle angles of a quadrangle formed by four license plate vertexes, and forming a triangle by the inner angle and two adjacent points, wherein the midpoint of a hypotenuse of the triangle is the midpoint of the opposite side of the inner angle. The accuracy and the robustness of license plate positioning points are improved.
In another embodiment of the present application, in step S1, please refer to fig. 5 and 6 together, a method for acquiring a front wheel midpoint from an external image of a vehicle includes the steps of:
Detecting an external image of the vehicle through a front wheel key point detection model, and outputting 2 front wheel key points;
taking the midpoints of the 2 front wheel key points as front wheel midpoints;
the front wheel key points are three-party junction points of the front wheel, the vehicle chassis and the background area.
It can be understood that, due to the insufficient brightness of the vehicle bottom part, the environment is complex, the boundary between the wheels and the vehicle bottom is fuzzy, if any point on the front wheels and the vehicle chassis is used as a target detection point, a large error exists, but the difference between the front wheels and the vehicle chassis and the background area is obvious. Therefore, three-party junction points of the front wheel, the vehicle chassis and the background area are used as front wheel key points, so that the false recognition rate can be reduced, and the detection precision can be improved.
In the embodiment, the wheel key point detection model is trained based on DSNTNN frames, so that in order to improve the overall performance of the vehicle bottom inspection robot, the embodiment simplifies the backstone of the network, greatly reduces network parameters and improves the reasoning speed; in order to improve the detection precision, the embodiment adds a new classification branch of the key points in the network to judge whether the detected key points are accurate or not. In this embodiment, the backup of DSNTNN is a UNet network, and a downsampling layer and an upsampling layer are removed to improve the performance of the model; meanwhile, a key point detection branch is added, and the key point detection branch consists of two convolution layers, a pooling layer and a sigmoid layer.
In another embodiment of the present application, in step S2, please refer to fig. 6, after positioning the vehicle head positioning point according to the abscissa of the license plate positioning point and the ordinate of the front wheel midpoint, the method further includes the steps of:
acquiring a rear wheel midpoint from an external image of the vehicle;
Acquiring a front wheel midpoint abscissa x f and a rear wheel midpoint abscissa x b;
Move x d coordinates to the right for the headstock anchor point, where x d=2（xf-xb).
It can be appreciated that, since the vehicle exterior image is a look-up view angle, when the inspection robot is close to the vehicle, there will be a distortion in the acquired vehicle exterior image, resulting in that the headstock positioning point positioned according to the abscissa of the license plate positioning point and the ordinate of the front wheel midpoint will deviate from the actual headstock midline.
In this embodiment, when used in a car, the degree of deviation is also calculated by detecting the rear wheel midpoint, that is, the difference x f-xb between the front wheel midpoint abscissa x f and the rear wheel midpoint abscissa x b. When the difference x f-xb is 0, the head positioning point is not offset, when the difference x f-xb is greater than 0, the head positioning point needs to be corrected in a right displacement mode, and when the difference x f-xb is less than 0, the head positioning point needs to be corrected in a left displacement mode. So as to improve the accuracy of the positioning point of the headstock.
In another embodiment of the present application, in step S2, please refer to fig. 6, after positioning the vehicle head positioning point according to the abscissa of the license plate positioning point and the ordinate of the front wheel midpoint, the method further includes the steps of:
acquiring a rear wheel midpoint from an external image of the vehicle;
Acquiring a distance L b between a pair of rear wheel key points, a distance L f between a pair of front wheel key points, a front wheel midpoint abscissa x f and a rear wheel midpoint abscissa x b;
The headstock anchor point is moved to the right by x d coordinates, where x d=（xf-xb）Lf/Lb.
It will be appreciated that the distortion is greater as the length of the vehicle body is longer, unlike the above embodiment, the dynamic coefficient is used in this embodiment, that is, it is applicable not only to cars but also to trucks, trailers, and the like. Because the distance L b between the pair of rear wheel key points and the distance L f between the pair of front wheel key points are also obtained in the embodiment, L f/Lb is taken as a displacement coefficient, when the length of the vehicle body is larger, L f/Lb is larger, and when the length of the vehicle body is smaller, L f/Lb is smaller, and compared with the previous embodiment, the adaptability and accuracy of positioning the positioning point of the vehicle head are improved.
In another embodiment of the present application, in step S3, please refer to fig. 7 and 8 together, a method for determining movement of a inspection robot to a center line of a vehicle head includes the steps of:
stopping moving when a headstock positioning point is not detected from a continuous multi-frame image outside the vehicle;
acquiring a last frame of vehicle external image containing a headstock positioning point, wherein the vehicle external image is acquired by a depth camera;
acquiring the distance L c from the inspection robot to a license plate locating point in the external image of the last frame of vehicle according to the depth information;
Calculating the distance from a projection point of a license plate locating point on the ground to the inspection robot based on the distance L c S last;
Checking the straight blind travel path S of the robot, wherein S robot is a check robot body length.
It will be appreciated that the inspection robot loses its head setpoint target when it approaches the bottom of the head, but is now a small distance from the head centerline. In the embodiment, the path S last from the projection point of the license plate positioning point on the ground to the inspection robot is calculated; because the depth camera is located at the head position of the inspection robot, the inspection robot reaches the center line of the head after blind path S last.
However, if the inspection robot adjusts the direction to the middle point of the rear wheel, the machine body can deviate by half the length of the machine body in the center line of the machine head. For this reason, the inspection robot also needs to walk half the inspection robot body length blindly. Therefore, accuracy of vehicle head centering navigation is further improved.
Further, in step S3, referring to fig. 7, after the inspection robot moves to the center line of the headstock, the method further includes the steps of:
detecting an included angle alpha between the normal vector direction of the head positioning point of the last frame of vehicle external image and the body of the inspection robot based on the depth information;
The inspection robot rotates in situ by an alpha degree, so that the normal vector direction of the body of the inspection robot and the positioning point of the headstock is parallel.
It will be appreciated that the direction of the robot body is checked as it is walking straight during blind walking. Therefore, the angle alpha calculated by the position before blind walking can be used for adjusting the body direction of the inspection robot after blind walking, so that the body direction of the inspection robot is parallel to the central line of the locomotive. On one hand, the vehicle bottom image acquired by the inspection robot can acquire the midpoint of the rear wheel, and on the other hand, the vehicle bottom image is prevented from being greatly turned in the process of centering, navigation and movement of the vehicle tail, and the distortion of spliced images of the vehicle bottom is prevented.
Further, in step S5, referring to fig. 7 and fig. 9 together, a method for determining that the inspection robot moves to the center line of the vehicle tail includes the steps of:
and checking the straight blind travel path 2S of the robot when the midpoint of the rear wheel is not detected from the continuous multi-frame images of the bottom of the vehicle.
It can be understood that, because the rear wheel midpoint is located based on the left and right rear wheels, when the rear wheel midpoint is not detected, the distance from the actual distance to the rear wheel midpoint is about S, and the distance from the rear wheel midpoint to the completely-outgoing vehicle tail is usually less than S, so that the inspection robot can check the straight blind travel path 2S, and the line-scan camera can ensure that a complete bottom image is acquired, and avoid acquiring too many background images.
Referring to fig. 10, the application further provides an underbody inspection system, which comprises an inspection robot, a workstation and a background control terminal, wherein the inspection robot is used for executing the underbody centering inspection method based on intelligent vision, the workstation is used for performing logistic service on the inspection robot and receiving monitoring data, uploading the monitoring data to the background control terminal, and the background control terminal processes, analyzes and detects the monitoring data.
Compared with the prior art, the vehicle bottom checking system provided by the application has the advantages that the checking robot is used for executing the vehicle bottom centering checking method based on intelligent vision, and the whole process is only dependent on the vision of the checking robot, so that centering correction, head centering navigation and tail centering navigation of a non-centered license plate are autonomously realized, monitoring data can be uploaded through a data interface after each checking and navigation to a workstation in an offline state, and the stability and the adaptability are improved.
The foregoing description of the preferred embodiments of the application is not intended to be limiting, but rather is intended to cover all modifications, equivalents, and alternatives falling within the spirit and principles of the application.

Claims (7)

1. The vehicle bottom centering checking method based on intelligent vision is based on a checking robot and is characterized by comprising the following steps:
the inspection robot acquires an external image of the vehicle in real time, and acquires a license plate positioning point and a front wheel midpoint from the external image of the vehicle;
a method of acquiring a front wheel midpoint from an image outside a vehicle, comprising the steps of:
Detecting an external image of the vehicle through a front wheel key point detection model, and outputting 2 front wheel key points;
taking the midpoints of the 2 front wheel key points as front wheel midpoints;
the front wheel key points are three-party junction points of a front wheel, a vehicle chassis and a background area;
positioning a headstock positioning point according to the abscissa of the license plate positioning point and the ordinate of the midpoint of the front wheel;
after the headstock positioning point is positioned according to the abscissa of the license plate positioning point and the ordinate of the front wheel midpoint, the method further comprises the following steps:
acquiring a rear wheel midpoint from an external image of the vehicle;
Acquiring a front wheel midpoint abscissa x f and a rear wheel midpoint abscissa x b;
Moving x d coordinates to the right for a headstock positioning point, wherein x d=2（xf-xb);
The inspection robot adjusts the direction to move to the head center line in real time based on the relative position of the head positioning point and the vision center line;
the inspection robot acquires a vehicle bottom image in real time, and acquires a rear wheel midpoint from the vehicle bottom image;
the checking robot adjusts the direction movement to the center line of the vehicle tail in real time based on the relative positions of the center point of the rear wheel and the center line of the visual field.
2. The intelligent vision-based vehicle bottom centering inspection method as claimed in claim 1, wherein the method for acquiring license plate locating points from the external image of the vehicle comprises the steps of:
detecting four license plate vertices from an external image of the vehicle;
outputting quadrangles formed by the vertexes of the four license plates;
And taking intersection points of the diagonal lines of the quadrangles as license plate positioning points.
3. The intelligent vision-based vehicle bottom centering inspection method as claimed in claim 1, wherein the method for acquiring license plate locating points from the external image of the vehicle comprises the steps of:
detecting four license plate vertices from an external image of the vehicle;
outputting quadrangles formed by the vertexes of the four license plates;
Detecting all internal angle angles of the quadrangle, and determining the internal angle closest to the right angle;
and taking the midpoint of the opposite sides of the triangle closest to the right-angle inner angle as a license plate positioning point.
4. A vehicle bottom centering inspection method based on intelligent vision as claimed in any one of claims 1 to 3, wherein the method for judging that the inspection robot moves to the center line of the vehicle head comprises the steps of:
stopping moving when a headstock positioning point is not detected from a continuous multi-frame image outside the vehicle;
acquiring a last frame of vehicle external image containing a headstock positioning point, wherein the vehicle external image is acquired by a depth camera;
acquiring the distance L c from the inspection robot to a license plate locating point in the external image of the last frame of vehicle according to the depth information;
Calculating the distance from a projection point of a license plate locating point on the ground to the inspection robot based on the distance L c S last;
Checking the straight blind travel path S of the robot, wherein S robot is a check robot body length.
5. The intelligent vision-based vehicle bottom centering inspection method as claimed in claim 4, wherein after the inspection robot moves to the center line of the vehicle head, further comprising the steps of:
detecting an included angle alpha between the normal vector direction of the head positioning point of the last frame of vehicle external image and the body of the inspection robot based on the depth information;
The inspection robot rotates in situ by an alpha degree, so that the normal vector direction of the body of the inspection robot and the positioning point of the headstock is parallel.
6. The intelligent vision-based vehicle bottom centering inspection method according to claim 5, wherein the method for judging that the inspection robot moves to the center line of the vehicle tail comprises the steps of:
and checking the straight blind travel path 2S of the robot when the midpoint of the rear wheel is not detected from the continuous multi-frame images of the bottom of the vehicle.
7. An underbody checking system, comprising a checking robot, a workstation and a background control terminal, wherein the checking robot is used for executing the underbody centering checking method based on intelligent vision as set forth in any one of claims 1 to 6, the workstation is used for performing logistical service on the checking robot and receiving monitoring data, and uploading the monitoring data to the background control terminal, and the background control terminal processes, analyzes and detects the monitoring data.
CN202410635508.2A 2024-05-22 2024-05-22 Vehicle bottom centering checking method and vehicle bottom checking system based on intelligent vision Active CN118225770B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202410635508.2A CN118225770B (en) 2024-05-22 2024-05-22 Vehicle bottom centering checking method and vehicle bottom checking system based on intelligent vision

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202410635508.2A CN118225770B (en) 2024-05-22 2024-05-22 Vehicle bottom centering checking method and vehicle bottom checking system based on intelligent vision

Publications (2)

Publication Number Publication Date
CN118225770A CN118225770A (en) 2024-06-21
CN118225770B true CN118225770B (en) 2024-08-27

Family

ID=91501219

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202410635508.2A Active CN118225770B (en) 2024-05-22 2024-05-22 Vehicle bottom centering checking method and vehicle bottom checking system based on intelligent vision

Country Status (1)

Country Link
CN (1) CN118225770B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN119501945A (en) * 2024-12-09 2025-02-25 南京索安电子有限公司 Automatic centering method for underbody inspection robot

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113822840A (en) * 2020-06-19 2021-12-21 现代自动车株式会社 Vehicle bottom inspection system and method
CN116743942A (en) * 2023-07-17 2023-09-12 盛视科技股份有限公司 A two-way splicing vehicle bottom imaging method based on camera array

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117011839B (en) * 2023-07-04 2025-11-04 盛视科技股份有限公司 Security inspection methods, devices, and robots based on point cloud processing
CN117773956A (en) * 2023-11-21 2024-03-29 盛视科技股份有限公司 Vehicle bottom inspection system and vehicle bottom inspection method

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113822840A (en) * 2020-06-19 2021-12-21 现代自动车株式会社 Vehicle bottom inspection system and method
CN116743942A (en) * 2023-07-17 2023-09-12 盛视科技股份有限公司 A two-way splicing vehicle bottom imaging method based on camera array

Also Published As

Publication number Publication date
CN118225770A (en) 2024-06-21

Similar Documents

Publication Publication Date Title
US11623494B1 (en) Sensor calibration and verification using induced motion
US11458912B2 (en) Sensor validation using semantic segmentation information
US10916035B1 (en) Camera calibration using dense depth maps
US11898873B2 (en) Calibrating multiple inertial measurement units
US11555903B1 (en) Sensor calibration using dense depth maps
US7746221B2 (en) Method for preventing lane departure for use with vehicle
US12235113B2 (en) Parking support apparatus
US11904843B2 (en) Autonomous parking systems and methods for vehicles
CN102037735A (en) Self calibration of extrinsic camera parameters for a vehicle camera
CN112257539A (en) Method, system and storage medium for detecting position relation between vehicle and lane line
CN111169468A (en) Automatic parking system and method
JP2011221983A (en) Vehicle environment recognition system
CN114961381B (en) A wheel alignment method
CN108470142B (en) Lane positioning method based on inverse perspective projection and lane distance constraint
CN114092683B (en) Tire deformation identification method and device based on visual feedback and depth network
CN118225770B (en) Vehicle bottom centering checking method and vehicle bottom checking system based on intelligent vision
CN112037265A (en) A storage location tracking method and device
CN112193241A (en) Automatic parking method
CN112215214A (en) Method and system for adjusting camera offset of intelligent vehicle-mounted terminal
CN114140535B (en) Trailer angle identification method and device
CN117036505B (en) Vehicle camera online calibration method and system
CN113147746A (en) Method and device for detecting ramp parking space
CN118226458B (en) Vehicle bottom centering checking method and vehicle bottom checking system based on laser radar
CN120308097A (en) Automatic parking optimization method, system and vehicle
CN118269939B (en) Automatic parking method and device, electronic device, vehicle and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant