CN110825088A - Multi-view vision guiding ship body cleaning robot system and cleaning method - Google Patents

Abstract

The invention relates to a multi-eye vision-guided ship hull cleaning robot system and a cleaning method. The system includes: a plurality of monocular cameras, a hull workstation module, a hull workstation control module and a cleaning robot module connected in sequence, each monocular camera is located on the side of the hull, and each monocular camera is used to collect part of the hull surface image within the visual range, The hull workstation module is used to receive the images of each part of the hull surface and transmit each hull surface image to the hull workstation control module. The image is marked with the area to be cleaned on the hull surface, the location information of the area to be cleaned, and the cleaning path. The cleaning robot module is used to clean the hull according to the received area on the hull surface, the location information of the area to be cleaned, and the cleaning path. The invention can improve the efficiency of ship hull cleaning and improve the effect of ship hull cleaning.

Description

A multi-eye vision-guided ship hull cleaning robot system and cleaning method

technical field

The invention relates to the field of ship hull cleaning, in particular to a multi-eye vision-guided ship hull cleaning robot system and a cleaning method.

Background technique

When a ship travels in the ocean for a long time, the long-term immersion and corrosion in seawater will accelerate the speed of the ship's rust, and the adhesion of a large number of marine microorganisms, rust and debris will reduce the speed of the ship and increase the fuel consumption. Not only delays the sailing period, but also reduces the service life of the ship. Therefore, impurity and rust removal is an indispensable process in the shipbuilding industry. However, at present, the manual cleaning method mainly uses sandblasting to remove impurities and rust, which is not only time-consuming and labor-intensive, but also inefficient. The cleaning effect is average. It will also destroy the wall coating, causing a certain degree of damage to the wall.

SUMMARY OF THE INVENTION

The purpose of the present invention is to provide a multi-eye vision-guided ship hull cleaning robot system and a cleaning method, which can improve the efficiency of ship hull cleaning and the effect of ship hull cleaning.

For achieving the above object, the present invention provides the following scheme:

A multi-eye vision-guided hull cleaning robot system, comprising: a plurality of monocular cameras, a hull workstation module, a hull workstation control module and a cleaning robot module connected in sequence, each of the monocular cameras is located on the side of the hull, and each of the The monocular camera is used to collect part of the hull surface images within the visual range, and the hull workstation module is used to receive each of the partial hull surface images and transmit each of the hull surface images to the hull workstation control module, and the hull workstation The control module processes each of the hull surface image information, completes the stitching of the entire hull image, determines the entire hull image, and marks the hull surface area to be cleaned, the location information of the area to be cleaned and the cleaning path, the cleaning robot module It is used for cleaning the hull according to the received area of the hull surface to be cleaned, the location information of the area to be cleaned and the cleaning path.

Optionally, the cleaning robot module includes a robot body, a lidar, and an infrared sensor, the lidar is used to collect obstacle information on the surface of the ship, and the infrared sensor is used to collect distance information between the robot body and the obstacle, The hull workstation control module is respectively connected with the laser radar and the infrared sensor, and the hull workstation control module is used to identify obstacles according to the obstacle information on the hull surface and the distance information.

Optionally, the cleaning robot module includes a positioning sub-module, and the positioning sub-module is connected to the hull workstation control module.

Optionally, the positioning sub-module includes a gyroscope and a Beidou navigation system, and the gyroscope and the Beidou navigation system are used to locate the position information of the robot body, and the hull workstation control module is respectively connected with the The gyroscope is connected to the Beidou navigation system.

Optionally, the cleaning robot module includes a servo motor and a high-pressure water gun, the hull workstation control module is respectively connected with the servo motor and the high-pressure water gun, and the high-pressure water gun is used to determine the value of the hull workstation control module. The area to be cleaned on the surface of the hull is cleaned, and the servo motor is used to provide power for the robot body.

Optionally, the high-pressure water gun adopts a horizontal 180-degree rotatable and pressure-adjustable high-pressure water gun.

Optionally, the hull workstation control module adopts a multi-threaded layered cooperative control structure.

Optionally, there are two cleaning methods for the cleaning robot module: overall cleaning of the hull and partial cleaning of the hull; for the overall cleaning of the hull, the hull workstation control module is used to control each of the monocular cameras to perform cleaning on the cleaning robot module. Visual guidance ensures that the cleaning robot module completes the tasks of image overlapping area cleaning and cross-area collaborative cleaning; for local hull cleaning, the hull workstation control module plans the cleaning robot module to reach the designated position according to the size and position of the stain area, and completes the Partial cleaning, and judge whether the cleaned area needs a second cleaning.

Optionally, the path planning of the cleaning path by the cleaning robot module is completed based on grid modeling.

A cleaning method for a multi-eye vision-guided ship hull cleaning robot system, comprising:

Obtain part of the hull images collected by multiple monocular cameras;

According to each part of the hull image, the overall image of the hull is stitched to obtain the overall image of the hull;

Determine the area to be cleaned according to the overall image of the hull;

According to the area to be cleaned, select a cleaning mode;

planning a cleaning path according to the cleaning mode and the cleaning area;

Control the cleaning robot body to reach the cleaning area according to the cleaning path to clean the hull;

Judging whether the cleaned hull is qualified;

If so, stop cleaning;

If not, continue to acquire part of the hull images collected by multiple monocular cameras.

According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects:

The invention provides a multi-eye vision-guided ship hull cleaning robot system, the system comprises: a plurality of monocular cameras, a hull workstation module, a hull workstation control module and a cleaning robot module connected in sequence, each of the monocular cameras is located beside the hull Each of the monocular cameras is used to collect part of the hull surface image within the visual range, and the hull workstation module is used to receive each of the part of the hull surface image and transmit each of the hull surface images to the hull workstation control module , the hull workstation control module processes the image information of each hull surface, completes the splicing of the entire hull image, determines the image of the entire hull, and marks the hull surface area to be cleaned, the location information of the area to be cleaned and the cleaning path, The cleaning robot module is used for cleaning the hull according to the received area of the hull surface to be cleaned, the position information of the area to be cleaned and the cleaning path. The invention adopts the method of cleaning the hull of the ship by a robot instead of the traditional method of cleaning the hull manually, thereby improving the cleaning efficiency of the hull and the effect of cleaning the hull.

Description of drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the accompanying drawings required in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some of the present invention. In the embodiments, for those of ordinary skill in the art, other drawings can also be obtained according to these drawings without creative labor.

Fig. 1 is the structure composition diagram of the multi-eye vision-guided hull cleaning robot system of the present invention;

Fig. 2 is the working schematic diagram of the multi-eye vision guidance system of the present invention;

Fig. 3 is the principle diagram of image matching of the present invention to polar constraint;

Fig. 4 is a flow chart of the cleaning method of the multi-vision vision-guided ship hull cleaning robot system of the present invention.

Detailed ways

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

The purpose of the present invention is to provide a multi-eye vision-guided hull cleaning robot system, which can improve the efficiency of hull cleaning and the effect of hull cleaning.

In order to make the above objects, features and advantages of the present invention more clearly understood, the present invention will be described in further detail below with reference to the accompanying drawings and specific embodiments.

Fig. 1 is a structural composition diagram of a multi-eye vision-guided hull cleaning robot system of the present invention. As shown in Figure 1, a multi-eye vision-guided hull cleaning robot system includes: a plurality of monocular cameras 1, a hull workstation module 2, a hull workstation control module 3 and a cleaning robot module 4 connected in sequence, each of the monocular cameras The camera 1 is located on the side of the hull, each of the monocular cameras 1 is used to collect part of the hull surface image within the visual range, and the hull workstation module 2 is used to receive each of the part of the hull surface image and transmit each of the hull surface images. To the hull workstation control module 3, the hull workstation control module 3 processes the image information of each hull surface, completes the splicing of the entire hull image, determines the image of the entire hull, and marks the area to be cleaned and the area to be cleaned on the hull surface. The location information and cleaning path of the cleaning area, the cleaning robot module 4 is configured to clean the hull according to the received area of the hull surface to be cleaned, the location information of the area to be cleaned, and the cleaning path. The hull workstation module 2 supplies power to the cleaning robot module 4. The hull workstation module 2 can monitor the power status of the cleaning robot in real time, predict the usage, and issue a voice prompt when the cleaning task is completed or the power is insufficient.

The cleaning robot module 4 includes a robot body, a lidar, and an infrared sensor. The lidar is used to collect information on obstacles on the surface of the hull. The infrared sensor is used to collect distance information between the robot body and the obstacles. The workstation control module 3 is respectively connected with the lidar and the infrared sensor, and the hull workstation control module 3 is used to detect obstacles according to the obstacle information on the hull surface and the distance information between the robot body and the obstacle. to identify. The cleaning robot module 4 includes a positioning sub-module, and the positioning sub-module is connected with the hull workstation control module 3 . The positioning sub-module includes a gyroscope and a Beidou navigation system. The gyroscope and the Beidou navigation system are used to locate the position information of the robot body. The hull workstation control module 3 is respectively connected with the gyroscope and the Beidou navigation system. The Beidou navigation system is connected. The cleaning robot module 4 includes a servo motor and a high-pressure water gun, and the hull workstation control module 3 is respectively connected with the servo motor and the high-pressure water gun, and the high-pressure water gun is used for the hull determined by the hull workstation control module 3. The surface area to be cleaned is cleaned, and the servo motor is used to provide power for the robot body. The high-pressure water gun adopts a horizontal 180-degree rotatable and pressure-adjustable high-pressure water gun. The robot body adopts the structure of permanent magnet adsorption and all-terrain dual track chassis.

The hull workstation control module 3 adopts a multi-threaded hierarchical cooperative control structure, which includes: a main thread, an equipment cooperation layer, a path planning layer and an action execution layer; The thread controls the cleaning robot module 4 to complete the cleaning work; the equipment cooperation layer includes communication and data transmission between various threads, assisting the sub-threads to process data; the path planning layer includes planning the cleaning route of the cleaning robot module 4 and the autonomous planning part Clean the route; the action execution layer completes the command issued by the above-mentioned sub-thread.

There are two cleaning methods for the cleaning robot module 4: the overall cleaning of the hull and the partial cleaning of the hull; for the overall cleaning of the hull, the hull workstation control module 3 is used to control each of the monocular cameras to perform vision on the cleaning robot module 4. Guidance to ensure that the cleaning robot module 4 completes the tasks of image overlapping area cleaning and cross-region collaborative cleaning; for local cleaning of the hull, the hull workstation control module 3 plans the cleaning robot module 4 to reach the designated position according to the size and location of the stain area , complete the local cleaning, and judge whether the cleaned area needs a second cleaning.

FIG. 2 is a working schematic diagram of the multi-eye vision guidance system of the present invention.

The path planning of the cleaning path by the cleaning robot module 4 is completed based on grid modeling. This method is a planning method that uses a grid array to represent the environment and can deal with changes in obstacles at the same time. The environmental information around the cleaning robot module 4 is stored in a two-dimensional circular buffer in the form of a grid. This buffer can be cyclically updated with the movement of the cleaning robot module 4, and a uniform B-spline is used to represent the trajectory , the trajectory is optimized in a non-linear fashion.

The local trajectory planning problem is represented as a B-spline optimization problem, and the spline value can be calculated by the following formula.

The p _i is a control point corresponding to time t, and B _i,k (t) is a basis function, which can be calculated by the De Boer-Cox recursion formula. The time interval between uniform B-spline control points is fixed.

In order to avoid obstacles on the way, the cleaning robot module 4 uses a 2D circular buffer to represent the environment map. In order to facilitate the query, the plane is discretized into a small square of size r, so that the mapping from any point p in the plane to a specific small square index x, and the inverse mapping are established. The circular buffer consists of a contiguous array of size N and an offset index o that defines the position of the coordinate system. The cleaning robot module 4 can check whether the small square corresponding to any point in the plane is within the range represented by the circular buffer area, and its specific storage location.

Constraining the size of the array to N=2 ^p , the above operation can be done in the following two ways:

insideVolume(x)=! ((xo)&(～(2 ^p -1)))

address(x)=(xo)&(2 ^p -1)

Starting from the center point of the sensor, the map is updated using the ray casting method, and the Euclidean distance transform (EDT) is used on the map to query the distance between a point and an obstacle within the map range and the gradient of the distance change.

The optimization of the B-spline is expressed as a nonlinear optimization method, and the optimization function is as follows:

E _total =E _ep +E _c +E _q

The E _ep represents the dissipation function of the global path tracking error

The p(t) is a spline value.

The E _c is the dissipation function of the obstacle distance

The E _q is the smoothness of the dissipation function

The global path is obtained by applying the optimization method, and then the current position is used as the starting point to iterate. Each moment is used as the input of the global path to calculate the tracking target point, which is used as the dissipation function parameter of the tracking error; the parameters of the obstacle dissipation function come from the circular buffer area and the EDT. After each optimization, the first control point among the currently optimized control points is fixed and sent to the controller to calculate the new control input. And new control points will be added, and the cleaning path of the cleaning robot module 4 is obtained in a cycle.

Using multiple monocular cameras on the side of the hull as the coordinate system, adjacent cameras observe the same area as the matching area, and by preprocessing the image, Hessian-affine feature detection is used to extract feature points, and compatibility-based mining is used. method to search for consistent neighbors for image matching. In image matching, if two pairs of points correspond correctly, then the local affine transformations corresponding to the two pairs of points should be similar.

The Harris-Laplace scale-invariant operator is used to detect corners on the scale space image according to the image collected by the monocular camera, and the scale parameter is added. Search each candidate point on the current scale image to calculate the Laplacian response value, and the feature points that satisfy the condition that the absolute maximum value of the Harris matrix is greater than the given threshold are reserved.

F(x, y, σ _n )=σ ² |L _xx (x, y, σ _n )+L _yy (x, y, σ _n )|≥threshold _L

where σ _n is the scale factor of each layer image, and threshold _L is the threshold condition.

The detected corners are compared with the adjacent Laplace response values of the upper and lower layers, and the response value of the current layer is greater than that of the adjacent upper and lower layers.

The scale feature that satisfies the above two steps is the scale-invariant feature point extracted in the scale space, and the local affine transformation information of the feature point is obtained when the feature point is extracted.

The above formula A is affine information.

The transformation relationship between matching points can be obtained from the above local affine transformation:

A_i为局部仿射信息，K_i为特征点的坐标。in

A _i is the local affine information, and K _i is the coordinate of the feature point.

For a pair of feature point correspondences, the transformation relationships between the points are matched respectively, and the similarity of the two transformation relationships is calculated, which is used as the consistency measure of the point correspondence:

Among them, ρ represents the transformed coordinates, e represents the degree of similarity of the corresponding points after transformation, and the Gaussian kernel is used to normalize the above indicators:

For any point corresponding to c _i , the closest k points corresponding to it are found through the above method, and the composition graph G _i is the corresponding set of local consistency points corresponding to the point.

FIG. 3 is a schematic diagram of the image matching polar constraint principle of the present invention. As shown in Figure 3, three points o ₁ , o ₂ and P can determine a plane, called the polar plane. The intersection points of the line connecting o ₁ and o ₂ with the image plane I ₁ and I ₂ are e ₁ and e ₂ respectively, e ₁ and e ₂ are called poles, and o ₁ o ₂ is called baseline. The intersection line l ₁ l ₂ between the polar plane and the two image planes I ₁ I ₂ is called the polar line. It can be seen from the image that the feature points on I ₂ corresponding to p ₁ must fall on the polar line of l ₂ .

F为已知基础矩阵，则p₁对应的极线方程为Assuming that the homogeneous coordinates of p ₁ are (x,y,1) ^T , the homogeneous coordinates of p ₂ are

F is a known fundamental matrix, then the epipolar equation corresponding to p ₁ is

The image feature point error is considered to satisfy the normal distribution of (u, σ), p ₂ must be on the epipolar line l, and the distance from the point p ₂ to the epipolar line l is less than 3σ.

a=f ₀₀ *x+f ₀₁ *y+f ₀₂

b=f ₁₀ *x+f ₁₁ *y+f ₁₂

c=f ₂₀ *x+f ₂₁ *y+f ₂₂

Through the above steps, the matching points between the images can be obtained, so as to complete the visual stitching and obtain the entire image of the hull.

Fig. 4 is a flow chart of the cleaning method of the multi-vision vision-guided ship hull cleaning robot system of the present invention. As shown in Figure 4, a cleaning method for a multi-eye vision-guided hull cleaning robot system includes:

Step 101: Acquire part of the hull images collected by multiple monocular cameras.

Step 102 : Perform overall image stitching of the hull according to each of the partial hull images to obtain the overall hull image. Specifically, the hull workstation module receives the partial hull images collected by a plurality of monocular cameras for stitching.

Step 103: Determine the area to be cleaned according to the overall image of the hull, specifically, determine the area size and location information of the area to be cleaned.

Step 104: Select a cleaning mode according to the area to be cleaned. Specifically, the cleaning mode includes two types: overall cleaning of the hull and partial cleaning of the hull; for the overall cleaning of the hull, control each of the monocular cameras to perform cleaning on the cleaning robot module. Visual guidance ensures that the cleaning robot module completes the tasks of image overlapping area cleaning and cross-area collaborative cleaning; for local cleaning of the hull, according to the size and location of the stain area, plan the cleaning robot module to reach the designated position, complete the local cleaning, and clean after cleaning. The area is judged whether it needs secondary cleaning.

Step 105: Plan a cleaning path according to the cleaning mode and the cleaning area.

Step 106: Control the cleaning robot body to reach the cleaning area to clean the hull according to the cleaning path.

Step 107: Determine whether the cleaned hull is qualified. Specifically, it is determined whether the hull is not cleaned properly by collecting images of the hull.

Step 108: If the cleaned hull is qualified, stop cleaning.

Step 109: If the cleaned hull is unqualified, continue to acquire part of the hull images collected by the multiple monocular cameras, that is, perform secondary cleaning.

The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same and similar parts between the various embodiments can be referred to each other.

In this paper, specific examples are used to illustrate the principles and implementations of the present invention. The descriptions of the above embodiments are only used to help understand the methods and core ideas of the present invention; meanwhile, for those skilled in the art, according to the present invention There will be changes in the specific implementation and application scope. In conclusion, the contents of this specification should not be construed as limiting the present invention.

Claims (10)

1. a multi-eye vision-guided hull cleaning robot system is characterized in that, comprising: a plurality of monocular cameras, hull workstation modules, hull workstation control modules and cleaning robot modules connected in turn, each described monocular camera is located in the hull. On the side, each of the monocular cameras is used to collect part of the hull surface image within the visual range, and the hull workstation module is used to receive each of the part of the hull surface image and transmit each of the hull surface images to the hull workstation for control module, the hull workstation control module processes the image information of each hull surface, completes the splicing of the entire hull image, determines the image of the entire hull, and marks the hull surface area to be cleaned, the location information of the area to be cleaned, and the cleaning path , the cleaning robot module is configured to clean the hull according to the received area of the hull surface to be cleaned, the position information of the area to be cleaned and the cleaning path. 2 . The multi-eye vision-guided ship hull cleaning robot system according to claim 1 , wherein the cleaning robot module comprises a robot body, a laser radar and an infrared sensor, and the laser radar is used to collect obstacle information on the hull surface. 3 . , the infrared sensor is used to collect the distance information between the robot body and the obstacle, the hull workstation control module is respectively connected with the lidar and the infrared sensor, and the hull workstation control module is used for according to the hull The obstacle is identified by the surface obstacle information and the distance information between the robot body and the obstacle. 3 . The multi-vision vision-guided hull cleaning robot system according to claim 2 , wherein the cleaning robot module comprises a positioning sub-module, and the positioning sub-module is connected with the hull workstation control module. 4 . 4. The multi-vision vision-guided hull cleaning robot system according to claim 3, wherein the positioning submodule comprises a gyroscope and a Beidou navigation system, and the gyroscope and the Beidou navigation system are used for The position information of the robot body is used for positioning, and the hull workstation control module is respectively connected with the gyroscope and the Beidou navigation system. 5. The multi-eye vision-guided hull cleaning robot system according to claim 4, wherein the cleaning robot module comprises a servo motor and a high-pressure water gun, and the hull workstation control module is respectively connected with the servo motor and the A high-pressure water gun is connected, the high-pressure water gun is used to clean the area to be cleaned on the hull surface determined by the hull workstation control module, and the servo motor is used to provide power for the robot body. 6 . The multi-eye vision-guided ship hull cleaning robot system according to claim 5 , wherein the high-pressure water gun is a high-pressure water gun that can be rotated by 180 degrees horizontally and can be adjusted in pressure. 7 . 7 . The multi-eye vision-guided hull cleaning robot system according to claim 1 , wherein the hull workstation control module adopts a multi-thread layered cooperative control structure. 8 . 8 . The multi-eye vision-guided hull cleaning robot system according to claim 1 , wherein the cleaning robot module has two cleaning methods: overall hull cleaning and partial hull cleaning; for overall hull cleaning, the hull cleaning The workstation control module is used to control each of the monocular cameras to visually guide the cleaning robot module, so as to ensure that the cleaning robot module completes the tasks of image overlapping area cleaning and cross-area collaborative cleaning; for local hull cleaning, the hull workstation controls The module plans the cleaning robot module to reach the designated position according to the size and position of the stained area, completes local cleaning, and judges whether secondary cleaning is required for the cleaned area. 9 . The multi-vision vision-guided hull cleaning robot system according to claim 1 , wherein the path planning of the cleaning path by the cleaning robot module is completed based on grid form modeling. 10 . 10. A method for cleaning a robot system for cleaning a ship hull with a multi-eye vision guide, wherein the method is applied to the robot system for cleaning a ship hull with a multi-eye vision guide according to any one of claims 1 to 9, and the method comprises the following steps: : Obtain part of the hull images collected by multiple monocular cameras; According to each part of the hull image, the overall image of the hull is stitched to obtain the overall image of the hull; Determine the area to be cleaned according to the overall image of the hull; According to the area to be cleaned, select a cleaning mode; planning a cleaning path according to the cleaning mode and the cleaning area; Control the cleaning robot body to reach the cleaning area according to the cleaning path to clean the hull; Judging whether the cleaned hull is qualified; If so, stop cleaning; If not, continue to acquire part of the hull images collected by multiple monocular cameras.

Images