CN116152208B - Defect detection method, device, equipment and storage medium - Google Patents
Defect detection method, device, equipment and storage mediumInfo
- Publication number
- CN116152208B CN116152208B CN202310172014.0A CN202310172014A CN116152208B CN 116152208 B CN116152208 B CN 116152208B CN 202310172014 A CN202310172014 A CN 202310172014A CN 116152208 B CN116152208 B CN 116152208B
- Authority
- CN
- China
- Prior art keywords
- point cloud
- component
- template
- measured
- depth map
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
- G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
- G06T7/337—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods involving reference images or patches
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/50—Depth or shape recovery
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
- G06T7/62—Analysis of geometric attributes of area, perimeter, diameter or volume
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10028—Range image; Depth image; 3D point clouds
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/30—Computing systems specially adapted for manufacturing
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Geometry (AREA)
- Quality & Reliability (AREA)
- Length Measuring Devices By Optical Means (AREA)
Abstract
The invention provides a defect detection method, device, equipment and storage medium, which are characterized in that a part to-be-detected point cloud is obtained, and a target part template point cloud is obtained, wherein the target part template point cloud is a reference template point cloud for judging whether the part to-be-detected point cloud has attachment defects, the part to-be-detected point cloud and the target part template point cloud are respectively reconstructed into a part to-be-detected depth map and a target part template depth map, and whether the part to-be-detected point cloud has attachment defects is determined by performing binarization processing and Blob analysis on the part to-be-detected depth map and the target part template depth map, so that the defect detection rate is effectively improved, and a large amount of manpower and material resources are saved.
Description
Technical Field
The present disclosure relates to the field of computer technologies, and in particular, to a defect detection method, device, apparatus, and storage medium.
Background
The defect detection method of the notebook internal structure based on 3D vision can be understood as a point cloud defect detection problem, specifically, a corresponding matching area is found in an image to be detected through a point cloud registration method based on a designated area in a template image, and defect detection is performed. The prior art commonly uses a defect detection method based on deep learning and a defect detection method based on template contrast for 3D visual detection methods.
Although the neural network training method based on the 3D point cloud has a certain effect on defect detection in theory, in the actual implementation process, collecting a large number of defect samples is very difficult, so that the detection accuracy is reduced.
The template comparison defect detection method based on the 3D point cloud is used for extracting the positions of the standard template and the part in the point cloud to be detected to compare the point cloud information, so that the problem of whether the part has the attachment defect or not is judged. The method has the problems of missing detection and false detection, and particularly has poor defect detection effect when the phenomenon of high protrusion exists near the position of the attached part or the phenomenon of little deviation exists on the attached part.
Disclosure of Invention
The present disclosure provides a defect detection method, apparatus, device, and storage medium, to at least solve the above technical problems in the prior art.
According to a first aspect of the present disclosure, there is provided a defect detection method, the method comprising:
Acquiring a part to-be-measured point cloud and a target part template point cloud, wherein the target part template point cloud is a reference template point cloud for judging whether the part to-be-measured point cloud has an attachment defect or not;
reconstructing the component to-be-measured point cloud and the target component template point cloud into a component to-be-measured depth map and a target component template depth map respectively;
And determining whether the cloud of the part to be measured point has an attachment defect or not by carrying out binarization processing and Blob analysis on the part to be measured depth map and the target part template depth map.
In an embodiment, the determining whether the cloud of the part to be measured has the attachment defect by performing binarization processing and Blob analysis on the depth map of the part to be measured and the depth map of the target part template includes:
Respectively carrying out binarization processing on the part to-be-detected depth map and the target part template depth map based on the depth mean value corresponding to the part template point cloud to obtain a part to-be-detected binary map and a target part template binary map;
respectively acquiring a first preset area in the binary image to be tested of the component and a second preset area in the binary image of the target component template;
comparing the area information of the first preset area and the second preset area to determine whether the cloud of the part to be tested has an attachment defect, wherein the area information comprises at least one of width, height and area.
In an embodiment, the comparing the area information of the first preset area and the second preset area to determine whether the cloud of the part to be tested has the attachment defect includes:
when the width difference value of the first preset area and the second preset area is smaller than a preset width threshold value, the height difference value of the first preset area and the second preset area is smaller than a preset height threshold value, and the area difference value of the first preset area and the second preset area is smaller than a preset area threshold value, determining that the part to-be-measured point cloud has no attachment defect, otherwise, determining that the part to-be-measured point cloud has the attachment defect.
In an embodiment, the obtaining the target component template point cloud includes:
Acquiring a point cloud to be detected comprising at least one attaching part to be detected and a template point cloud of a sample attaching part, wherein labels of all the sample attaching parts are marked in the template point cloud;
Calibrating the template point cloud based on a point cloud registration algorithm and the point cloud to be detected comprising at least one attached component to be detected, so as to obtain a corrected template point cloud;
and determining at least one sample part template point cloud on the corrected template point cloud, and taking each sample part template point cloud as a target part template point cloud in sequence.
In an embodiment, the obtaining the template point cloud of the sample attachment component includes:
Determining source point cloud data of the sample attached component by shooting the sample attached component through an original 3D design drawing of the sample attached component or through a 3D camera;
Modeling the source point cloud data of the sample attached component to obtain a template point cloud of the sample attached component.
In an embodiment, the acquiring the component point cloud to be measured includes:
performing dimension reduction processing on the corrected template point cloud and the point cloud to be measured containing at least one attached component to be measured to obtain dimension reduction template point cloud and dimension reduction point cloud to be measured;
Determining a dimension reduction component template point cloud corresponding to the target component template point cloud in the dimension reduction template point cloud;
And determining a dimension reduction component point cloud corresponding to the dimension reduction component template point cloud in the dimension reduction component point cloud based on a neighborhood minimum method, and determining a to-be-measured point cloud area corresponding to the dimension reduction component point cloud as a component to-be-measured point cloud on the to-be-measured point cloud containing at least one to-be-measured attachment component through a mapping relation between the dimension reduction to-be-measured point cloud and the to-be-measured point cloud containing at least one to-be-measured attachment component.
In an embodiment, the reconstructing the component to-be-measured point cloud and the target component template point cloud into a component to-be-measured depth map and a target component template depth map respectively includes:
respectively acquiring coordinates of each voxel in the component to-be-measured point cloud and the target component template point cloud, wherein the coordinates comprise an X value, a Y value and a Z value;
taking the X value and the Y value of each voxel in the component to-be-measured point cloud as coordinate values of the component to-be-measured depth map, and taking the Z value as pixel values of the component to-be-measured depth map to reconstruct the component to-be-measured depth map;
And taking the X value and the Y value of each voxel in the target component template point cloud as coordinate values of the target component template depth map, and taking the Z value as pixel values of the target component template depth map so as to reconstruct the target component template depth map.
According to a second aspect of the present disclosure, there is provided a defect detection apparatus, the apparatus comprising:
The device comprises a point cloud acquisition module, a target component template point cloud, a target component detection module and a target component detection module, wherein the point cloud acquisition module is used for acquiring a component to-be-detected point cloud and acquiring the target component template point cloud, wherein the target component template point cloud is a reference template point cloud for judging whether the component to-be-detected point cloud has an attachment defect or not;
the depth map reconstruction module is used for reconstructing the component to-be-measured point cloud and the target component template point cloud into a component to-be-measured depth map and a target component template depth map respectively;
And the defect analysis module is used for determining whether the cloud of the part to be measured points has attachment defects or not by carrying out binarization processing and Blob analysis on the part to be measured depth map and the target part template depth map.
In one embodiment, the defect analysis module is specifically configured to:
Respectively carrying out binarization processing on the part to-be-detected depth map and the target part template depth map based on the depth mean value corresponding to the part template point cloud to obtain a part to-be-detected binary map and a target part template binary map;
respectively acquiring a first preset area in the binary image to be tested of the component and a second preset area in the binary image of the target component template;
comparing the area information of the first preset area and the second preset area to determine whether the cloud of the part to be tested has an attachment defect, wherein the area information comprises at least one of width, height and area.
In an embodiment, the defect analysis module is further specifically configured to:
when the width difference value of the first preset area and the second preset area is smaller than a preset width threshold value, the height difference value of the first preset area and the second preset area is smaller than a preset height threshold value, and the area difference value of the first preset area and the second preset area is smaller than a preset area threshold value, determining that the part to-be-measured point cloud has no attachment defect, otherwise, determining that the part to-be-measured point cloud has the attachment defect.
In an embodiment, the point cloud acquisition module is specifically configured to:
Acquiring a point cloud to be detected comprising at least one attaching part to be detected and a template point cloud of a sample attaching part, wherein labels of all the sample attaching parts are marked in the template point cloud;
Calibrating the template point cloud based on a point cloud registration algorithm and the point cloud to be detected comprising at least one attached component to be detected, so as to obtain a corrected template point cloud;
and determining at least one sample part template point cloud on the corrected template point cloud, and taking each sample part template point cloud as a target part template point cloud in sequence.
In an embodiment, the point cloud acquisition module is further specifically configured to:
Determining source point cloud data of the sample attached component by shooting the sample attached component through an original 3D design drawing of the sample attached component or through a 3D camera;
Modeling the source point cloud data of the sample attached component to obtain a template point cloud of the sample attached component.
In an embodiment, the point cloud acquisition module is further specifically configured to:
performing dimension reduction processing on the corrected template point cloud and the point cloud to be measured containing at least one attached component to be measured to obtain dimension reduction template point cloud and dimension reduction point cloud to be measured;
Determining a dimension reduction component template point cloud corresponding to the target component template point cloud in the dimension reduction template point cloud;
And determining a dimension reduction component point cloud corresponding to the dimension reduction component template point cloud in the dimension reduction component point cloud based on a neighborhood minimum method, and determining a to-be-measured point cloud area corresponding to the dimension reduction component point cloud as a component to-be-measured point cloud on the to-be-measured point cloud containing at least one to-be-measured attachment component through a mapping relation between the dimension reduction to-be-measured point cloud and the to-be-measured point cloud containing at least one to-be-measured attachment component.
In an embodiment, the depth map reconstruction module is specifically configured to:
respectively acquiring coordinates of each voxel in the component to-be-measured point cloud and the target component template point cloud, wherein the coordinates comprise an X value, a Y value and a Z value;
taking the X value and the Y value of each voxel in the component to-be-measured point cloud as coordinate values of the component to-be-measured depth map, and taking the Z value as pixel values of the component to-be-measured depth map to reconstruct the component to-be-measured depth map;
And taking the X value and the Y value of each voxel in the target component template point cloud as coordinate values of the target component template depth map, and taking the Z value as pixel values of the target component template depth map so as to reconstruct the target component template depth map.
According to a third aspect of the present disclosure, there is provided an electronic device comprising:
At least one processor, and
A memory communicatively coupled to the at least one processor, wherein,
The memory stores instructions executable by the at least one processor to enable the at least one processor to perform the methods described in the present disclosure.
According to a fourth aspect of the present disclosure, there is provided a non-transitory computer readable storage medium storing computer instructions for causing the computer to perform the method of the present disclosure.
The defect detection method, device, equipment and storage medium are characterized by acquiring a part to-be-detected point cloud and a target part template point cloud, wherein the target part template point cloud is a reference template point cloud for judging whether the part to-be-detected point cloud has attachment defects, reconstructing the part to-be-detected point cloud and the target part template point cloud into a part to-be-detected depth map and a target part template depth map respectively, and determining whether the part to-be-detected point cloud has attachment defects through binarization processing and Blob analysis on the part to-be-detected depth map and the target part template depth map.
It should be understood that the description in this section is not intended to identify key or critical features of the embodiments of the disclosure, nor is it intended to be used to limit the scope of the disclosure. Other features of the present disclosure will become apparent from the following specification.
Drawings
The above, as well as additional purposes, features, and advantages of exemplary embodiments of the present disclosure will become readily apparent from the following detailed description when read in conjunction with the accompanying drawings. Several embodiments of the present disclosure are illustrated by way of example, and not by way of limitation, in the figures of the accompanying drawings, in which:
In the drawings, the same or corresponding reference numerals indicate the same or corresponding parts.
Fig. 1 is a schematic implementation flow diagram of a defect detection method according to a first embodiment of the disclosure;
fig. 2 is a schematic implementation flow diagram of a defect detection method according to a second embodiment of the disclosure;
fig. 3 is a flowchart illustrating an implementation of a defect detection method according to a second embodiment of the present disclosure;
fig. 4 is a schematic structural diagram of a defect detecting device according to a third embodiment of the present disclosure;
Fig. 5 shows a schematic diagram of a composition structure of an electronic device according to an embodiment of the present disclosure.
Detailed Description
In order to make the objects, features and advantages of the present disclosure more comprehensible, the technical solutions in the embodiments of the present disclosure will be clearly described in conjunction with the accompanying drawings in the embodiments of the present disclosure, and it is apparent that the described embodiments are only some embodiments of the present disclosure, but not all embodiments. Based on the embodiments in this disclosure, all other embodiments that a person skilled in the art would obtain without making any inventive effort are within the scope of protection of this disclosure.
Example 1
Fig. 1 is a flowchart of a defect detection method according to a first embodiment of the present disclosure, where the method may be performed by a defect detection apparatus according to an embodiment of the present disclosure, and the apparatus may be implemented in software and/or hardware. The method specifically comprises the following steps:
s110, acquiring a part to-be-measured point cloud and a target part template point cloud.
The target component template point cloud may be a reference template point cloud for determining whether the component to be tested has an attachment defect, and is denoted as roi_model. The component to-be-measured point cloud may be a point cloud obtained by shooting the to-be-measured component by using a 3D camera, and is denoted as roi_dst.
Specifically, the attached positions in the notebook are different, the requirements on the types of attached components are different, and the diversity and complexity of the attached positions can seriously influence the attached detection effect of the components. The attached parts in the notebook can be of conductive cloth, foot pads, iron sheets, iron rods, wiredrawing and the like, and the attached parts of the same type are easy to have different colors due to different manufacturers, and the attached positions can be flat positions, corner positions, interference and the like according to the diversity and complexity of the attached positions, and the detection accuracy can be influenced.
The method comprises the steps of obtaining a target component template point cloud, obtaining a to-be-detected point cloud containing at least one to-be-detected attached component and obtaining a template point cloud of a sample attached component, wherein labels of all sample attached components are marked in the template point cloud, calibrating the template point cloud based on a point cloud registration algorithm and the to-be-detected point cloud containing at least one to-be-detected attached component to obtain a corrected template point cloud, determining at least one sample component template point cloud on the corrected template point cloud, and sequentially taking each sample component template point cloud as the target component template point cloud.
The point cloud registration algorithm may be an algorithm for extracting feature points based on voxel segmentation matching, and by way of example, the point cloud registration algorithm adopted in the embodiment may be a nearest point iterative algorithm (ITERATIVE CLOSEST POINT, ICP). The template point cloud, which may be a source point cloud containing the individual sample attachment component identifications that has not been calibrated, is denoted pt_model. The point cloud to be measured is a point cloud including a plurality of component point clouds to be measured, and is used for measuring whether the point cloud has defects, and is denoted as pt_dst.
Specifically, in order to obtain an accurate template point cloud, in this embodiment, the point cloud to be measured including at least one attached component to be measured and the template point cloud with the identifier of each attached component sample need to be obtained first. In theory, if the template point cloud and the point cloud to be detected are shot under the conditions of ensuring the same detection mechanism, the same position, the same camera, the same design parameters and the like, correction is not needed. However, in the actual operation process, due to the possible tiny deviation of each link and the interference of factors such as the deviation of the design accuracy, the deviation may exist between the template point cloud and the point cloud to be measured, so in order to solve the above problem, in this embodiment, the calibration is performed on the template point cloud by the point cloud-based registration algorithm and the point cloud to be measured including at least one attachment component to be measured, so that the template point cloud is corrected according to the point cloud to be measured, thereby obtaining the precisely calibrated template point cloud, which is denoted as pt_model'.
Specifically, since at least one sample attached part point cloud exists on the calibrated template point cloud, the embodiment can sequentially extract and segment each sample attached part point cloud, and sequentially uses the segmented independent parts as target part template point clouds, so that the subsequent comparison with the corresponding attached part point clouds to be detected is facilitated.
In the embodiment of the disclosure, the method for obtaining the template point cloud of the sample attachment part comprises the steps of shooting the sample attachment part through an original 3D design drawing of the sample attachment part or through a 3D camera, determining source point cloud data of the sample attachment part, and modeling the source point cloud data of the sample attachment part to obtain the template point cloud of the sample attachment part.
The source point cloud data may be obtained through a 3D camera or a 3D design drawing, and includes point cloud data of all sample attachment parts.
Specifically, in order to obtain a template point cloud of a sample attached component, in this embodiment, source point cloud data including all sample attached components may be obtained through an original 3D design drawing of the sample attached component or a sample attached component captured by a 3D camera, and then the obtained source point cloud data is subjected to modeling processing, so as to obtain a template point cloud including each attached component, that is, a template point cloud of the sample attached component, and then each sample attached component is marked, so that a subsequent segmentation comparison is facilitated. For example, in this embodiment, if the source point cloud data of the sample attachment component is obtained by directly shooting with a 3D camera, modeling may be performed manually, and each attachment component to be tested may be marked.
The method comprises the steps of carrying out dimension reduction processing on a corrected template point cloud and a point cloud to be measured containing at least one to-be-measured attached component to obtain a dimension reduction template point cloud and a dimension reduction point cloud, determining a dimension reduction component template point cloud corresponding to a target component template point cloud in the dimension reduction template point cloud, determining a dimension reduction component point cloud corresponding to the dimension reduction component template point cloud in the dimension reduction point cloud based on a neighborhood minimum method, and determining a point cloud region to be measured corresponding to the dimension reduction component point cloud as the point cloud to be measured of the component on the point cloud to be measured containing at least one to-be-measured attached component through a mapping relation between the dimension reduction point cloud and the point cloud to be measured containing at least one to-be-measured attached component.
The dimension-reducing template point cloud may be a template point cloud with a Z-axis coordinate value of 0, and is denoted as pt_model. The dimension-reducing point cloud to be measured can be a point cloud to be measured with Z-axis coordinate value of 0, and is recorded as pt_dst. The neighborhood minimum method may be a method for matching the dimension-reduced point cloud to be measured with the corresponding dimension-reduced component template point cloud, and the neighborhood minimum method adopted in this embodiment is an 8-neighborhood minimum method. The dimension reduction component template point cloud refers to an independent component region segmented in the dimension reduction template point cloud and is marked as ROI_model' [ N ]. The dimension reduction component point cloud can be an attached component point cloud with the Z-axis coordinate value of 0, and is marked as ROI_dst' [ N ] corresponding to the dimension reduction component template point cloud. The component to-be-measured point cloud can be a point cloud region corresponding to the dimension reduction component point cloud on the to-be-measured point cloud, and is recorded as ROI_dst [ N ].
Specifically, because the information amount of the point clouds is large, the operation speed of the system is slow, and because the interference of the height difference of the notebook internal structure is easy to cause the system judgment error, in order to improve the operation efficiency of the system and reduce the judgment error rate of the system, in this embodiment, the obtained corrected template point clouds and the point clouds to be measured containing at least one attached component to be measured are subjected to dimension reduction processing, that is, the Z-axis coordinate values of the point clouds are set to 0, so as to obtain the template point clouds with the Z-axis coordinate value of 0, and the point clouds to be measured containing at least one attached component to be measured with the Z-axis coordinate value of 0, so as to obtain the dimension reduction template point clouds and the dimension reduction point clouds to be measured. Because the dimension reduction template point cloud is an integral template, in the dimension reduction template point cloud, the dimension reduction component template point cloud corresponding to the local target component template point cloud can be determined according to the selected local target component template point cloud. In the dimension reduction to-be-measured point cloud, the embodiment obtains the dimension reduction component point cloud corresponding to the dimension reduction component template point cloud by using a neighborhood minimum method according to the dimension reduction component template point cloud and the X coordinate and the Y coordinate of the dimension reduction component point cloud. Then, according to the mapping relation between the dimension reduction to-be-measured point cloud and the to-be-measured point cloud containing at least one to-be-measured attached component, the embodiment determines a to-be-measured point cloud area corresponding to the dimension reduction component point cloud, namely the component to-be-measured point cloud, on the to-be-measured point cloud.
According to the dimension reduction method, the dimension reduction component template point cloud which needs to be matched is obtained from the dimension reduction template point cloud after dimension reduction processing, the dimension reduction component point cloud is obtained through two-dimensional matching, and finally, the mapping relation between the dimension reduction component point cloud and the three-dimensional to-be-measured point cloud is determined, so that the to-be-measured point cloud of the component can be determined, the interference of the height of the structure in the notebook can be eliminated, and meanwhile, the possibility of judging errors is reduced.
S120, reconstructing the component to-be-measured point cloud and the target component template point cloud into a component to-be-measured depth map and a target component template depth map respectively.
The depth map to be measured of the component can be obtained through component measurement point cloud and is used for generating an image of a binary map to be measured of the component, and the image is recorded as dep_dst. The target component template depth map may be obtained through a target component template point cloud, and is used to generate an image of a target component template binary map, and is denoted as dep_model.
In the embodiment of the disclosure, reconstructing a part to-be-measured point cloud and a target part template point cloud into a part to-be-measured depth map and a target part template depth map respectively, wherein the coordinates of each voxel in the part to-be-measured point cloud and the target part template point cloud are obtained respectively, the coordinates comprise an X value, a Y value and a Z value, the X value and the Y value of each voxel in the part to-be-measured point cloud are used as coordinate values of the part to-be-measured depth map, the Z value is used as a pixel value of the part to-be-measured depth map to reconstruct the part to-be-measured depth map, the X value and the Y value of each voxel in the target part template point cloud are used as coordinate values of the target part template depth map, and the Z value is used as a pixel value of the target part template depth map to reconstruct the target part template depth map.
Specifically, in order to determine whether the component to be tested has an attachment abnormality, in this embodiment, the component to be tested point cloud and the target component template point cloud need to be converted into a component to be tested depth map and a target component template depth map that can perform binarization processing and Blob analysis operation. The method comprises the steps of firstly obtaining X, Y and Z-axis coordinate values of each voxel corresponding to a component to-be-measured point cloud and a target component template point cloud, then taking X, Y-axis coordinate values of each voxel in the component to-be-measured point cloud as component to-be-measured depth map coordinate values after dimension reduction, and taking Z-axis coordinate values of each voxel as pixel values of the component to-be-measured depth map, so that a reconstructed component to-be-measured depth map is obtained. Similarly, in this embodiment, the X, Y axis coordinate value of each voxel in the target component template point cloud is used as the coordinate value of the target component template depth map after the dimension reduction, and the Z axis coordinate value of each voxel is used as the pixel value of the target component template depth map, so as to obtain the reconstructed target component template depth map.
S130, determining whether the cloud of the part to be tested has the attachment defect or not by carrying out binarization processing and Blob analysis on the part to be tested depth map and the target part template depth map.
The binarization processing may be an operation of converting the part to-be-measured depth map and the target part template depth map into a gray scale map. The Blob analysis may be an operation of analyzing physical characteristics such as width, height, area and the like of the binary image to be tested of the component and the binary image of the target component template, where the Blob analysis method adopted in the embodiment may be FindContours in an Open cv algorithm library.
Specifically, in order to determine whether the component to be measured point cloud has an attachment defect, the embodiment obtains the component to be measured binary image and the target component template binary image by binarizing the obtained component to be measured depth image and the target component template depth image, and then determines whether the component to be measured point cloud has the attachment defect by Blob analysis.
The existing defect detection method is a neural network training method based on 3D point cloud, and has a certain effect on whether the attached defect detection exists or not, but a large amount of manpower is required for collecting, labeling and training samples, and the method is time-consuming, labor-consuming and inconvenient to operate. In addition, this method is good in detecting defects of a component having no high protrusions around a standard attachment position, but is poor in detecting defects when there are high protrusions or little deviation of attachment around the attachment position. The method adopted by the embodiment not only effectively solves the problem of low detection accuracy when the position of the attached component is highly protruding or the attached component is slightly offset, but also overcomes the defect that a neural network training method based on 3D point cloud wastes a large amount of manpower and material resources, and in addition, the algorithm detection speed is effectively improved.
Example two
Fig. 2 is a flowchart of a defect detection method provided by a second embodiment of the present disclosure, where based on the foregoing embodiments, determining whether a part to be measured point cloud has an attachment defect by performing binarization processing and Blob analysis on a part to be measured depth map and a target part template depth map, where the method includes performing binarization processing on the part to be measured depth map and the target part template depth map based on depth averages corresponding to the part template point cloud to obtain a part to be measured binary map and a target part template binary map, respectively obtaining a first preset area in the part to be measured binary map and a second preset area in the target part template binary map, and comparing area information of the first preset area and the second preset area to determine whether the part to be measured point cloud has the attachment defect, where the area information includes at least one of width, height and area, and the method specifically includes:
s210, acquiring a part to-be-measured point cloud and a target part template point cloud.
S220, reconstructing the component to-be-measured point cloud and the target component template point cloud into a component to-be-measured depth map and a target component template depth map respectively.
S230, based on the depth average value corresponding to the component template point cloud, respectively performing binarization processing on the component to-be-detected depth map and the target component template depth map to obtain a component to-be-detected binary map and a target component template binary map.
The binary image to be measured of the component can be a gray image obtained by binarizing the cloud of the component to be measured, and the gray image is recorded as bin_dep_model. The target component template binary image can be a gray image obtained by binarizing a target component template point cloud and is recorded as bin_dep_dst.
Specifically, in order to simplify the operation and improve the working efficiency, the embodiment obtains the Z-axis coordinate value of the component template point cloud, so as to obtain a depth average value corresponding to the component template point cloud, and then binarizes the obtained component to-be-detected depth map and the target component template depth map based on the depth average value of the component template point cloud, so as to obtain a component to-be-detected binary map and a target component template binary map corresponding to the component to-be-detected binary map. The binary image to be measured of the component and the binary image of the target component template are black and white gray images.
S240, respectively acquiring a first preset area in the binary image of the component to be detected and a second preset area in the binary image of the target component template.
The first preset area may be a white area in the binary image to be tested of the component. The second preset area may be a white area within the target component template binary image.
In the embodiment, a depth average value corresponding to the component template point cloud is used as a reference and converted into a component to-be-detected binary image and a target component template binary image. Since the attachment part area is higher than the area of the internal structure of the notebook, the area is displayed as a white area in the above-mentioned binary image. In this embodiment, the white area in the binary image to be tested of the component may be directly used as the first preset area. And taking the white area in the target component template binary image as a second preset area.
S250, comparing the area information of the first preset area and the area information of the second preset area, and determining whether the cloud of the part to be tested points has the attachment defect or not.
Wherein the area information includes at least one of width, height and area.
Specifically, in this embodiment, the first preset area in the obtained binary image of the component to be tested and the second preset area in the binary image of the target component template are compared in a Blob analysis manner, that is, the widths, heights, areas and the like of the white areas in the first preset area and the second preset area are compared, so as to determine whether a defect exists in an attached component in the binary image of the component to be tested, and further determine whether an attached defect exists in the cloud of the component to be tested.
In the embodiment of the disclosure, comparing the area information of the first preset area and the second preset area to determine whether the cloud of the part to be measured points has the attachment defect, wherein the method comprises the steps of determining that the cloud of the part to be measured points does not have the attachment defect when the width difference value of the first preset area and the second preset area is smaller than a preset width threshold value, the height difference value of the first preset area and the second preset area is smaller than a preset height threshold value and the area difference value of the first preset area and the second preset area is smaller than a preset area threshold value, and otherwise determining that the cloud of the part to be measured points has the attachment defect.
The preset wide threshold value, the preset high threshold value, and the preset area threshold value may be any values set according to actual situations, and the present embodiment is not limited thereto.
Specifically, in this embodiment, the obtained width, height and area of the white area in the first preset area are compared with the width, height and area of the white area in the second preset area, respectively, and if the difference between the width, height and area of the white area in the first preset area and the corresponding white area in the second preset area is smaller than the corresponding preset threshold, it is determined that the cloud to be tested of the component has no attachment defect. If the difference value of the width, the height and the area of the white area in the first preset area and the second preset area corresponding to the first preset area is larger than or equal to a preset threshold value corresponding to the first preset area, the cloud to be measured of the component is judged to have the attachment defect.
Fig. 3 is a flow chart of an implementation of a defect detection method according to a second embodiment of the present disclosure, in which the defect detection effect is better for a component having no high protrusions around a standard attachment position, but the defect detection effect is worse when there are high protrusions or little offset around the attachment position, so the present embodiment provides an effective solution to the above problem, and the detailed steps are as follows:
1. Reading a 3D template point cloud pt_model as a source point cloud, reading a 3D to-be-detected point cloud pt_dst as a target point cloud, and performing point cloud registration to obtain a 3D conversion matrix T;
2. Performing cross multiplication on the 3D template point cloud pt_model and the 3D conversion matrix T to obtain a corrected template point cloud pt_model ', wherein the corrected template point cloud pt_model' is shown as a formula [1 ];
pt_model'=pt_model×T[1]
3. Extracting attached component areas of a template point cloud pt_model' and dividing the attached component areas into independent component areas ROI_model [ N ] (namely target component template point cloud), wherein N is the total number of independent component areas contained in template information;
4. Respectively carrying out dimension reduction treatment on the template point cloud pt_model ' and the point cloud pt_dst to be measured, namely setting the z coordinate value to 0 to obtain dimension reduction template point cloud pt_model ' and dimension reduction point cloud pt_dst ';
5. Extracting attached component areas of a dimension reduction template point cloud pt_model 'and dividing the attached component areas into independent component areas ROI_model' N ] (namely dimension reduction template point cloud), wherein N is the total number of independent component areas contained in template information;
6. Based on each independent component ROI_model "[ N ], acquiring a corresponding component point cloud ROI_dst" [ N ] (namely a dimension reduction component point cloud) from a dimension reduction to-be-measured point cloud pt_dst' through an 8-neighborhood minimum method, and further mapping the component point cloud ROI_dst "[ N ] (namely the dimension reduction component point cloud) into the to-be-measured point cloud pt_dst, so that a component to-be-measured point cloud ROI_dst [ N ]. Corresponding to a target component template point cloud ROI_model [ N ];
7. And respectively converting the target component template point cloud ROI_model [ i ] and the component to-be-measured point cloud ROI_dst [ i ] into a depth map dep_model [ i ] (namely a target component template depth map) and a dep_dst [ i ] (namely a component to-be-measured depth map), wherein i is any independent component region.
8. Based on the depth in the target component template point cloud ROI_model [ i ], binarizing a target component template depth map dep_model [ i ] and a component to-be-detected depth map dep_dst [ i ] to obtain a component to-be-detected binary map bin_dep_model [ i ] and a target component template binary map bin_dep_dst [ i ]. The calculation method is as shown in the formulas [2], [3]:
wherein bin_dep_model [ i ] (x, y) is a pixel value corresponding to a template depth binary image, bin_dep_dst [ i ] (x, y) is a pixel value corresponding to a depth binary image to be detected, ROI_model [ i ] _z is a z-axis depth value of a target component template point cloud, dep_dst [ i ] (x, y) is a depth value of each pixel point in the component depth image to be detected, and mean represents an average value;
9. respectively carrying out Blob analysis on the bin_dep_model [ i ] binary image and the bin_dep_dst [ i ] binary image to respectively acquire information such as width, height, area and the like of a white area;
10. when the difference between the information of the width, the height, the area and the like of the template and the white area of the binary image to be detected is small, the area to be detected is considered to be normally attached, otherwise, the area to be detected is considered to be unattached, and the calculation method is as shown in the formula [4];
Wherein, the flag_pass is a normal flag bit, 0 indicates abnormality, 1 indicates normal, w_mdl, h_mdl and area_mdl are respectively the width, the height and the area of a white area of a template depth binary image, w_dst, h_dst and area_dst are respectively the width, the height and the area of the white area of the depth binary image to be detected, and T_w, T_h and T_area are respectively the width threshold, the height threshold and the area threshold of the white area of the binary image.
The method and the device not only effectively solve the problem of low detection accuracy when the position of the attached component is highly protruding or the attached component is slightly offset, but also overcome the defect that a neural network training method based on 3D point cloud wastes a large amount of manpower and material resources, and improve detection efficiency.
Example III
Fig. 4 is a schematic structural diagram of a defect detecting device according to an embodiment of the present disclosure, where the device specifically includes:
the point cloud acquisition module 410 is configured to acquire a component to-be-measured point cloud and acquire a target component template point cloud, where the target component template point cloud is a reference template point cloud for determining whether an attachment defect exists in the component to-be-measured point cloud;
The depth map reconstructing module 420 is configured to reconstruct the component to-be-measured point cloud and the target component template point cloud into a component to-be-measured depth map and a target component template depth map, respectively;
The defect analysis module 430 is configured to determine whether the cloud of the component to be tested has an attachment defect by performing binarization processing and Blob analysis on the depth map of the component to be tested and the depth map of the target component template.
In one embodiment, the defect analysis module 430 is specifically configured to:
respectively carrying out binarization processing on the part to-be-detected depth map and the target part template depth map based on the depth mean value corresponding to the part template point cloud to obtain a part to-be-detected binary map and a target part template binary map;
Respectively acquiring a first preset area in a binary image of the component to be detected and a second preset area in a binary image of the target component template;
Comparing the area information of the first preset area with the area information of the second preset area to determine whether the cloud of the part to be tested points has attachment defects, wherein the area information comprises at least one of width, height and area.
In one embodiment, the defect analysis module 430 is further specifically configured to:
When the width difference value of the first preset area and the second preset area is smaller than a preset width threshold value, the height difference value of the first preset area and the second preset area is smaller than a preset height threshold value, and the area difference value of the first preset area and the second preset area is smaller than a preset area threshold value, determining that the cloud of the part to be measured points has no attachment defect, otherwise, determining that the cloud of the part to be measured points has the attachment defect.
In an embodiment, the point cloud obtaining module 410 is specifically configured to:
Acquiring a point cloud to be detected comprising at least one to-be-detected attaching part and a template point cloud of a sample attaching part, wherein labels of the sample attaching parts are marked in the template point cloud;
calibrating the template point cloud based on a point cloud registration algorithm and a point cloud to be detected containing at least one attached component to be detected, so as to obtain a corrected template point cloud;
and determining at least one sample part template point cloud on the corrected template point cloud, and taking each sample part template point cloud as a target part template point cloud in sequence.
In an embodiment, the point cloud obtaining module 410 is further specifically configured to:
Determining source point cloud data of the sample attachment part by shooting the sample attachment part through an original 3D design drawing of the sample attachment part or through a 3D camera;
Modeling the source point cloud data of the sample attached part to obtain a template point cloud of the sample attached part.
In an embodiment, the point cloud obtaining module 410 is further specifically configured to:
Performing dimension reduction processing on the corrected template point cloud and the point cloud to be measured containing at least one attached component to be measured to obtain dimension reduction template point cloud and dimension reduction point cloud to be measured;
Determining a dimension reduction component template point cloud corresponding to the target component template point cloud in the dimension reduction template point cloud;
And determining a dimension reduction component point cloud corresponding to the dimension reduction component template point cloud in the dimension reduction component point cloud based on a neighborhood minimum value method, and determining a to-be-measured point cloud area corresponding to the dimension reduction component point cloud as a component to-be-measured point cloud on the to-be-measured point cloud containing at least one to-be-measured attached component through a mapping relation between the dimension reduction to-be-measured point cloud and the to-be-measured point cloud containing at least one to-be-measured attached component.
In an embodiment, the depth map reconstruction module 420 is specifically configured to:
Respectively acquiring coordinates of each voxel in a component to-be-measured point cloud and a target component template point cloud, wherein the coordinates comprise an X value, a Y value and a Z value;
Taking the X value and the Y value of each voxel in the component to-be-measured point cloud as coordinate values of a component to-be-measured depth map, and taking the Z value as a pixel value of the component to-be-measured depth map so as to reconstruct the component to-be-measured depth map;
And taking the X value and the Y value of each voxel in the target component template point cloud as coordinate values of the target component template depth map, and taking the Z value as pixel values of the target component template depth map so as to reconstruct the target component template depth map.
According to embodiments of the present disclosure, the present disclosure also provides an electronic device and a readable storage medium.
Fig. 5 illustrates a schematic block diagram of an example electronic device 500 that may be used to implement embodiments of the present disclosure. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular telephones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the disclosure described and/or claimed herein.
As shown in fig. 5, the electronic device 500 includes a computing unit 501 that can perform various appropriate actions and processes according to a computer program stored in a Read Only Memory (ROM) 502 or a computer program loaded from a storage unit 508 into a Random Access Memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the electronic device 500 may also be stored. The computing unit 501, ROM 502, and RAM 503 are connected to each other by a bus 504. An input/output (I/O) interface 505 is also connected to bus 504.
Various components in the electronic device 500 are connected to the I/O interface 505, including an input unit 506 such as a keyboard, a mouse, etc., an output unit 507 such as various types of displays, speakers, etc., a storage unit 508 such as a magnetic disk, optical disk, etc., and a communication unit 509 such as a network card, modem, wireless communication transceiver, etc. The communication unit 509 allows the electronic device 500 to exchange information/data with other devices via a computer network such as the internet and/or various telecommunication networks.
The computing unit 501 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of computing unit 501 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various specialized Artificial Intelligence (AI) computing chips, various computing units running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 501 performs the respective methods and processes described above, such as a defect detection method. For example, in some embodiments, the defect detection method may be implemented as a computer software program tangibly embodied on a machine-readable medium, such as storage unit 508. In some embodiments, part or all of the computer program may be loaded and/or installed onto the electronic device 500 via the ROM 502 and/or the communication unit 509. When the computer program is loaded into the RAM 503 and executed by the computing unit 501, one or more steps of the defect detection method described above may be performed. Alternatively, in other embodiments, the computing unit 501 may be configured to perform the defect detection method by any other suitable means (e.g. by means of firmware).
Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuitry, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), systems-on-a-chip (SOCs), complex Programmable Logic Devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include being implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be a special or general purpose programmable processor, operable to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.
Program code for carrying out methods of the present disclosure may be written in any combination of one or more programming languages. These program code may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus such that the program code, when executed by the processor or controller, causes the functions/operations specified in the flowchart and/or block diagram to be implemented. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.
In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user, for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback), and input from the user may be received in any form, including acoustic input, speech input, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a background component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such background, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a Local Area Network (LAN), a Wide Area Network (WAN), and the Internet.
The computer system may include a client and a server. The client and server are typically remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server may be a cloud server, a server of a distributed system, or a server incorporating a blockchain.
It should be appreciated that various forms of the flows shown above may be used to reorder, add, or delete steps. For example, the steps recited in the present disclosure may be performed in parallel or sequentially or in a different order, provided that the desired results of the technical solutions of the present disclosure are achieved, and are not limited herein.
Furthermore, the terms "first," "second," and the like, are used for descriptive purposes only and are not to be construed as indicating or implying a relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defining "a first" or "a second" may explicitly or implicitly include at least one such feature. In the description of the present disclosure, the meaning of "a plurality" is two or more, unless explicitly defined otherwise.
The foregoing is merely specific embodiments of the disclosure, but the protection scope of the disclosure is not limited thereto, and any person skilled in the art can easily think about changes or substitutions within the technical scope of the disclosure, and it is intended to cover the scope of the disclosure. Therefore, the protection scope of the present disclosure shall be subject to the protection scope of the claims.
Claims (7)
1. A method of defect detection, the method comprising:
Acquiring a part to-be-measured point cloud and a target part template point cloud, wherein the target part template point cloud is a reference template point cloud for judging whether the part to-be-measured point cloud has an attachment defect or not;
reconstructing the component to-be-measured point cloud and the target component template point cloud into a component to-be-measured depth map and a target component template depth map respectively;
The method comprises the steps of determining whether the cloud of the part to be measured has attachment defects or not through binarization processing and Blob analysis on the part to be measured depth map and the target part template depth map;
the obtaining the target component template point cloud comprises the following steps:
Acquiring a point cloud to be detected comprising at least one attaching part to be detected and a template point cloud of a sample attaching part, wherein labels of all the sample attaching parts are marked in the template point cloud;
Calibrating the template point cloud based on a point cloud registration algorithm and the point cloud to be detected comprising at least one attached component to be detected, so as to obtain a corrected template point cloud;
Determining at least one sample component template point cloud on the corrected template point cloud, and sequentially taking each sample component template point cloud as a target component template point cloud;
The method for obtaining the template point cloud of the sample attaching part comprises the following steps:
Determining source point cloud data of the sample attached component by shooting the sample attached component through an original 3D design drawing of the sample attached component or through a 3D camera;
Modeling the source point cloud data of the sample attached component to obtain a template point cloud of the sample attached component;
the method for acquiring the component to-be-measured point cloud comprises the following steps:
performing dimension reduction processing on the corrected template point cloud and the point cloud to be measured containing at least one attached component to be measured to obtain dimension reduction template point cloud and dimension reduction point cloud to be measured;
Determining a dimension reduction component template point cloud corresponding to the target component template point cloud in the dimension reduction template point cloud;
And determining a dimension reduction component point cloud corresponding to the dimension reduction component template point cloud in the dimension reduction component point cloud based on a neighborhood minimum method, and determining a to-be-measured point cloud area corresponding to the dimension reduction component point cloud as a component to-be-measured point cloud on the to-be-measured point cloud containing at least one to-be-measured attachment component through a mapping relation between the dimension reduction to-be-measured point cloud and the to-be-measured point cloud containing at least one to-be-measured attachment component.
2. The method according to claim 1, wherein the determining whether the component to-be-measured point cloud has an attachment defect by performing binarization processing and Blob analysis on the component to-be-measured depth map and the target component template depth map includes:
Respectively carrying out binarization processing on the part to-be-detected depth map and the target part template depth map based on the depth mean value corresponding to the part template point cloud to obtain a part to-be-detected binary map and a target part template binary map;
respectively acquiring a first preset area in the binary image to be tested of the component and a second preset area in the binary image of the target component template;
comparing the area information of the first preset area and the second preset area to determine whether the cloud of the part to be tested has an attachment defect, wherein the area information comprises at least one of width, height and area.
3. The method of claim 2, wherein comparing the area information of the first preset area and the second preset area to determine whether the component-to-be-measured-point cloud has an attachment defect comprises:
when the width difference value of the first preset area and the second preset area is smaller than a preset width threshold value, the height difference value of the first preset area and the second preset area is smaller than a preset height threshold value, and the area difference value of the first preset area and the second preset area is smaller than a preset area threshold value, determining that the part to-be-measured point cloud has no attachment defect, otherwise, determining that the part to-be-measured point cloud has the attachment defect.
4. The method of claim 1, wherein reconstructing the component-to-be-measured point cloud and the target component template point cloud into a component-to-be-measured depth map and a target component template depth map, respectively, comprises:
respectively acquiring coordinates of each voxel in the component to-be-measured point cloud and the target component template point cloud, wherein the coordinates comprise an X value, a Y value and a Z value;
taking the X value and the Y value of each voxel in the component to-be-measured point cloud as coordinate values of the component to-be-measured depth map, and taking the Z value as pixel values of the component to-be-measured depth map to reconstruct the component to-be-measured depth map;
And taking the X value and the Y value of each voxel in the target component template point cloud as coordinate values of the target component template depth map, and taking the Z value as pixel values of the target component template depth map so as to reconstruct the target component template depth map.
5. A defect detection apparatus, the apparatus comprising:
The device comprises a point cloud acquisition module, a target component template point cloud, a target component detection module and a target component detection module, wherein the point cloud acquisition module is used for acquiring a component to-be-detected point cloud and acquiring the target component template point cloud, wherein the target component template point cloud is a reference template point cloud for judging whether the component to-be-detected point cloud has an attachment defect or not;
the depth map reconstruction module is used for reconstructing the component to-be-measured point cloud and the target component template point cloud into a component to-be-measured depth map and a target component template depth map respectively;
The defect analysis module is used for determining whether the cloud of the part to be measured points has attachment defects or not through binarization processing and Blob analysis on the part to be measured depth map and the target part template depth map;
The point cloud acquisition module is further used for acquiring a point cloud to be detected comprising at least one to-be-detected attaching part and a template point cloud for acquiring a sample attaching part, wherein labels of the sample attaching parts are marked in the template point cloud; calibrating the template point cloud based on a point cloud registration algorithm and the point cloud to be detected comprising at least one attached component to be detected, so as to obtain a corrected template point cloud; the method comprises the steps of determining at least one sample component template point cloud on the corrected template point cloud, taking each sample component template point cloud as a target component template point cloud in sequence, shooting the sample attachment component through an original 3D design diagram of the sample attachment component or through a 3D camera, determining source point cloud data of the sample attachment component, modeling the source point cloud data of the sample attachment component to obtain the template point cloud of the sample attachment component, carrying out dimension reduction processing on the corrected template point cloud and the point cloud to be measured containing at least one attachment component to obtain dimension reduction template point cloud and dimension reduction point cloud to be measured, determining dimension reduction component template point cloud corresponding to the target component template point cloud in the dimension reduction template point cloud, determining dimension reduction component point cloud corresponding to the dimension reduction component template point cloud in the dimension reduction point cloud based on a neighborhood minimum method, and mapping a relationship between the dimension reduction point cloud and the point cloud to be measured containing at least one attachment component to be measured to the point cloud to be measured, wherein the dimension reduction point cloud is used as a region of the point cloud to be measured.
6. An electronic device, comprising:
At least one processor, and
A memory communicatively coupled to the at least one processor, wherein,
The memory stores instructions for execution by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-4.
7. A non-transitory computer readable storage medium storing computer instructions for causing a computer to perform the method of any one of claims 1-4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310172014.0A CN116152208B (en) | 2023-02-23 | 2023-02-23 | Defect detection method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310172014.0A CN116152208B (en) | 2023-02-23 | 2023-02-23 | Defect detection method, device, equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116152208A CN116152208A (en) | 2023-05-23 |
| CN116152208B true CN116152208B (en) | 2025-11-07 |
Family
ID=86338874
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310172014.0A Active CN116152208B (en) | 2023-02-23 | 2023-02-23 | Defect detection method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116152208B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116486126B (en) * | 2023-06-25 | 2023-10-27 | 合肥联宝信息技术有限公司 | Template determination method, device, equipment and storage medium |
| CN116862898B (en) * | 2023-07-27 | 2024-08-20 | 小米汽车科技有限公司 | Defect detection method and device for parts, storage medium and electronic equipment |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112730251A (en) * | 2020-12-30 | 2021-04-30 | 凌云光技术股份有限公司 | Device and method for detecting color defects of screen |
| CN115035031A (en) * | 2022-05-07 | 2022-09-09 | 浙江华睿科技股份有限公司 | Defect detection method and device for PIN (personal identification number) PIN, electronic equipment and storage medium |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112734760B (en) * | 2021-03-31 | 2021-08-06 | 高视科技(苏州)有限公司 | Semiconductor bump defect detection method, electronic device and storage medium |
| CN113344917B (en) * | 2021-07-28 | 2021-11-23 | 浙江华睿科技股份有限公司 | Detection method, detection device, electronic equipment and storage medium |
| CN114972235B (en) * | 2022-05-20 | 2024-09-13 | 成都飞机工业（集团）有限责任公司 | Wafer oxidation defect detection method and device, electronic equipment and storage medium |
| CN115661211A (en) * | 2022-11-11 | 2023-01-31 | 联宝(合肥)电子科技有限公司 | Object detection method, device, equipment and medium based on point cloud |
-
2023
- 2023-02-23 CN CN202310172014.0A patent/CN116152208B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112730251A (en) * | 2020-12-30 | 2021-04-30 | 凌云光技术股份有限公司 | Device and method for detecting color defects of screen |
| CN115035031A (en) * | 2022-05-07 | 2022-09-09 | 浙江华睿科技股份有限公司 | Defect detection method and device for PIN (personal identification number) PIN, electronic equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116152208A (en) | 2023-05-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2024002187A1 (en) | Defect detection method, defect detection device, and storage medium | |
| CN116152208B (en) | Defect detection method, device, equipment and storage medium | |
| CN116309660B (en) | A method, apparatus, device and storage medium for linear detection | |
| CN116205889B (en) | Offset detection method, device, electronic device and storage medium | |
| CN114445318A (en) | A defect detection method, device, electronic device and storage medium | |
| CN108876842A (en) | A kind of measurement method, system, equipment and the storage medium of sub-pixel edge angle | |
| CN120344992A (en) | Chip quality detection method and device, electronic device and storage medium | |
| CN114882059A (en) | Dimension measuring method, device and equipment based on image analysis and storage medium | |
| CN116124081B (en) | Non-contact workpiece detection method and device, electronic equipment and medium | |
| CN117974576A (en) | Image quality detection method, device, computer equipment and storage medium | |
| CN117169806A (en) | Broken code screen electric meter display defect detection method, device, equipment and medium | |
| CN115661211A (en) | Object detection method, device, equipment and medium based on point cloud | |
| CN119599941B (en) | Deviation defect detection method and device, electronic equipment and storage medium | |
| CN115422617A (en) | Frame image size measuring method, device and medium based on CAD | |
| CN114821274A (en) | A method and device for identifying the state of an opening and closing sign | |
| CN120356194A (en) | Automatic reading method, device, equipment and storage medium for pointer instrument | |
| CN116486126B (en) | Template determination method, device, equipment and storage medium | |
| CN118154556A (en) | Defect detection method, device, electronic equipment and storage medium | |
| CN117036282A (en) | Defect detection methods, devices, equipment and storage media | |
| CN115035481A (en) | Image object distance fusion method, device, equipment and storage medium | |
| CN116433661A (en) | Semiconductor wafer inspection method, device, equipment and medium for multi-task learning | |
| CN115393430A (en) | Displacement measurement method, system, storage medium and electronic equipment | |
| CN116778232B (en) | SAR image vertical stripe detection method, device, equipment and storage medium | |
| CN119251483B (en) | Object position detection method and device, electronic equipment and storage medium | |
| CN116258714B (en) | Defect identification method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20241226 Address after: 230601 No. 3188-1, Yungu Road, Hefei Economic and Technological Development Zone, Anhui Province (in the comprehensive bonded zone) Applicant after: LCFC (HEFEI) ELECTRONICS TECHNOLOGY Co.,Ltd. Country or region after: China Address before: Room 1605, Lenovo Technology Port, No. 5899, Xiyou Road, Hefei Economic and Technological Development Zone, Anhui Province, 230000 Applicant before: HEFEI LCFC INFORMATION TECHNOLOGY Co.,Ltd. Country or region before: China |
|
| TA01 | Transfer of patent application right | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |