CN121258912A - Computer vision-based glass flaw identification method, system, equipment and medium - Google Patents

Computer vision-based glass flaw identification method, system, equipment and medium

Info

Publication number
CN121258912A
CN121258912A CN202511348353.5A CN202511348353A CN121258912A CN 121258912 A CN121258912 A CN 121258912A CN 202511348353 A CN202511348353 A CN 202511348353A CN 121258912 A CN121258912 A CN 121258912A
Authority
CN
China
Prior art keywords
defect
feature
region
real
vector
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202511348353.5A
Other languages
Chinese (zh)
Inventor
吴小平
夏大文
白维维
陈守维
唐厚炳
罗帝红
吴育波
向位
库进锋
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Kaili University
Original Assignee
Kaili University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Kaili University filed Critical Kaili University
Priority to CN202511348353.5A priority Critical patent/CN121258912A/en
Publication of CN121258912A publication Critical patent/CN121258912A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0004—Industrial image inspection
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
    • G06V10/443—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components by matching or filtering
    • G06V10/449—Biologically inspired filters, e.g. difference of Gaussians [DoG] or Gabor filters
    • G06V10/451—Biologically inspired filters, e.g. difference of Gaussians [DoG] or Gabor filters with interaction between the filter responses, e.g. cortical complex cells
    • G06V10/454—Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G06V10/765—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects using rules for classification or partitioning the feature space
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/772—Determining representative reference patterns, e.g. averaging or distorting patterns; Generating dictionaries
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
    • G06V10/806—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level of extracted features
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30108—Industrial image inspection
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
    • Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
    • Y02P90/30—Computing systems specially adapted for manufacturing

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Databases & Information Systems (AREA)
  • Software Systems (AREA)
  • Medical Informatics (AREA)
  • Computing Systems (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biodiversity & Conservation Biology (AREA)
  • Biomedical Technology (AREA)
  • Molecular Biology (AREA)
  • Quality & Reliability (AREA)
  • Image Analysis (AREA)

Abstract

The application relates to a glass defect identification method, a system, equipment and a medium based on computer vision. The method comprises the steps of firstly obtaining original image data of a glass product, carrying out feature extraction by adopting a deep learning model to obtain a defect candidate region set, extracting pixel-level boundary information from the defect candidate region set by utilizing an example segmentation algorithm, fusing multi-scale morphological features, dividing feature clusters, matching with a preset defect feature library to obtain a defect labeling result comprising defect types and defect region feature vectors, constructing a reference threshold model based on the defect types, carrying out dynamic calibration on the model by combining the defect region feature vectors and production condition parameters to generate a real-time detection threshold range, and carrying out secondary scanning on the original image data by utilizing the real-time detection threshold range to generate a defect distribution map. The method enhances the adaptability to environmental changes by improving the accuracy of defect identification, and can efficiently and stably complete the task of identifying glass defects.

Description

Computer vision-based glass flaw identification method, system, equipment and medium
Technical Field
The invention belongs to the technical field of computer vision detection, and particularly relates to a glass defect identification method, system, equipment and medium based on computer vision.
Background
In the glass manufacturing industry, quality control of glass articles is critical. The glass is easy to generate various flaws such as bubbles, cracks, scratches, stains and the like in the production process, and the flaws not only damage the appearance of the glass product, but also are more likely to influence the service performance and the safety performance of the glass product. Currently, glass flaw detection methods are numerous. Part of traditional schemes rely on manual visual inspection, however, manual detection efficiency is low, and the influence of subjective factors is large, and is difficult to accurately identify small flaws, and the missing detection rate is high, so that the requirements of modern large-scale and high-precision production cannot be met. With the development of technology, detection means based on computer vision are gradually rising. If the patent CN105243656A is constructed to detect the pipeline and the characteristic extraction equipment structure, the defect detection is carried out by means of an infrared detector and a high-definition camera and by matching with an edge detection algorithm, but the mode has strong dependence on the infrared refraction characteristic of glass and is limited in application scene. For another example, in CN103344651a, moire images are obtained by using a grating and a high-speed line camera to determine defects, but the requirements on the optical line environment are severe, and the defects are difficult to stably run in a production environment with changeable external light. In summary, the existing glass defect identification method based on computer vision has the defects in the aspects of detection precision, environmental adaptability, detection efficiency and the like.
Disclosure of Invention
Accordingly, it is desirable to provide a method, a system, a device and a medium for identifying glass defects based on computer vision, which can improve the accuracy of defect identification, enhance the adaptability to environmental changes, and efficiently and stably perform the task of identifying glass defects.
In a first aspect, the present application provides a method for identifying glass flaws based on computer vision, comprising:
and obtaining the original image data of the glass product, and extracting the characteristics by adopting a deep learning model to obtain a defect candidate region set.
And extracting pixel-level boundary information from the defect candidate region set by using an example segmentation algorithm, merging multi-scale morphological features, dividing feature clusters, and matching with a preset defect feature library to obtain a defect labeling result, wherein the defect labeling result comprises defect types and defect region feature vectors.
And constructing a reference threshold model based on the defect category, and constructing the reference threshold model by combining the defect region feature vector and the production condition parameters to generate a real-time detection threshold range.
And carrying out secondary scanning on the original image data by utilizing the real-time detection threshold range, calculating the region defect probability and weight distribution, and generating a defect distribution map.
In one embodiment, obtaining raw image data of a glass product, performing feature extraction by using a deep learning model to obtain a defect candidate region set, including:
and acquiring the original image data of the glass product, wherein the original image data is shot by a high-resolution industrial camera.
And performing field alignment and preprocessing on the original image data, and performing feature extraction by using a depth separable convolution network to obtain a feature map set.
And performing region segmentation based on the feature graph set, and generating a preliminary defect candidate region through an adaptive threshold value, wherein the preliminary defect candidate region comprises a boundary box, a mask and a classification confidence.
And extracting texture features and morphological features from the preliminary defect candidate regions, performing cluster analysis, and screening out high-confidence defect regions.
And optimizing the boundary of the high-confidence defect region by adopting an edge detection operator, generating a high-precision defect positioning mask, performing consistency check with the original region characteristics, and outputting a defect candidate region set.
In one embodiment, extracting pixel-level boundary information from a defect candidate region set by using an example segmentation algorithm, merging multi-scale morphological features, dividing feature clusters, and matching with a preset defect feature library to obtain a defect labeling result, wherein the method comprises the following steps:
And processing pixel-level boundary information of the defect candidate region set by adopting an improved example segmentation algorithm to obtain accurate boundary segmentation data.
And extracting multi-scale morphological characteristics based on the boundary segmentation data, and carrying out weighted fusion on the characteristics of different scales to obtain a comprehensive characteristic vector.
And carrying out cluster analysis on the comprehensive feature vectors by using a self-adaptive density clustering algorithm, and determining a feature cluster dividing result.
And carrying out semantic matching on the feature cluster dividing result and a preset defect feature library to obtain preliminary defect marking information, wherein the preset defect feature library comprises standard form vectors, typical texture patterns, outline templates and category labels of various defects.
And extracting the multidimensional feature vector from the preliminary defect labeling information, and calculating the matching degree score of the multidimensional feature vector and the standard template.
And if the matching degree score reaches a preset threshold value, confirming that the defect marking information is a defect marking result.
In one embodiment, the matching score of the multi-dimensional feature vector and the standard template is calculated by the following formula:
wherein S represents a matching degree score, Represents the ith dimension characteristic vector of the defect to be detected,Representing the ith dimension feature vector of the standard template, D cos representing cosine distance, omega i representing the feature weight coefficient of the ith dimension, dynamically distributing the feature weight coefficient by the distinguishing degree of various features in a preset defect feature library,The distinction is determined by the reciprocal variance of various defects counted in the training stage on the corresponding feature dimension, n represents the total number of feature dimensions, P represents the outline point set of the defect to be detected, T represents the outline point set of the standard template, D Hausdorff represents the bidirectional Hastedor distance, per represents the outline perimeter, and alpha E [0.6,0.8] represents the weight of the balance feature and structure.
In one embodiment, constructing a reference threshold model based on defect categories, constructing a reference threshold model in combination with defect region feature vectors and production condition parameters, and generating a real-time detection threshold range includes:
and extracting standard morphological vectors of corresponding categories from a preset defect feature library based on the defect categories.
And carrying out statistical analysis on the standard form vector to calculate a mean vector and a covariance matrix, and constructing a parameterized reference threshold model.
And carrying out space-time dimension enhancement processing on the defect region feature vector, and generating a space-time feature tensor by combining production condition parameters, wherein the production condition parameters comprise a production line running state parameter, an environment illumination parameter, a glass self attribute parameter, an imaging equipment parameter and a mechanical vibration parameter.
The reference threshold model parameters and the space-time characteristic tensors are input into a pre-trained attention mechanism network to generate an adaptive weight matrix.
And carrying out space-time dimension optimization on the reference threshold model parameters based on the self-adaptive weight matrix, and constructing a threshold response surface of space-time perception.
And generating a real-time detection threshold range which dynamically changes along with the production environment through the feature mapping of the threshold response curved surface and the current detection area.
In one embodiment, the spatio-temporal feature tensor is calculated by the following formula:
Wherein T st represents a space-time feature tensor, F d represents a defect region feature vector, G i represents a production condition parameter vector, the vector of five production parameters represents G 1 represents a production line running state parameter, G 2 represents an ambient illumination parameter, G 3 represents a glass self attribute parameter, G 4 represents an imaging device parameter, G 5 represents a mechanical vibration parameter, θ i represents a parameter attention weight, norm (. Cndot.) represents a normalization function, and ST_enhancement (. Cndot.) represents a space-time enhancement operator.
In one embodiment, the method for generating the defect distribution map includes performing secondary scanning on original image data by using a real-time detection threshold range, calculating region defect probability and weight distribution, and generating the defect distribution map, including:
dividing the original image data into multi-scale detection windows based on the real-time detection threshold range and extracting local feature vectors of each window.
And comparing the local feature vector of each detection window with a real-time detection threshold range, and calculating the probability value of the defect in the window.
The probability value for the presence of a defect within the window is calculated using the following formula:
wherein Y represents a probability value of the presence of a defect within the detection window, A local feature vector representing the current window,Standard feature vectors representing corresponding categories in a preset defect feature library, J real represents a real-time detection threshold range,The degree of similarity of the features is indicated,Represents the distance of the feature from the threshold range, μ represents the sensitivity coefficient, and ε represents the minimum value.
And combining the spatial position relation and the probability value of each detection window, and calculating the regional defect weight distribution through a Gaussian kernel function.
And performing space mapping on the probability value and the defect weight distribution to generate an initial defect distribution map.
And carrying out morphological filtering and connected domain analysis on the initial defect distribution map to obtain a refined defect distribution map.
In a second aspect, the present application also provides a computer vision-based glass flaw identification system, the system comprising:
and the candidate region detection module is used for acquiring the original image data of the glass product, and extracting the characteristics by adopting a deep learning model to obtain a defect candidate region set.
The defect feature labeling module is used for extracting pixel-level boundary information from the defect candidate region set by utilizing an example segmentation algorithm, classifying feature clusters after fusing multi-scale morphological features, and matching the feature clusters with a preset defect feature library to obtain a defect labeling result, wherein the defect labeling result comprises defect types and defect region feature vectors.
The dynamic threshold generation module is used for constructing a reference threshold model based on the defect category, constructing the reference threshold model by combining the defect region feature vector and the production condition parameter, and generating a real-time detection threshold range.
And the defect map construction module is used for carrying out secondary scanning on the original image data by utilizing the real-time detection threshold range, calculating the region defect probability and the weight distribution, and generating a defect distribution map.
In a third aspect, the application also provides a computer device comprising a memory storing a computer program and a processor implementing the method as before when executing the computer program.
In a fourth aspect, the application also provides a computer readable storage medium having stored thereon a computer program which when executed by a processor implements a method as before.
The glass flaw identification method, the system, the computer equipment and the storage medium based on computer vision comprise the steps of firstly obtaining original image data of a glass product, carrying out feature extraction by adopting a deep learning model to obtain a defect candidate region set, extracting pixel-level boundary information from the defect candidate region set by utilizing an example segmentation algorithm, dividing feature clusters after fusing multiscale morphological features, matching with a preset defect feature library to obtain a defect labeling result comprising defect types and defect region feature vectors, constructing a reference threshold model based on the defect types, carrying out dynamic calibration on the model by combining the defect region feature vectors and production condition parameters to generate a real-time detection threshold range, carrying out secondary scanning on the original image data by utilizing the real-time detection threshold range, calculating region defect probability and weight distribution, and generating a defect distribution map. The method realizes accurate marking by deep learning extraction of defect candidate areas, instance segmentation and feature matching, dynamically generates detection threshold values by combining production condition parameters and generates defect distribution maps by secondary scanning, multi-scale feature fusion and pixel-level boundary extraction improve defect identification accuracy on detection accuracy, enhances adaptability to environmental changes such as illumination and vibration on the basis of dynamic threshold adjustment of production condition parameters on the basis of environmental adaptability, and combines detection speed and result fineness on the basis of multi-stage progressive processing and secondary scanning mechanism on detection efficiency, so that the glass defect identification task can be completed in an integral manner efficiently and stably.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the related art, the drawings that are required to be used in the embodiments or the related technical descriptions will be briefly described, and it is apparent that the drawings in the following description are only some embodiments of the present application, and other drawings may be obtained according to the drawings without inventive effort for those skilled in the art.
FIG. 1 is a flowchart of a method for identifying glass defects based on computer vision according to an embodiment of the present invention;
Fig. 2 is a block diagram of a glass defect recognition system based on computer vision according to an embodiment of the present invention.
Detailed Description
The present application will be described in further detail with reference to the drawings and examples, in order to make the objects, technical solutions and advantages of the present application more apparent. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the application.
In one embodiment, as shown in fig. 1, the present application provides a method for identifying glass flaws based on computer vision, which may include the following steps:
Step S101, obtaining original image data of a glass product, and extracting features by adopting a deep learning model to obtain a defect candidate region set.
Specifically, the original image data of the glass product is obtained, and the deep learning model is adopted to perform feature extraction to obtain a defect candidate region set. The method comprises the steps of shooting surface images of glass products from different angles and under illumination conditions through a high-resolution industrial camera to form original image data, inputting the original image data into a pre-trained deep learning model (such as an improved Faster R-CNN or YOLO model), extracting deep semantic features and shallow texture features of the images layer by layer through structures such as a convolution layer and a pooling layer, generating candidate areas possibly containing defects through a regional suggestion network, and obtaining a defect candidate area set consisting of boundary frame coordinates and confidence scores after non-maximum suppression processing.
Step S102, extracting pixel-level boundary information from the defect candidate region set by using an example segmentation algorithm, merging multi-scale morphological features, dividing feature clusters, and matching with a preset defect feature library to obtain a defect labeling result, wherein the defect labeling result comprises defect types and defect region feature vectors.
Firstly, extracting pixel-level boundary information from a defect candidate region set by using an example segmentation algorithm, fusing multiscale morphological characteristics, dividing characteristic clusters, and matching with a preset defect characteristic library to obtain a defect labeling result, wherein the defect labeling result comprises defect types and defect region characteristic vectors. Firstly, carrying out pixel level segmentation on each defect candidate region by adopting an example segmentation algorithm such as Mask R-CNN and the like to determine the accurate boundary of the defect in an image to obtain a corresponding Mask matrix, then extracting morphological characteristics such as area, perimeter, external rectangular length-width ratio, euler number and the like under different scales from the Mask matrix, integrating the multi-scale characteristics into a unified comprehensive characteristic vector through a characteristic fusion network, carrying out cluster analysis on the comprehensive characteristic vector by using a clustering algorithm such as K-means and the like to divide the characteristic cluster with similar characteristics, finally, comparing and matching each characteristic cluster with standard characteristics of various defects in a preset defect characteristic library, determining defect types according to the matching degree, extracting the characteristic vector corresponding to the defect region, and forming a defect marking result.
And step S103, constructing a reference threshold model based on the defect category, and constructing the reference threshold model by combining the defect region feature vector and the production condition parameters to generate a real-time detection threshold range.
Specifically, a reference threshold model is built based on the defect category, and the reference threshold model is built by combining the defect region feature vector and the production condition parameters, so that a real-time detection threshold range is generated. The method comprises the steps of firstly, extracting historical feature data of corresponding defect types from a preset defect feature library, calculating parameters such as mean value vectors and covariance matrixes through statistical analysis, constructing a parameterized reference threshold model, then carrying out enhancement treatment on time dimensions (such as different production moments) and space dimensions (such as different image areas) on the defect region feature vectors, collecting production condition parameters (including production line speed, ambient temperature, illumination intensity, glass thickness, camera exposure time, mechanical vibration frequency and the like), inputting the parameters of the reference threshold model, the enhanced defect region feature vectors and the production condition parameters into a dynamic calibration network, and finally carrying out adjustment and optimization on the reference threshold model through a multi-layer perceptron or attention mechanism, so as to finally generate a real-time detection threshold range which can adapt to the current production environment.
And step S104, performing secondary scanning on the original image data by utilizing the real-time detection threshold range, calculating the region defect probability and weight distribution, and generating a defect distribution map.
Specifically, the original image data is scanned twice by utilizing the real-time detection threshold range, the region defect probability and the weight distribution are calculated, and a defect distribution map is generated. Firstly, carrying out multi-scale sliding window scanning on original image data according to preset window sizes and step sizes, extracting local feature vectors in each window, comparing the local feature vectors with a real-time detection threshold range, calculating probability values of defects of each window through a preset probability calculation function (such as a Sigmoid function based on feature distances), calculating influence weights among adjacent windows by using a Gaussian kernel function in combination with spatial position relations of the windows to obtain distribution weights of regional defects, then mapping the probability values and the distribution weights to corresponding pixel positions of an image to form an initial defect distribution map, finally carrying out morphological open operation, closed operation and other filtering processing on the initial map, removing noise interference, and analyzing and merging adjacent defect regions through a connected region to obtain a final refined defect distribution map.
The glass flaw identification method based on computer vision comprises the steps of firstly obtaining original image data of a glass product, carrying out feature extraction by adopting a deep learning model to obtain a defect candidate region set, extracting pixel-level boundary information from the defect candidate region set by utilizing an example segmentation algorithm, dividing feature clusters after fusing multiscale morphological features, matching with a preset defect feature library to obtain a defect labeling result comprising defect types and defect region feature vectors, constructing a reference threshold model based on the defect types, carrying out dynamic calibration on the model by combining the defect region feature vectors and production condition parameters to generate a real-time detection threshold range, carrying out secondary scanning on the original image data by utilizing the real-time detection threshold range, calculating region defect probability and weight distribution, and generating a defect distribution map. The method realizes accurate marking by deep learning extraction of defect candidate areas, instance segmentation and feature matching, dynamically generates detection threshold values by combining production condition parameters and generates defect distribution maps by secondary scanning, multi-scale feature fusion and pixel-level boundary extraction improve defect identification accuracy on detection accuracy, enhances adaptability to environmental changes such as illumination and vibration on the basis of dynamic threshold adjustment of production condition parameters on the basis of environmental adaptability, and combines detection speed and result fineness on the basis of multi-stage progressive processing and secondary scanning mechanism on detection efficiency, so that the glass defect identification task can be completed in an integral manner efficiently and stably.
In one embodiment, obtaining raw image data of a glass product, and performing feature extraction by using a deep learning model to obtain a defect candidate region set may include the following steps:
Step S201, obtaining original image data of the glass product, wherein the original image data is shot by a high-resolution industrial camera.
Step S202, performing field alignment and preprocessing on original image data, and performing feature extraction by using a depth separable convolution network to obtain a feature map set.
Preferably, the visual field alignment is realized by carrying out position calibration on the images of the glass products photographed at different angles and different moments, so that the corresponding areas of the glass products in the images are positioned under the same coordinate system, the position deviation caused by the difference of photographing visual angles is eliminated, and the accuracy of subsequent feature extraction and area analysis is ensured. Preprocessing the raw image data typically includes denoising (removing random noise in the image), contrast adjustment (enhancing the difference between defects and background in the image), size normalization (adjusting the image to a uniform size), etc., in order to improve image quality and provide a more reliable input for subsequent feature extraction. The depth separable convolution network decomposes the traditional convolution operation into the depth convolution (independently convolving each input channel) and the point-by-point convolution (combining the output channels of the depth convolution), thereby greatly reducing network parameters and calculated amount while guaranteeing the feature extraction capability, and being suitable for real-time processing in industrial scenes.
Step S203, region segmentation is performed based on the feature image set, and a preliminary defect candidate region is generated through an adaptive threshold, wherein the preliminary defect candidate region comprises a boundary box, a mask and a classification confidence.
Further, region segmentation is a process of dividing an image into several regions with similar features based on a set of feature maps, with the aim of separating regions that may contain defects from the image.
The self-adaptive threshold value is dynamically adjusted according to the characteristics of the local area of the image, and different from the fixed threshold value, the self-adaptive threshold value can better adapt to the brightness and contrast difference of different areas in the image, and is used for distinguishing a foreground (possible defect area) from a background (normal glass area) in area segmentation.
The preliminary defect candidate region is a region which may contain a defect and is obtained through region segmentation and adaptive thresholding, and consists of a bounding box (determining the position and the range of the region in the image), a mask (marking pixels belonging to the defect in the region), and a classification confidence (indicating the likelihood that the region is defective).
And S204, extracting texture features and morphological features from the preliminary defect candidate region, performing cluster analysis, and screening out a high-confidence defect region.
Preferably, the texture features and morphological features of the preliminary defect candidate regions are used as inputs, and the regions with similar features are grouped into a class by an algorithm (e.g., K-means) in order to distinguish between true defect regions and false defect regions (e.g., false detection regions caused by noise, reflection, etc.).
The high-confidence defect area is a preliminary defect candidate area with the classification confidence higher than a set threshold after cluster analysis, has high possibility of being a defect and is an object of subsequent fine processing.
Step S205, optimizing the boundary of the high-confidence defect area by adopting an edge detection operator, generating a high-precision defect positioning mask, performing consistency check with the original area characteristics, and outputting a defect candidate area set.
The edge detection operator is an algorithm (such as a Canny operator) for detecting pixel points (namely edges) with the gray values changed drastically in the image, and is used for optimizing the boundary of a high-confidence defect area in the glass defect detection process so that the boundary is more attached to the actual outline of the defect.
The high-precision defect positioning mask is a mask obtained by optimizing the boundary through an edge detection operator, can more accurately mark pixels in a defect area, and has higher boundary positioning precision compared with the mask of a preliminary defect candidate area.
And (3) comparing the high-precision defect positioning mask with original region features (such as texture features and morphological features of the primary defect candidate region) through consistency verification, verifying the consistency of the high-precision defect positioning mask and the original region features, ensuring that the optimized defect region still accords with the feature attribute of the defect, and eliminating the error region caused by boundary optimization.
The method comprises the steps of obtaining original image data of a glass product, shooting the original image data by a high-resolution industrial camera, performing visual field alignment and preprocessing (such as denoising, contrast adjustment and the like) on the original image data, inputting the processed image into a depth separable convolution network for feature extraction to obtain feature map sets containing semantic information of different levels, dividing the feature map sets into areas, generating preliminary defect candidate areas by an adaptive threshold algorithm, wherein the areas comprise boundary boxes, masks and classification confidence levels, extracting texture features (such as gray level co-occurrence matrix features) and morphological features (such as areas and circumferences) from the preliminary defect candidate areas, performing clustering analysis on the features, screening out high-confidence defect areas with the classification confidence level higher than a set threshold, optimizing boundaries of the high-confidence defect areas by adopting edge detection operators such as Canny, generating a high-precision defect positioning mask, performing consistency check (such as contrast boundary coincidence degree and feature similarity) on the mask and the original area features, and outputting the defect candidate area sets after the check is passed.
The method and the device ensure complete reservation of defect information through high-resolution image acquisition, and the feature map extracted through the preprocessing and depth separable convolution network can give consideration to details and efficiency, a preliminary candidate region generated by the self-adaptive threshold lays a foundation for subsequent screening, false defects can be effectively removed through clustering analysis of texture and morphological features, edge optimization and consistency verification further improve region positioning accuracy, and finally the output defect candidate region set has the characteristics of accurate boundary and high confidence.
In one embodiment, extracting pixel-level boundary information from a defect candidate region set by using an example segmentation algorithm, fusing multi-scale morphological features, dividing feature clusters, and matching with a preset defect feature library to obtain a defect labeling result, which may include the following steps:
Step S301, processing pixel-level boundary information of the defect candidate region set by adopting an improved example segmentation algorithm to obtain accurate boundary segmentation data.
Preferably, the improved example segmentation algorithm is an algorithm which is optimized on the basis of a traditional example segmentation algorithm (such as Mask R-CNN), and the segmentation accuracy of the targets with blurred edges and fine features such as glass flaws is improved by introducing attention mechanisms or improving loss functions, so that pixel-level boundary information of a defect candidate region set can be processed more accurately, and more practical boundary segmentation data can be output.
The pixel-level boundary information refers to information whether each pixel in the defect candidate area belongs to a defect boundary, and is a detailed description of a defect contour.
Step S302, multi-scale morphological features are extracted based on the boundary segmentation data, and the different-scale features are subjected to weighted fusion to obtain a comprehensive feature vector.
Further, the multi-scale morphological characteristics describe the defects from different scales (such as local detail scale and overall outline scale), and the morphological characteristics comprise edge curvature, hole number, area, perimeter, circumscribed rectangle proportion and the like under a small scale and can comprehensively reflect morphological properties of the defects.
And step S303, performing cluster analysis on the comprehensive feature vectors by using a self-adaptive density clustering algorithm, and determining a feature cluster division result.
Schematically, the self-adaptive density clustering algorithm is an algorithm capable of automatically adjusting clustering parameters according to data distribution density, and can automatically identify areas with higher density in a feature space as clustering clusters without pre-specifying the clustering quantity, thereby being applicable to scenes with complex feature distribution such as glass defects and being capable of clustering comprehensive feature vectors more reasonably.
The feature cluster division result is to divide the comprehensive feature vector into a plurality of feature-similar groups (feature clusters), and each feature cluster represents a class of defects with similar features.
And step S304, carrying out semantic matching on the feature cluster division result and a preset defect feature library to obtain preliminary defect marking information, wherein the preset defect feature library comprises standard form vectors, typical texture patterns, outline templates and category labels of various defects.
Preferably, the preset defect feature library is a pre-constructed database containing various known glass defect feature information, and stores standard morphological vectors, typical texture maps, contour templates, class labels and the like, which are reference standards for defect identification and labeling.
The semantic matching is carried out on the feature cluster division result and various defect features in a preset defect feature library to carry out meaning level comparison matching, so that not only is the numerical difference of the features compared, but also the semantic information (such as the feature meaning of 'cracks' and 'bubbles') of the defects is combined, the matching accuracy is improved, and the preliminary defect marking information is obtained.
Step S305, extracting multi-dimensional feature vectors from the preliminary defect labeling information, and calculating the matching degree scores of the multi-dimensional feature vectors and the standard templates.
The standard template is a standard characteristic template for presetting various defects in a defect characteristic library, comprises typical characteristics of the defects in each dimension, and is a reference for judging the accuracy of the primary defect marking information.
Step S306, if the matching degree score reaches a preset threshold value, confirming that the defect labeling information is a defect labeling result.
The matching degree score is a quantized value obtained by calculating the similarity degree between the multidimensional feature vector and the standard template, and the higher the score is, the higher the feature matching degree of the multidimensional feature vector and the standard template is, so that the matching degree is used for measuring the coincidence degree of the preliminary defect marking information and the standard defect feature, and the quantitative value is an important index for judging whether the preliminary marking information is accurate or not.
The preset threshold is a preset critical value for judging whether the matching degree score meets the standard or not, and is a standard for confirming whether the preliminary defect marking information can become a defect marking result or not.
The method comprises the steps of processing pixel-level boundary information of a defect candidate area set by adopting an improved example segmentation algorithm to obtain accurate boundary segmentation data, extracting multi-scale morphological characteristics based on the boundary segmentation data, integrating different scale characteristics into comprehensive characteristic vectors by weighting fusion, carrying out cluster analysis on the comprehensive characteristic vectors by utilizing a self-adaptive density clustering algorithm to determine characteristic cluster division results, carrying out semantic matching on the characteristic cluster division results and a preset defect characteristic library (comprising standard morphological vectors, typical texture patterns, outline templates and category labels of various defects) to obtain preliminary defect labeling information, extracting multi-dimensional characteristic vectors from the preliminary defect labeling information, calculating matching degree scores of the vectors and the standard templates, and confirming that the information is the defect labeling result if the scores reach a preset threshold.
The improved example segmentation algorithm in the embodiment provides high-precision basic data for boundary segmentation, the multi-scale morphological feature fusion enriches feature dimensions, the self-adaptive density clustering enhances the rationality of feature cluster division, the accuracy of a defect labeling result is ensured by semantic matching and matching degree score verification, the defect labeling result finally output can accurately reflect defect types and feature vectors, and a reliable basis is provided for the construction of a follow-up reference threshold model.
In one embodiment, the matching score of the multi-dimensional feature vector and the standard template can be calculated by the following formula:
wherein S represents a matching degree score, Represents the ith dimension characteristic vector of the defect to be detected,Representing the ith dimension feature vector of the standard template, D cos representing cosine distance, omega i representing the feature weight coefficient of the ith dimension, dynamically distributing the feature weight coefficient by the distinguishing degree of various features in a preset defect feature library,The distinction is determined by the reciprocal variance of various defects counted in the training stage on the corresponding feature dimension, n represents the total number of feature dimensions, P represents the outline point set of the defect to be detected, T represents the outline point set of the standard template, D Hausdorff represents the bidirectional Hastedor distance, per represents the outline perimeter, and alpha E [0.6,0.8] represents the weight of the balance feature and structure.
According to the embodiment, the cosine distance, the feature weight coefficient, the bidirectional Haoskov distance, the contour circumference and the balance weight of each dimension feature vector of the defect to be detected and the standard template are combined, the matching condition of the feature and the structural layer is comprehensively considered, the feature weight coefficient is dynamically distributed according to the feature discrimination degree, the matching degree of the feature and the structural layer can be accurately quantized, and the accuracy and the reliability of the defect marking result are improved.
In one embodiment, constructing the reference threshold model based on the defect type, constructing the reference threshold model by combining the defect region feature vector and the production condition parameter, and generating the real-time detection threshold range may include the following steps:
Step S401, extracting standard form vectors of corresponding categories from a preset defect feature library based on the defect categories.
Preferably, the standard morphological vector is extracted from a preset defect feature library, and the vector representation of the standard morphological feature corresponding to the specific defect category contains the information of typical geometric form, contour feature and the like of the defect.
And step S402, carrying out statistical analysis on the standard form vector to calculate a mean vector and a covariance matrix, and constructing a parameterized reference threshold model.
The parameterized reference threshold model is a threshold judgment model with clear parameter characterization constructed through statistical analysis based on a standard morphological vector corresponding to the defect category. The core parameters of the method comprise a mean vector and a covariance matrix, wherein the mean vector and the covariance matrix are calculated from standard morphological vectors, the mean vector reflects the average level of morphological characteristics of the defects and serves as a central reference for threshold judgment, and the covariance matrix describes the relativity and the discrete degree among dimensions of the characteristics and is used for defining the acceptable fluctuation range of the characteristics.
Step S403, performing space-time dimension enhancement processing on the feature vector of the defect area, and generating a space-time feature tensor by combining production condition parameters, wherein the production condition parameters comprise production line running state parameters, environment illumination parameters, glass self attribute parameters, imaging equipment parameters and mechanical vibration parameters.
The space-time dimension enhancement processing is performed on the feature vector of the defect area, and the processing of the information of the time dimension (such as feature change at different moments) and the space dimension (such as feature association of different areas) is integrated, so that the feature vector can reflect the space-time characteristics of the defect more comprehensively.
Step S404, inputting the reference threshold model parameters and the space-time feature tensors into a pre-trained attention mechanism network to generate an adaptive weight matrix.
The space-time feature tensor is a tensor formed by fusing the defect region feature vector subjected to space-time dimension enhancement processing with production condition parameters, and integrates the defect features and space-time information of multiple production parameters.
The self-adaptive weight matrix is used for dynamically adjusting the importance matrix of each parameter of the reference threshold model, and the weight reflects the influence degree of the corresponding parameter on the threshold under the current production environment.
And step S405, performing space-time dimension optimization on the reference threshold model parameters based on the self-adaptive weight matrix, and constructing a threshold response surface of space-time perception.
The threshold response curved surface is a curved surface constructed by optimizing the reference threshold model parameters through the self-adaptive weight matrix, so that the space-time change of the production environment can be perceived, the shape of the threshold response curved surface is adjusted along with the change of the space-time characteristics, and the mapping relation between the threshold and the space-time characteristics can be intuitively reflected.
Step S406, a real-time detection threshold range dynamically changing along with the production environment is generated through the feature mapping of the threshold response curved surface and the current detection area.
The real-time detection threshold range is generated through the feature mapping of the threshold response curved surface and the current detection area, and can dynamically change with the production environment, so that the defect can be accurately judged under different production conditions.
The method comprises the steps of extracting standard form vectors of corresponding types from a preset defect feature library based on defect types, carrying out statistical analysis on the standard form vectors, calculating to obtain mean value vectors and covariance matrixes, constructing a parameterized reference threshold model, carrying out space-time dimension enhancement processing on the feature vectors of a defect area, combining production condition parameters such as production line running state parameters, environment illumination parameters, glass self attribute parameters, imaging equipment parameters and mechanical vibration parameters to generate space-time feature tensors, inputting the reference threshold model parameters and the space-time feature tensors into a pre-trained attention mechanism network to generate an adaptive weight matrix, carrying out space-time dimension optimization on the reference threshold model parameters based on the adaptive weight matrix, constructing a space-time perception threshold response curve, and generating a real-time detection threshold range dynamically changing along with the production environment through feature mapping of the threshold response curve and a current detection area.
According to the embodiment, the statistical analysis result of the standard form vector, the space-time enhanced defect characteristic and various production condition parameters are integrated, the reference threshold model is dynamically optimized through the attention mechanism, the real-time detection threshold range which can dynamically change along with the production environment is finally generated, and the adaptability and the accuracy of glass flaw detection under the complex working condition are integrally improved.
In one embodiment, the spatio-temporal feature tensor may be calculated by the following formula:
Wherein T st represents a space-time feature tensor, F d represents a defect region feature vector, G i represents a production condition parameter vector, the vector of five production parameters represents G 1 represents a production line running state parameter, G 2 represents an ambient illumination parameter, G 3 represents a glass self attribute parameter, G 4 represents an imaging device parameter, G 5 represents a mechanical vibration parameter, θ i represents a parameter attention weight, norm (. Cndot.) represents a normalization function, and ST_enhancement (. Cndot.) represents a space-time enhancement operator.
Preferably, the spatio-temporal enhancement operator enhances the spatio-temporal correlation of features by 3D convolution and a spatio-temporal attention mechanism.
Where st_attention (X) represents a spatiotemporal Attention mechanism focusing on key features through channels and space.
The parameter attention weight θ i is a coefficient for quantifying the importance degree of five types of production condition parameters (production line running state, ambient light, glass self attribute, imaging device, mechanical vibration parameter) in feature fusion in the process of generating a space-time feature tensor. The determination mode is usually obtained through pre-trained attention mechanism network learning, the network is trained based on the influence degree of various parameters in historical production data on a defect detection result, so that weights are dynamically adapted to different production scenes, namely parameters with larger influence on the current detection scene (such as illumination parameters in a strong light environment) are given higher weights, otherwise, the weights are lower, effective fusion of production condition parameters and defect region feature vectors is finally realized, and the characterization capability of feature tensors on time-space change is improved.
According to the method, five types of production parameter vectors are processed through a standardized function, weighted summation is carried out by combining with the parameter attention weight, tensor multiplication is carried out on the five types of production parameter vectors and the defect area feature vectors, and then the tensor is processed through a space-time enhancement operator, so that the generated space-time feature tensor can effectively fuse space-time information of defect features and multisource production parameters, and comprehensive and accurate feature support is provided for the generation of a subsequent real-time detection threshold range.
In one embodiment, the method for performing secondary scanning on the original image data by using the real-time detection threshold range, calculating the region defect probability and the weight distribution, and generating the defect distribution map may include the following steps:
step S501, dividing the multi-scale detection window for the original image data based on the real-time detection threshold range and extracting the local feature vector of each window.
Preferably, the real-time detection threshold range is a dynamic determination standard for defect detection, wherein the numerical value range for determining whether the image area has defects is dynamically generated according to the production environment and can be adjusted along with the change of conditions such as the running state of the production line and illumination.
The multi-scale detection window is a local area unit with different sizes (such as size and resolution) divided on the original image, and is used for adapting to the defect detection requirements of different sizes and forms, so that the small defects are not missed, and the large defects are not excessively segmented.
Step S502, comparing the local feature vector of each detection window with the real-time detection threshold range, and calculating the probability value of the defect in the window.
The probability value for the presence of a defect within the window is calculated using the following formula:
wherein Y represents a probability value of the presence of a defect within the detection window, A local feature vector representing the current window,Standard feature vectors representing corresponding categories in a preset defect feature library, J real represents a real-time detection threshold range,The degree of similarity of the features is indicated,Represents the distance of the feature from the threshold range, μ represents the sensitivity coefficient, and ε represents the minimum value.
Preferably, feature similarityAnd (4) calculating by adopting cosine similarity:
Distance of feature from threshold range J low represents the lower limit value of the real-time detection threshold range, J high represents the upper limit value of the real-time detection threshold range, the formula value is used for measuring the characteristic vector module length deviation and can be dynamically adjusted in real time according to the defect type, such as the larger value of crack defects
The mu sensitivity coefficient can be dynamically adjusted according to the defect type, for example, the larger value mu=5 is taken by the crack type defect, and the smaller the value is, the more the threshold value requirement is met.
Step S503, combining the spatial position relation and probability value of each detection window, calculating the regional defect weight distribution through Gaussian kernel function.
Step S504, space mapping is carried out on the probability value and the defect weight distribution, and an initial defect distribution map is generated.
The distribution result of the probability weights of the defects of different areas is obtained by calculation through a Gaussian kernel function by combining the spatial position relation (such as the distance and the position relevance of adjacent windows) of each detection window and the probability value of the defects, and the higher the weight is, the greater the probability of the defects of the area is.
The initial defect distribution map is used for representing the image formed by mapping the defect existence probability value of each window and the regional defect weight distribution according to the corresponding space position, and the distribution condition of the defects in the image is initially presented.
Step S505, morphological filtering and connected domain analysis are carried out on the initial defect distribution map, and a refined defect distribution map is obtained.
Preferably, morphological filtering is based on the manipulation of the image morphology (e.g., erosion, dilation, open, close) to process the initial defect distribution map for removing noise interference, smoothing edges, and enhancing the integrity of the defect region.
The connected domain analysis can determine the boundary, size and other information of a single defect through the process of identifying and marking the mutually connected regions with pixel values meeting specific conditions (such as belonging to defect regions) in the morphologically filtered map, and finally, a refined defect distribution map, namely, a final detection result map which clearly and accurately reflects the position, shape and range of the defect, is formed.
The method comprises the steps of dividing original image data into a plurality of scale detection windows based on a real-time detection threshold range, extracting local feature vectors of all windows, comparing the local feature vectors of all detection windows with the real-time detection threshold range, calculating probability values of defects in the windows through a preset formula, calculating regional defect weight distribution through a Gaussian kernel function by combining spatial position relation and the probability values of all detection windows, performing spatial mapping on the probability values and the defect weight distribution to generate an initial defect distribution map, and performing morphological filtering and connected domain analysis on the initial defect distribution map to obtain a refined defect distribution map.
According to the embodiment, the multi-scale detection window and the local feature vector extraction provide a basic unit for defect probability calculation, the probability value formula ensures the accuracy of probability calculation through quantitative comparison of feature similarity and threshold distance, the spatial relevance of a defect area is reflected by the weight distribution generated by combining a Gaussian kernel function with a spatial position relation, the quality of a map is further optimized through morphological filtering and connected domain analysis, data flow in each link is continuous, and the finally obtained refined defect distribution map can accurately reflect the position, probability and distribution characteristics of the surface defects of a glass product and provides a reliable basis for accurate identification and evaluation of glass defects.
In one embodiment, as shown in fig. 2, the present application also provides a computer vision-based glass flaw identification system, which may include:
The candidate region detection module 601 is configured to obtain raw image data of a glass product, and perform feature extraction by using a deep learning model to obtain a defect candidate region set.
The defect feature labeling module 602 is configured to extract pixel-level boundary information from the defect candidate region set by using an example segmentation algorithm, fuse multi-scale morphological features, divide feature clusters, and match with a preset defect feature library to obtain a defect labeling result, where the defect labeling result includes a defect category and a defect region feature vector.
The dynamic threshold generation module 603 is configured to construct a reference threshold model based on the defect category, and construct a reference threshold model by combining the defect region feature vector and the production condition parameter, so as to generate a real-time detection threshold range.
The defect map construction module 604 is configured to perform secondary scanning on the original image data by using the real-time detection threshold range, calculate the region defect probability and the weight distribution, and generate a defect distribution map.
The glass defect identification system based on computer vision comprises a candidate region detection module, a defect feature labeling module, a dynamic threshold generation module, a defect map construction module and a defect map construction module, wherein the candidate region detection module acquires original image data of a glass product, a deep learning model is adopted to conduct feature extraction to obtain a defect candidate region set, the defect feature labeling module utilizes an example segmentation algorithm to extract pixel-level boundary information for the defect candidate region set, a feature cluster is divided after multi-scale morphological features are fused and matched with a preset defect feature library to obtain a defect labeling result comprising defect types and defect region feature vectors, the dynamic threshold generation module constructs a reference threshold model based on the defect types, optimizes the reference threshold model by combining the defect region feature vectors and production condition parameters to generate a real-time detection threshold range, and the defect map construction module utilizes the real-time detection threshold range to conduct secondary scanning on the original image data to calculate region defect probability and weight distribution to generate a defect distribution map. In the system, the candidate region detection module provides basic region data, the defect feature labeling module realizes accurate classification and feature extraction of defects, the dynamic threshold generation module enables the detection threshold to adapt to production environment changes, the defect map construction module finally presents clear defect distribution conditions, and the data flows of the modules are smooth and work cooperatively, so that the efficiency and the precision of glass defect detection are improved, the adaptability to complex production environments is enhanced, and comprehensive and reliable technical support is provided for glass production quality control.
It should be understood that, although the steps in the flowcharts related to the embodiments described above are sequentially shown as indicated by arrows, these steps are not necessarily sequentially performed in the order indicated by the arrows. The steps are not strictly limited to the order of execution unless explicitly recited herein, and the steps may be executed in other orders. Moreover, at least some of the steps in the flowcharts described in the above embodiments may include a plurality of steps or a plurality of stages, which are not necessarily performed at the same time, but may be performed at different times, and the order of the steps or stages is not necessarily performed sequentially, but may be performed alternately or alternately with at least some of the other steps or stages.
In one embodiment, a computer device is provided that includes a memory storing a computer program and a processor that when executed implements the steps of the computer vision-based glass flaw identification method, system, device, and medium as described above.
In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored which, when executed by a processor, implements the steps of the method embodiments described above.
For the device embodiments, reference is made to the description of the method embodiments for the relevant points, since they essentially correspond to the method embodiments. The above-described apparatus embodiments are merely illustrative, wherein the components illustrated as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, may be located in one place, or may be distributed over a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the objectives of the disclosed solution. Those of ordinary skill in the art will understand and implement the present invention without undue burden.
The above examples merely represent a few implementations of the examples of the present application, which are described in more detail and are not to be construed as limiting the scope of the claims of the examples. It should be noted that it will be apparent to those skilled in the art that various modifications and improvements can be made to the present application without departing from the spirit of the embodiments of the application.

Claims (10)

1. A method for identifying glass flaws based on computer vision, the method comprising:
obtaining original image data of a glass product, and extracting features by adopting a deep learning model to obtain a defect candidate region set;
Extracting pixel-level boundary information from the defect candidate region set by using an example segmentation algorithm, merging multi-scale morphological features, dividing feature clusters, and matching with a preset defect feature library to obtain a defect labeling result, wherein the defect labeling result comprises defect types and defect region feature vectors;
constructing a reference threshold model based on the defect category, constructing a reference threshold model by combining the defect region feature vector and the production condition parameters, and generating a real-time detection threshold range;
and performing secondary scanning on the original image data by utilizing the real-time detection threshold range, calculating the region defect probability and weight distribution, and generating a defect distribution map.
2. The method of claim 1, wherein obtaining raw image data of the glass article, performing feature extraction using a deep learning model to obtain a set of defect candidate regions, comprises:
the method comprises the steps of obtaining original image data of glass products, wherein the original image data is obtained by shooting by a high-resolution industrial camera;
Performing visual field alignment and preprocessing on the original image data, and performing feature extraction by using a depth separable convolution network to obtain a feature map set;
performing region segmentation based on the feature map set, and generating a preliminary defect candidate region through an adaptive threshold value, wherein the preliminary defect candidate region comprises a boundary box, a mask and a classification confidence;
Extracting texture features and morphological features from the preliminary defect candidate regions, performing cluster analysis, and screening out high-confidence defect regions;
And optimizing the boundary of the high-confidence defect region by adopting an edge detection operator, generating a high-precision defect positioning mask, performing consistency check with the original region characteristics, and outputting a defect candidate region set.
3. The method according to claim 1, wherein the extracting pixel-level boundary information from the defect candidate region set by using an instance segmentation algorithm, classifying feature clusters after merging multi-scale morphological features, and matching with a preset defect feature library to obtain a defect labeling result includes:
processing pixel-level boundary information of the defect candidate region set by adopting an improved example segmentation algorithm to obtain accurate boundary segmentation data;
extracting multi-scale morphological characteristics based on the boundary segmentation data, and carrying out weighted fusion on the characteristics of different scales to obtain a comprehensive characteristic vector;
Performing cluster analysis on the comprehensive feature vectors by using a self-adaptive density clustering algorithm to determine feature cluster division results;
Carrying out semantic matching on the feature cluster dividing result and a preset defect feature library to obtain preliminary defect marking information, wherein the preset defect feature library comprises standard form vectors, typical texture atlas, outline templates and category labels of various defects;
extracting a multi-dimensional feature vector from the preliminary defect labeling information, and calculating a matching degree score of the multi-dimensional feature vector and a standard template;
and if the matching degree score reaches a preset threshold value, confirming that the defect labeling information is a defect labeling result.
4. A method according to claim 3, wherein the matching degree score of the multi-dimensional feature vector and the standard template is calculated by the following formula:
wherein S represents a matching degree score, Represents the ith dimension characteristic vector of the defect to be detected,Representing the ith dimension feature vector of the standard template, D cos representing cosine distance, omega i representing the feature weight coefficient of the ith dimension, dynamically distributing the feature weight coefficient by the distinguishing degree of various features in a preset defect feature library,The distinction is determined by the reciprocal variance of various defects counted in the training stage on the corresponding feature dimension, n represents the total number of feature dimensions, P represents the outline point set of the defect to be detected, T represents the outline point set of the standard template, D Hausdorff represents the bidirectional Hastedor distance, per represents the outline perimeter, and alpha E [0.6,0.8] represents the weight of the balance feature and structure.
5. The method of claim 1, wherein constructing a reference threshold model based on the defect classification, constructing a reference threshold model in combination with the defect region feature vector and the production condition parameters, generating a real-time detection threshold range, comprises:
Extracting standard morphological vectors of corresponding categories from the preset defect feature library based on the defect categories;
Carrying out statistical analysis on the standard form vector to calculate a mean vector and a covariance matrix, and constructing a parameterized reference threshold model;
Performing space-time dimension enhancement processing on the defect region feature vector, and generating a space-time feature tensor by combining production condition parameters, wherein the production condition parameters comprise production line running state parameters, environment illumination parameters, glass self attribute parameters, imaging equipment parameters and mechanical vibration parameters;
inputting the reference threshold model parameters and the space-time feature tensors into a pre-trained attention mechanism network to generate an adaptive weight matrix;
performing space-time dimension optimization on the reference threshold model parameters based on the self-adaptive weight matrix, and constructing a threshold response surface of space-time perception;
And generating a real-time detection threshold range which dynamically changes along with the production environment through the feature mapping of the threshold response curved surface and the current detection area.
6. The method of claim 5, wherein the spatio-temporal feature tensor is calculated by the following formula:
Wherein T st represents a space-time feature tensor, F d represents a defect region feature vector, G i represents a production condition parameter vector, the vector of five production parameters represents G 1 represents a production line running state parameter, G 2 represents an ambient illumination parameter, G 3 represents a glass self attribute parameter, G 4 represents an imaging device parameter, G 5 represents a mechanical vibration parameter, θ i represents a parameter attention weight, norm (. Cndot.) represents a normalization function, and ST_enhancement (. Cndot.) represents a space-time enhancement operator.
7. The method of claim 1, wherein the performing the secondary scan on the raw image data using the real-time detection threshold range, calculating a region defect probability and a weight distribution, and generating a defect distribution map, comprises:
dividing a multi-scale detection window for the original image data based on the real-time detection threshold range and extracting local feature vectors of each window;
comparing the local feature vector of each detection window with the real-time detection threshold range, and calculating the probability value of the defect in the window;
The probability value for the presence of a defect within the window is calculated using the following formula:
wherein Y represents a probability value of the presence of a defect within the detection window, A local feature vector representing the current window,Standard feature vectors representing corresponding categories in a preset defect feature library, J real represents a real-time detection threshold range,The degree of similarity of the features is indicated,Representing the distance between the feature and the threshold range, μ representing the sensitivity coefficient, ε representing the minimum value;
Combining the spatial position relation of each detection window with the probability value, and calculating the regional defect weight distribution through a Gaussian kernel function;
performing space mapping on the probability value and the defect weight distribution to generate an initial defect distribution map;
and carrying out morphological filtering and connected domain analysis on the initial defect distribution map to obtain a refined defect distribution map.
8. A computer vision-based glass flaw identification system, the system comprising:
The candidate region detection module is used for acquiring original image data of the glass product, and extracting features by adopting a deep learning model to obtain a defect candidate region set;
The defect feature labeling module is used for extracting pixel-level boundary information from the defect candidate region set by utilizing an example segmentation algorithm, fusing multi-scale morphological features, dividing feature clusters, and matching with a preset defect feature library to obtain a defect labeling result, wherein the defect labeling result comprises defect types and defect region feature vectors;
the dynamic threshold generation module is used for constructing a reference threshold model based on the defect category, constructing the reference threshold model by combining the defect region feature vector and the production condition parameter, and generating a real-time detection threshold range;
And the defect map construction module is used for carrying out secondary scanning on the original image data by utilizing the real-time detection threshold range, calculating the region defect probability and the weight distribution, and generating a defect distribution map.
9. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor implements the steps of the method of any of claims 1 to 7 when the computer program is executed.
10. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the steps of the method of any of claims 1 to 7.
CN202511348353.5A 2025-09-20 2025-09-20 Computer vision-based glass flaw identification method, system, equipment and medium Pending CN121258912A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202511348353.5A CN121258912A (en) 2025-09-20 2025-09-20 Computer vision-based glass flaw identification method, system, equipment and medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202511348353.5A CN121258912A (en) 2025-09-20 2025-09-20 Computer vision-based glass flaw identification method, system, equipment and medium

Publications (1)

Publication Number Publication Date
CN121258912A true CN121258912A (en) 2026-01-02

Family

ID=98187476

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202511348353.5A Pending CN121258912A (en) 2025-09-20 2025-09-20 Computer vision-based glass flaw identification method, system, equipment and medium

Country Status (1)

Country Link
CN (1) CN121258912A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN121476234A (en) * 2026-01-06 2026-02-06 河海大学 Non-destructive testing method for road panel defects integrating ultrasonic pulse echo and visual information

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN121476234A (en) * 2026-01-06 2026-02-06 河海大学 Non-destructive testing method for road panel defects integrating ultrasonic pulse echo and visual information

Similar Documents

Publication Publication Date Title
CN119444658A (en) A method for locating textile defects
CN120612327B (en) A visual inspection system and method for minor defects in industrial products
CN120747040B (en) Cutting workpiece defect detection method and system based on image characteristic feedback
CN118506067B (en) Image processing method and system for power grid identification
CN120235871B (en) Intelligent drug particle detection system and method based on image processing
CN118279304B (en) Abnormal recognition method, device and medium for special-shaped metal piece based on image processing
CN120580226B (en) Automatic detection and classification method of intelligent manufacturing defects based on machine vision
CN119091236B (en) Ceramic packaging substrate detection method and system based on visual inspection and meta-learning
CN119417817B (en) Paint defect detection method and system based on machine vision
CN119399187A (en) A method for identifying textile defect types
CN119919387B (en) Method, system, product and medium for detecting surface flaws of optical film
CN120580702B (en) Few-sample defect identification method based on cross-modal text semantic driving
CN113313678A (en) Automatic sperm morphology analysis method based on multi-scale feature fusion
CN121258912A (en) Computer vision-based glass flaw identification method, system, equipment and medium
CN119784741A (en) A training method and system for 5G flexible circuit board defect detection model
CN120951053B (en) Foreign matter layering detection method, system and electronic equipment
CN120375363B (en) Image Recognition-Based Food Conveying Detection Method and Device
CN120894345A (en) Machine Vision-Based Defect Detection Method and System for Automotive Injection Molded Parts
CN120877276A (en) Litchi phenotype identification and weight estimation method based on multi-modal learning
CN119540941A (en) A three-dimensional target detection method based on stereo vision technology
CN120088182A (en) Precision injection mold surface defect visual recognition system and method
CN118587496A (en) Automatic identification system and method of parts processing accuracy based on computer vision
CN120656168B (en) PE film surface electrostatic dust recognition system and method
CN121563979B (en) A Semiconductor Wafer Visual Inspection Method Based on Interactive Feature Learning
CN117315342B (en) YOLOv 7-based target detection method under special weather conditions under visible light

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination