CN119048496B

CN119048496B - PCB defect detection method, system, equipment and storage medium

Info

Publication number: CN119048496B
Application number: CN202411523259.4A
Authority: CN
Inventors: 邓承志; 张优; 吴朝明; 徐晨光; 李彦
Original assignee: Nanchang Institute of Technology
Current assignee: Jiangxi University Of Water Resources And Electric Power
Priority date: 2024-10-30
Filing date: 2024-10-30
Publication date: 2025-02-18
Anticipated expiration: 2044-10-30
Also published as: CN119048496A

Abstract

The application discloses a PCB defect detection method, a system, equipment and a storage medium, which belong to the technical field of industrial product defect detection, wherein the method comprises the steps of obtaining a PCB data set, carrying out image preprocessing and defect labeling on the PCB data set, creating a virtual operation environment, constructing a YOLOv model in the virtual operation environment, carrying out light weight processing on a YOLOv model, optimizing a space-to-depth convolution SPD_conv module into a DSPD module, configuring the DSPD module into the YOLOv model to obtain a DSPD_ YOLOv10 model, constructing a bidirectional feature pyramid network BiFPN, configuring the bidirectional feature pyramid network into a neck network of the DSPD_ YOLOv10 model, training the DSPD_ YOLOv10 model according to the PCB data set, detecting a sample to be detected according to the trained DSPD_ YOLOv10 model, and detecting the PCB defect by using the DSPD_ YOLOv10 model, so that high-precision and high-efficiency detection of the PCB defect is realized.

Description

PCB defect detection method, system, equipment and storage medium

Technical Field

The application belongs to the technical field of industrial product defect detection, and particularly relates to a PCB defect detection method, a system, equipment and a storage medium.

Background

The printed circuit board (printed circuit board, PCB) is an important component in electronic products, and its surface has circuit elements and dense wire layout, etc., and the advantages and disadvantages of the PCB are related to the performance and lifetime of the electronic products. The development of integrated circuit (INTEGRATED CIRCUIT, IC) packaging technology has driven the development of electronic products toward lighter, thinner, and smaller designs, which also requires a more compact circuit layout of PCBs, and thus industry demands for quality of PCBs are increasing.

The defect detection is a key link for ensuring the quality of the PCB, the defect detection faces a plurality of difficulties which need to be overcome urgently, the defect sizes of a hole (missing hole), a mouse bit, an open circuit (open circuit), a short circuit (short), a burr (spray), a false copper (spurious copper) and the like are relatively smaller on the whole PCB picture, the background picture of the PCB also has an influence on the detection, and the background picture of the PCB has a great influence on the detection precision of the PCB, so the problems are solved, and the detection precision of the PCB is improved.

Disclosure of Invention

The embodiment of the application aims to provide a PCB defect detection method, a system, equipment and a storage medium, which can solve the technical problem of low detection precision of PCB defects in the prior art.

In order to solve the technical problems, the application is realized as follows:

In a first aspect, an embodiment of the present application provides a method for detecting a defect of a PCB, including:

acquiring a PCB data set, and performing image preprocessing and defect labeling on the PCB data set;

creating a virtual operation environment, constructing YOLOv a model in the virtual operation environment, and performing light weight processing on the YOLOv model;

Optimizing a space-to-depth convolution SPD_conv module into a DSPD module, and configuring the DSPD module into the YOLOv model to obtain a DSPD_ YOLOv10 model;

Constructing a bi-directional feature pyramid network BiFPN and configuring the bi-directional feature pyramid network BiFPN into the neck network of the DSPD YOLOv model;

inputting the PCB data set into the DSPD_ YOLOv10 model for training, and evaluating the training result;

And detecting the sample to be detected according to the trained DSPD_ YOLOv10 model to obtain a detection result.

As an optional implementation manner of the first aspect of the present application, the obtaining a PCB dataset, performing image preprocessing and defect labeling on the PCB dataset, and specifically includes:

imaging the PCB produced industrially, finishing to obtain a PCB data set containing defects including holes, mouse bites, open circuits, short circuits, burrs and false copper, expanding the PCB data set in a data enhancement mode, performing self-adaptive linear interpolation super-resolution reconstruction processing on the expanded PCB data set, marking the defects, and finally dividing the PCB data set into a training set, a testing set and a verification set.

According to the alternative embodiment, the diversity and the definition of the data set are increased through data enhancement and self-adaptive linear interpolation super-resolution reconstruction processing, so that the recognition capability of a model on tiny PCB defects is improved, accurate defect labeling and reasonable data set division are helpful for the model to learn more accurate feature representation, so that the defect detection accuracy is improved, although the defect labeling needs a certain manpower investment, the burden can be greatly reduced through an automatic and semi-automatic tool, and meanwhile, once model training is finished, a large number of PCB images can be efficiently detected, so that the time and the cost of manual inspection are remarkably reduced.

As an optional implementation manner of the first aspect of the present application, the process of performing the light weight processing on the YOLOv model is:

The normal convolution in the YOLOv model is replaced with a lightweight phantom convolution GhostConv.

According to the alternative embodiment, the calculation amount of the YOLOv model can be remarkably reduced by introducing GhostConv, the generation process of the phantom feature map is more efficient than the traditional convolution operation, so that the calculation complexity of the model is reduced, the memory occupation is reduced, the YOLOv model subjected to light weight processing has smaller memory occupation, the model is beneficial to being deployed in an environment with limited resources (such as embedded equipment or mobile equipment), the practicability is improved, and the YOLOv model subjected to light weight processing has an improved operation speed due to the reduction of the calculation amount and the memory occupation. The method is particularly important for real-time defect detection tasks, can shorten detection time and improve production efficiency, and can ensure that the model is not greatly influenced in performance through reasonable network design and parameter adjustment although the calculation amount and memory occupation of the model are reduced through light-weight treatment, which means that the YOLOv model after light weight can still keep higher defect detection precision.

As an optional implementation manner of the first aspect of the present application, the space-to-depth convolution spd_conv module includes a space-to-depth layer and a non-strided convolution layer, and the space-to-depth convolution spd_conv module is optimized as a DSPD module, and the DSPD module is configured into the YOLOv model, specifically:

Deleting the non-strided convolution layers in the space-to-depth convolution SPD_conv module, and optimizing adaptive weighting processing and depth ordering of the space-to-depth layers to obtain the DSPD module;

The step size of the first convolution of the backbone network in the YOLOv model is kept unchanged, and the DSPD module is configured after the rest of convolution layers.

According to the alternative embodiment, the traditional non-strided convolution layers are deleted, information loss caused by a compression channel is avoided, the detection precision of a YOLOv model can be improved by deleting the non-strided convolution layers, the defect characteristic proportion of each characteristic sub-image can be obtained through self-adaptive weighting processing, corresponding weight is further given, the model is more targeted to processing of defects, unnecessary resource waste is reduced, the detection precision of the model is further improved, the space dimension of each characteristic sub-image is ensured to be kept the same through space dimension processing, the depth ordering of the space-to-depth layers is optimized, the operation efficiency of the YOLOv model is improved, particularly in a resource limited environment, the DSPD module can better extract space characteristics, convert the space dimension of an input image into the depth dimension, compared with the compression channel of the non-strided convolution layers, the channel processing is more targeted, loss of characteristic information is reduced, the DSPD module needs to reserve as much space information as possible when processing low-resolution images and small objects, the DSPD module can convert the space dimension into the depth dimension, the depth dimension of the conventional model, and the depth dimension can be prevented from being lost, and the depth ordering operation can be accelerated through the conventional pool.

As an alternative implementation of the first aspect of the present application, the DSPD module processes the image as follows,

And slicing the input feature map to obtain a plurality of feature subgraphs, wherein the specific formula is as follows:

;

Performing self-adaptive weighting processing and space dimension processing on the characteristic subgraph to enable the space dimensions of the characteristic subgraph to be consistent, wherein the specific formula is as follows:

;

And then performing splicing operation on the characteristic subgraphs subjected to self-adaptive weighting treatment and space dimension treatment, wherein the specific formula is as follows:

;

Wherein, Represents a sampling slice function, X represents an input feature map, s represents a scale, e represents a sampling step size, X _s-1,s-1 represents a feature whose position is (s-1 ) in the input feature information, P _s-1,s-1 represents X _s-1,s-1 for performing adaptive weighting processing and spatial dimension processing,、、AndThe characteristic weight values corresponding to X _s-1,s-1、X_s-2,s-1、X_s-1,s-2 and X _0,0 are respectively shown,The representation is a spatial dimension of the feature,And the cat represents output characteristic information corresponding to the proportion s, and is a function for splicing the characteristic graphs.

According to the alternative embodiment, the step length e and the proportion s are set, the sampling slice function samples the feature information and slices the feature information into a plurality of slices with the same size, and the slices which are selected to contain defects according to preset conditions, for example, X _s-2,s-1 is one slice in the slices, subscripts s-2 and s-1 respectively represent the horizontal position and the vertical position corresponding to the slices, after the slices are selected, adaptive weighting processing and spatial dimension processing are carried out on each feature map, the model is more specific to the defect processing, unnecessary resource waste is reduced, the detection precision of the model is further improved, the spatial dimension of each feature map is guaranteed to be kept the same, the cat function splices the slices which are selected by the sampling slice function and meet preset features according to preset sequences, so that a large number of slices containing the defect features are obtained in the new feature map, the DSPD module generates new feature representation in a mode of deep dimension splicing the slices through the slices, the process is beneficial to capturing the spatial features more effectively, in the process, the proportion s is selected, the step length e processing, the offset and the index is adjusted to the depth dimension is designed to be more effectively, and the model is integrated into the model according to the performance factor YOLOv, and the performance factor of the model is improved effectively, and the model is integrated to the performance factor 39is designed.

As an optional implementation manner of the first aspect of the present application, the configuring the bidirectional feature pyramid network BiFPN into the neck network of the dspd_ YOLOv10 model specifically is:

The part Concat layer in the YOLOv model neck network is replaced with the bi-directional feature pyramid network BiFPN.

According to the alternative implementation mode, the BiFPN module can generate a characteristic diagram with higher expressive force through trans-scale characteristic fusion, so that the detection capability of the model to targets with different scales is improved, biFPN is configured in a neck network of the DSPD_ YOLOv10 model, the detection precision and efficiency of the model can be improved, the selected part Concat layer is more beneficial to capturing small target information than the rest Concat layer, and the optimal neutralization is selected in terms of precision and reasoning speed, the requirement of meeting industrial speed is omitted for pursuing precision, and the number of input and output sizes and characteristic diagrams of the model is required to be kept unchanged when the Concat layer is replaced with the BiFPN module for pursuing speed, so that the compatibility of the model is maintained.

As an optional implementation manner of the first aspect of the present application, in the evaluating the training result, the evaluation index includes a recall rate, an accuracy, an average accuracy mean value, and a frame rate;

the recall rate is calculated by the following formula:

;

the accuracy is calculated by the following formula:

;

The average accuracy is calculated by the following formula:

;

The average accuracy mean value is calculated by the following formula:

;

the frame rate is calculated by the following formula:

;

Wherein Recall represents Recall, precision represents Precision, AP represents average Precision, mAP represents average Precision mean, FPS represents frame rate, F _N represents the number of positive samples predicted as negative samples, F _P represents the number of positive samples predicted as negative samples, T _P represents the number of positive samples predicted as positive samples, P (r) represents a functional image surrounded by Recall and Precision, n represents defect class number, i represents ith class, F _n represents the number of pictures to be detected, and T represents time taken to detect all pictures to be detected.

According to the alternative embodiment, the average accuracy is an area value obtained by integrating accuracy-recall (P-R) curves under different confidence thresholds; it provides a single performance index for measuring the accuracy of the model at different recall levels; the average accuracy mean value is one of key indexes for measuring the performance of a target detection algorithm, the average detection accuracy of all categories is considered, the average accuracy mean value can comprehensively reflect the detection performance of the algorithm on different categories by calculating the accuracy of each category and taking the average value, the higher the average accuracy mean value is, the more accurate the algorithm can identify a target object and is more stable on different categories, the accuracy rate reflects the proportion of the actual positive sample in the example of the algorithm predicted as the positive sample, the higher the accuracy rate is, the more reliable the result of the algorithm prediction is, the false alarm condition is reduced, the higher the accuracy rate can ensure that the detected defects are more accurate in PCB defect detection, unnecessary reworking and cost waste are reduced, the higher the recall rate is, the algorithm can detect more actual defects, the missing report condition is reduced, the higher the recall rate is vital to ensure the quality and the safety of the product, the frame rate indicates the number of pictures which can be processed in each second, the higher the frame rate is, the frame rate can be processed at a higher rate, the frame rate can be processed in the high-speed or the frame rate can be processed in real-time, and the frame rate can be processed in the rate is high, and the cost can be processed in real time, and the time is close to the real-time, and the cost can be processed in the frame rate is reduced.

In a second aspect, an embodiment of the present application provides a PCB defect detection system, the system comprising:

the first acquisition module is used for establishing YOLOv a network model and acquiring a PCB data set required by the network model, wherein the PCB data set comprises a training set, a verification set and a test set;

The first processing module is used for preprocessing the PCB data set acquired by the first acquisition module to obtain the preprocessed PCB data set;

The second processing module is used for carrying out light weight processing on the YOLOv model established by the first acquisition module;

The first improvement module optimizes a space-to-depth convolution (DSPD_conv) module to be a DSPD module, and configures the DSPD module into the YOLOv model after light weight processing to obtain a DSPD_ YOLOv model;

a second improvement module, configured to configure a preset bi-directional feature pyramid network BiFPN into a neck network of the dspd_ YOLOv10 model;

A first training module, which trains the DSPD_ YOLOv10 model modified by the second modification module through the PCB data set preprocessed by the first processing module;

and the first evaluation module is used for evaluating the training result of the first training module.

According to the PCB defect detection system, the first acquisition module can establish a YOLOv model and effectively acquire and divide a PCB data set into a training set, a verification set and a test set, which ensures that sufficient and diversified data support is provided in the model training process, so that the generalization capability of the model is improved, the first processing module carries out preprocessing on the PCB data set, which generally comprises operations such as image enhancement and the like, is beneficial to improving the recognition accuracy of the model on the PCB defect and reducing the influence of noise on model training, the second processing module carries out light-weight processing on the YOLOv model, which can remarkably reduce the calculation complexity and the storage requirement of the model, so that the model is more suitable for running on hardware with limited resources, the practical applicability of the system is improved, and the first improvement module can further improve the detection accuracy of the model by optimizing a space-to-depth convolution SPD_conv module into a DSPD module and configuring the DSPD module into a YOLOv model with light weight. The DSPD module can enhance the recognition capability of the model to the PCB defects through more effective feature extraction and conversion, the second improvement module configures a preset bidirectional feature pyramid network BiFPN into a neck network of the DSPD_ YOLOv model, which is favorable for the model to better fuse multi-scale features and improve the detection capability of the PCB defects with different sizes and shapes, the first training module trains the improved DSPD_ YOLOv model by using a preprocessed PCB data set to ensure that the model can learn the effective features of the PCB defects, and the first evaluation module evaluates training results, can quantify the performance of the model, including indexes such as detection precision, recall rate and the like, and provides data support for further optimization of the model.

In a third aspect, an embodiment of the present application provides an electronic device comprising a processor, a memory and a program or instructions stored on the memory and executable on the processor, the program or instructions implementing the steps of the method as described in the first aspect when executed by the processor.

In a fourth aspect, embodiments of the present application provide a readable storage medium having stored thereon a program or instructions which when executed by a processor perform the steps of the method according to the first aspect.

According to the embodiment of the application, the SPD_conv module is optimized to be the DSPD module and is configured with BiFPN networks, the DSPD_ YOLOv10 model can more effectively capture and utilize characteristic information in images, so that the defect detection precision is improved, the YOLOv model is subjected to light weight treatment, the calculated amount and memory occupation of the model can be reduced, the running efficiency of the model is improved, the model is more suitable for being deployed in practical application, the dependence relationship in the model development process can be conveniently managed by creating a virtual running environment, the consistency and repeatability of the model in different environments are ensured, and the model extensibility and maintainability are improved.

Drawings

FIG. 1 is a flowchart of a method for detecting PCB defects according to some embodiments of the present application;

fig. 2 is a schematic diagram of a dspd_ YOLOv10 model in a method for detecting a PCB defect according to some embodiments of the present application.

Detailed Description

The following description of the embodiments of the present application will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are some, but not all embodiments of the application. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.

The terms first, second and the like in the description and in the claims, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that embodiments of the application may be practiced otherwise than as specifically illustrated or described herein. Furthermore, in the description and claims, "and/or" means at least one of the connected objects, and the character "/", generally means that the associated object is an "or" relationship.

The method, the system, the equipment and the storage medium for detecting the PCB defects provided by the embodiment of the application are described in detail through specific embodiments and application scenes thereof with reference to the accompanying drawings.

Examples

A method of detecting defects in a PCB comprising the steps of:

s100, acquiring a PCB data set, and performing image preprocessing and defect labeling on the PCB data set;

The method comprises the steps of imaging an industrially produced PCB, sorting to obtain a PCB data set containing defects including holes, mouse bites, open circuits, short circuits, burrs and false copper, expanding the PCB data set in a data enhancement mode, performing self-adaptive linear interpolation super-resolution reconstruction processing on the expanded PCB data set, marking the defects, and finally dividing the PCB data set into a training set, a testing set and a verification set.

It will be appreciated that based on the above feature definition, the produced PCB is first imaged by an industrial camera or other imaging device, which images should clearly show various details on the PCB, including possible defects, and the imaged PCB images are consolidated to form a PCB dataset containing various defects (such as holes, rat bites, opens, shorts, burrs and copper shavings), which is the basis for subsequent processing, and the PCB dataset is extended in a data enhancement manner in order to increase the diversity and robustness of the dataset. The data enhancement technique may include rotation, scaling, flipping, color transformation, etc. which can generate more training samples without changing the essential content of the image, performing adaptive linear interpolation super-resolution reconstruction processing on the extended PCB data set, which aims to improve the resolution and definition of the image, thereby helping the model to identify defects more accurately, adaptive linear interpolation is an image interpolation method, which can adaptively select interpolation weights according to local features of the image, thereby obtaining better reconstruction effect, and after image preprocessing, accurate labeling of defects in the image is required, which usually involves manual or semi-automatic tools to mark the positions and types of defects. The accuracy of the labeling is critical to the training effect of the model, and finally, the processed PCB data set is divided into a training set, a testing set and a verification set, wherein the training set is used for the training process of the model, the testing set is used for evaluating the performance of the model, namely the performance of the model on unseen data, and the verification set is used for adjusting the model parameters in the training process so as to avoid overfitting. This partitioning helps to ensure generalization and stability of the model.

It should be noted that, through data enhancement and adaptive linear interpolation super-resolution reconstruction processing, diversity and definition of data sets are increased, so that recognition capability of a model on tiny PCB defects is improved, accurate defect labeling and reasonable data set division are helpful for model learning to more accurate feature representation, so that defect detection accuracy is improved, although defect labeling requires a certain manpower input, the burden can be greatly reduced through an automatic and semi-automatic tool, and meanwhile, once model training is completed, a large number of PCB images can be efficiently detected, so that time and cost of manual inspection are remarkably reduced.

S200, creating a virtual operation environment, constructing YOLOv a model in the virtual operation environment, and carrying out light weight treatment on the YOLOv model;

further, the process of performing the light weight processing on the YOLOv model is as follows:

It should be understood that, based on the above feature definition, in the PCB defect detection method, the light-weight processing of the YOLOv model is a key step, aiming at reducing the calculation amount and memory occupation of the model and improving the operation efficiency in the resource-constrained environment, specifically, the light-weight processing is to replace the common convolution in the YOLOv model with the light-weight phantom convolution (GhostConv), the common convolution directly generates all feature images (feature maps) in the convolution operation, while GhostConv generates the feature images in a more efficient manner, firstly generates a part of the feature images (called basic feature images) through a small amount of traditional convolution operation, then generates the rest of the feature images (called phantom feature images) through linear transformation (such as cheap operation or depth separable convolution), and the phantom feature images have similar feature information with the basic feature images, but the calculation amount is greatly reduced;

It should be noted that by introducing GhostConv, the calculation amount of the YOLOv model can be significantly reduced, the generation process of the phantom feature map is more efficient than the traditional convolution operation, so that the calculation complexity of the model is reduced, the memory occupation is reduced, the YOLOv model subjected to light weight processing has smaller memory occupation, the model is beneficial to being deployed in an environment with limited resources (such as embedded equipment or mobile equipment), the practicability is improved, and the YOLOv model subjected to light weight processing has improved operation speed due to the reduction of the calculation amount and the memory occupation. The method is particularly important for real-time defect detection tasks, can shorten detection time and improve production efficiency, and can ensure that the model is not greatly influenced in performance through reasonable network design and parameter adjustment although the calculation amount and memory occupation of the model are reduced through light-weight treatment, which means that the YOLOv model after light weight can still keep higher defect detection precision.

S300, optimizing a space-to-depth convolution SPD_conv module into a DSPD module, and configuring the DSPD module into a YOLOv model to obtain a DSPD_ YOLOv10 model;

further, the space-to-depth convolution SPD_conv module comprises a space-to-depth layer and a non-strided convolution layer, and is optimized to be a DSPD module and is configured into a YOLOv model, specifically:

Deleting a non-strided convolution layer in the space-to-depth convolution SPD_conv module, and optimizing self-adaptive weighting treatment and depth ordering of the space-to-depth layer to obtain a DSPD module;

Maintaining the step length of the first convolution of the backbone network in the YOLOv model unchanged, and configuring a DSPD module after the rest convolution layers;

Specifically, the DSPD modules are configured after GhostConv, ghostConv3, SCDown, and SCDown2 of the backbone network and GhostConv, and SCDown of the neck network in the YOLOv model, while the convolution steps of GhostConv, ghostConv3, SCDown, and SCDown2 in the backbone network are set to 1, the steps of GhostConv, and SCDown1 in the neck network are set to 1, and the upsampling structure in the neck network remains unchanged.

It will be appreciated that based on the above feature definition, the original space-to-depth (spd_conv) module includes a space-to-depth layer and a non-strided convolution layer, in the optimization process, the non-strided convolution layer in the spd_conv module is deleted first, which aims to improve the capability of the module to acquire information, because the non-strided convolution layer compresses channels of the spliced features, which can result in partial information loss, after deleting the non-strided convolution layer, the depth ordering of the space-to-depth layer is optimized, the depth ordering refers to rearranging the channel dimensions of the input data to better utilize the characteristics of the space-to-depth layer, the optimized depth ordering can enable the space-to-depth layer to extract the space features more effectively, and reduce the computation amount, and finally, the optimized DSPD module is configured into the YOLOv model, in order to maintain the structural integrity and performance of the model, maintain the first convolved layer of the main network in YOLOv, and fully utilize the computation amount of the DSPD module to reduce the computation amount of the remaining convolved layers.

It is noted that the traditional non-strided convolution layers are deleted to avoid information loss caused by compression channels, the detection precision of the YOLOv model can be improved by deleting the non-strided convolution layers, the defect characteristic proportion of each characteristic sub-image can be obtained through self-adaptive weighting processing, corresponding weights are further given to the defect characteristic proportion, the model is more targeted to processing of the defects, unnecessary resource waste is reduced, the detection precision of the model is further improved, the spatial dimension of each characteristic sub-image is ensured to be kept the same through spatial dimension processing, the depth ordering of the space-to-depth layers is optimized, the operation efficiency of the YOLOv model is improved, particularly in a resource limited environment, the DSPD module can better extract spatial features, convert the spatial dimension of an input image into the depth dimension, compared with the compression channels of the non-strided convolution layers, the channel processing is more targeted, loss of characteristic information is reduced, the DSPD module needs to retain as much spatial information as possible when processing low-resolution images and small objects, the DSPD module converts the information of the spatial dimension into the depth dimension, the depth dimension is converted into the traditional dimension, and the depth dimension is prevented from being lost to the model through the depth ordering operation in the conventional pool.

Still further, the process of the DSPD module processing the image is as follows:

the input feature map is sliced to obtain a plurality of feature subgraphs, and the specific formula is as follows:

;

Performing self-adaptive weighting processing and space dimension processing on the feature subgraph to enable the space dimensions of the feature subgraph to be consistent, wherein the specific formula is as follows:

;

Specifically, based on the above formula, X is set as an input feature map, the dimensions of C X H X W and C, H, W respectively represent the channel number, height and width of the feature map X, an initial feature map X is sliced by a DSPD module with the proportion of 2 and the step length of 1 to obtain four feature subgraphs X _0,0、X_1,0、X_0,1、X_1,1, the channel number of each feature subgraph is 1/4 times of the original one, the length and width of each feature subgraph are 1/2 times of the original feature map X, then the self-adaptive weighting treatment and the space dimension treatment are carried out to obtain P _0,0、P_1,0、P_0,1、P_1,1, finally the channel dimensions are combined according to the optimization to obtain a new feature map with the depth of the original feature map, and the depth of the new feature map is the original oneThe splicing process is as follows:

。

It is to be understood that, based on the feature definition, the sampling slice function samples the feature information by setting the step e and the proportion s and cuts the feature information into a plurality of slices with the same size, for example, X _s-2,s-1 is one slice of the slices, subscripts s-2 and s-1 respectively indicate the horizontal position and the vertical position corresponding to the slice, adaptive weighting processing and spatial dimension processing are performed on the slices after the slicing operation is completed, and the cat function sequentially performs stacking and splicing of the spatial depth dimension on the slices subjected to the adaptive weighting processing and the spatial dimension processing, thereby improving the capability of capturing the features of the model.

It should be noted that, by setting step e and proportion s, the sampling slicing function samples the feature information into a plurality of slices with the same size, and selecting a slice containing a defect according to a preset condition, for example, X _s-2,s-1 is a slice in the slices, subscripts s-2 and s-1 respectively represent the horizontal position and the vertical position corresponding to the slice, after the slice is selected, adaptive weighting processing and space dimension processing are performed on each feature map, so that the model has more pertinence to the defect processing, unnecessary resource waste is reduced, the detection precision of the model is further improved, the space dimension of each feature sub-map is ensured to be the same, cat function splices the slices which are selected by the sampling slicing function and meet the preset feature according to the preset sequence, thus obtaining a large number of slices containing the defect feature in the new feature map, the DSPD module generates a new feature representation in a mode of slicing and depth dimension splicing the slices, the process is helpful for capturing the space feature more effectively, during the processing, the proportion s is more pertinence to the processing, the step e processing, the index and the offset adjustment are all effective in the depth of the model, and the performance of the model is reasonably improved by the DSPD module 5310, and the performance of the model is improved by the effect of the model is reasonably integrating the performance factors in the model design.

S400, constructing a bidirectional feature pyramid network BiFPN, and configuring the bidirectional feature pyramid network BiFPN into a neck network of the DSPD_ YOLOv10 model;

further, the configuration process is as follows:

Replacing part Concat layer in YOLOv model neck network with a bi-directional feature pyramid network BiFPN;

Specifically, the twelfth layer Concat0 and the fifteenth layer Concat1 in the YOLOv original neck network are replaced with BiFPN, and the remaining Concat layers remain unchanged.

It will be appreciated that based on the feature definition described above, in the neck network of the YOLOv model, the Concat layers are used to stitch the different layers of feature maps, which typically have different resolutions and channel numbers, that these Concat layers need to be identified and that it is determined which layers can be replaced with the BiFPN module, that the selected twelfth Concat and fifteenth Concat layers are replaced with the BiFPN module, that the BiFPN module fuses features through top-down and bottom-up paths and uses weighted features and cross-scale connections to enhance the effect of feature fusion, that during replacement it is necessary to ensure that the inputs and outputs of the BiFPN module are consistent with the inputs and outputs of the original Concat layers to maintain the integrity and performance of the model, that the BiFPN module contains a number of learnable parameters, such as weights and offsets, that need to be optimized during training, that the parameters of the BiFPN module need to be adjusted according to the requirements and characteristics of the data set of the DSPD_ YOLOv model, that after the replacement and adjustment parameters, that the DSPD_2 needs to be retrained to effectively monitor the model and that the model needs to be adjusted in part of the model and that the model needs to be efficiently trained and the rest of the model.

It should be noted that, the BiFPN module can generate a feature map with better expressive force through trans-scale feature fusion, so as to improve the detection capability of the model to targets with different scales, the BiFPN is configured in the neck network of the DSPD_ YOLOv10 model, so that the detection precision and efficiency of the model can be improved, and the selected part Concat layer is more beneficial to capturing small target information than the rest Concat layer, and the selected part Concat layer is more beneficial to capturing small target information, because the optimal neutralization is selected in terms of precision and reasoning speed, the requirement of meeting industrial speed is not ignored for pursuing precision, and the number of input and output dimensions and feature maps of the model is required to be kept unchanged when the Concat layer is replaced with the BiFPN module because of pursuing speed, so that the compatibility of the model is maintained.

S500, inputting the PCB data set into a DSPD_ YOLOv10 model for training, and evaluating a training result;

Further, in the process of evaluating the training result, the evaluation indexes comprise recall rate, accuracy, average accuracy mean value and frame rate;

recall is calculated by the following formula:

;

the accuracy is calculated by the following formula:

;

the average accuracy is calculated by the following formula:

;

the average accuracy mean is calculated by the following formula:

;

The frame rate is calculated by the following formula:

;

It should be understood that, based on the above feature definition, the average accuracy is the area value obtained by integrating the accuracy-recall (P-R) curves at different confidence thresholds; it provides a single performance index for measuring the accuracy of the model at different recall levels; the average accuracy mean value is one of key indexes for measuring the performance of a target detection algorithm, the average detection accuracy of all categories is considered, the average accuracy mean value can comprehensively reflect the detection performance of the algorithm on different categories by calculating the accuracy of each category and taking the average value, the higher the average accuracy mean value is, the more accurate the algorithm can identify a target object and is more stable on different categories, the accuracy rate reflects the proportion of the actual positive sample in the example of the algorithm predicted as the positive sample, the higher the accuracy rate is, the more reliable the result of the algorithm prediction is, the false alarm condition is reduced, the higher the accuracy rate can ensure that the detected defects are more accurate in PCB defect detection, unnecessary reworking and cost waste are reduced, the higher the recall rate is, the algorithm can detect more actual defects, the missing report condition is reduced, the higher the recall rate is vital to ensure the quality and the safety of the product, the frame rate indicates the number of pictures which can be processed in each second, the higher the frame rate is, the frame rate can be processed at a higher rate, the frame rate can be processed in the high-speed or the frame rate can be processed in real-time, and the frame rate can be processed in the rate is high, and the cost can be processed in real time, and the time is close to the real-time, and the cost can be processed in the frame rate is reduced.

S600, detecting a sample to be detected according to a trained DSPD_ YOLOv10 model to obtain a detection result;

According to a method for detecting defects of a PCB of the present embodiment, first, a large amount of PCB (printed circuit board) image data including various possible defect types such as breakage, short circuit, missing elements, etc. needs to be collected, and the collected PCB image is subjected to preprocessing such as denoising, contrast enhancement, etc. to improve the image quality. Meanwhile, defects in the image are accurately marked, which is the basis for identifying defects by training the model, a virtual running environment is created for isolating and managing the dependency relationship in the model development process, which is helpful for ensuring the consistency and repeatability of the model in different environments, and a YOLOv model is built in the virtual running environment, YOLOv is an advanced target detection model, and has high precision and real-time performance. In order to further improve the running efficiency of the model, the model is subjected to light weight processing such as pruning, quantization and the like, a space-to-depth convolution (SPD_conv) module is optimized to be a more efficient DSPD module and is configured into a YOLOv model, the step aims to improve the processing capacity of the model to space features and reduce the calculation amount, a bidirectional feature pyramid network BiFPN is configured, a bidirectional feature pyramid network (BiFPN) is configured in a neck network of the DSPD_ YOLOv10 model, biFPN can be used for effectively fusing feature information of different scales, so that the capturing capacity of the model to shallow feature information is improved, the defect detection accuracy is improved, and the DSPD_ YOLOv10 model is trained by using a preprocessed PCB data set. In the training process, the model learns how to identify and locate the defects in the images, and after the training is finished, the main performance indexes of the model, such as accuracy, recall rate and the like, are analyzed. The indexes can reflect the performance of the model on the defect detection task, and the performance of the model is evaluated according to the analysis result. If the model performs poorly, it may be necessary to adjust the model structure, optimize parameters, or augment training data, etc., and if the model performs well, the next application may be performed.

It is to be noted that, by optimizing the spd_conv module to be a DSPD module and configuring BiFPN network, the dspd_ YOLOv10 model can more effectively capture and utilize the characteristic information in the image, thereby improving the accuracy of defect detection, carrying out light weight processing on the YOLOv model, reducing the calculation amount and memory occupation of the model, thereby improving the operation efficiency of the model, making the model more suitable for deployment in practical application, conveniently managing the dependency relationship in the model development process by creating a virtual operation environment, ensuring the consistency and repeatability of the model in different environments, which is helpful for enhancing the expandability and maintainability of the model.

It should be noted that, in the method for detecting a PCB defect provided in the embodiment of the present application, the execution body may be a PCB defect detection system, or a control module in the PCB defect detection system for executing and loading a method for detecting a PCB defect. In the embodiment of the application, a method for detecting a PCB defect is described by taking a method for detecting a PCB defect carried out and loaded by a PCB defect detecting system as an example.

A PCB defect detection system comprising the following modules:

The first acquisition module is used for establishing YOLOv a model and acquiring a PCB data set required by the network model, wherein the PCB data set comprises a training set, a verification set and a test set;

The first processing module is used for preprocessing the PCB data set acquired by the first acquisition module to obtain a preprocessed PCB data set;

The first improvement module optimizes the space-to-depth convolution SPD_conv module into a DSPD module, and configures the DSPD module into a YOLOv model after light weight processing to obtain a DSPD_ YOLOv model;

The second improvement module configures a preset bidirectional feature pyramid network BiFPN into a neck network of the dspd_ YOLOv10 model;

the first training module trains the DSPD_ YOLOv model improved by the second improvement module through the PCB data set preprocessed by the first processing module;

A PCB defect detection system in the embodiment of the application can be a device, a component in a terminal, an integrated circuit or a chip. The device may be a mobile electronic device or a non-mobile electronic device. By way of example, the mobile electronic device may be a mobile phone, a tablet computer, a notebook computer, a palm computer, an ultra-mobile personal computer (UMPC), a netbook, a Personal Digital Assistant (PDA), or the like, and the non-mobile electronic device may be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (personal computer, PC), a Television (TV), or the like, and the embodiments of the present application are not limited in particular.

A PCB defect detection system in an embodiment of the application is a device with an operating system. The operating system may be an Android operating system, an ios operating system, or other possible operating systems, and the embodiment of the present application is not limited specifically.

The PCB defect detection system provided in the embodiment of the present application can implement each process implemented by a PCB defect detection method in the method embodiments of fig. 1 to 2, and in order to avoid repetition, a description is omitted here.

According to the PCB defect detection system, the first acquisition module can establish a YOLOv model and effectively acquire and divide a PCB data set into a training set, a verification set and a test set, which ensures that sufficient and diversified data support is provided in the model training process, so that the generalization capability of the model is improved, the first processing module carries out preprocessing on the PCB data set, which generally comprises operations such as image enhancement and the like, so that the recognition accuracy of the model on the PCB defect is improved, the influence of noise on the model training is reduced, the second processing module carries out light-weight processing on the YOLOv model, the calculation complexity and the storage requirement of the model can be remarkably reduced, the model is more suitable for running on hardware with limited resources, the practical applicability of the system is improved, and the first improvement module can further improve the detection accuracy and speed of the model by optimizing a space-to-depth convolution SPD_conv module into a DSPD module and configuring the model YOLOv after light-weight. The DSPD module can enhance the recognition capability of the model to the PCB defects through more effective feature extraction and conversion, the second improvement module configures a preset bidirectional feature pyramid network BiFPN into a neck network of the DSPD_ YOLOv model, which is favorable for the model to better fuse multi-scale features and improve the detection capability of the PCB defects with different sizes and shapes, the first training module trains the improved DSPD_ YOLOv model by using a preprocessed PCB data set to ensure that the model can learn the effective features of the PCB defects, and the first evaluation module evaluates training results, can quantify the performance of the model, including indexes such as detection precision, recall rate and the like, and provides data support for further optimization of the model.

Optionally, in order to avoid repeating the embodiment of the present application, a readable storage medium is further provided, where the readable storage medium stores a program or an instruction, and the program or the instruction is executed by the processor to implement each process of the embodiment of the method for detecting a PCB defect, and the same technical effects can be achieved.

The processor is a processor in the electronic device in the above embodiment. Readable storage media include computer readable storage media such as Read-Only Memory (ROM), random access Memory (Random Access Memory, RAM), magnetic or optical disks, and the like.

It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element. Furthermore, it should be noted that the scope of the methods and apparatus in the embodiments of the present application is not limited to performing the functions in the order shown or discussed, but may also include performing the functions in a substantially simultaneous manner or in an opposite order depending on the functions involved, e.g., the described methods may be performed in an order different from that described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

From the above description of the embodiments, it will be clear to those skilled in the art that the above-described embodiment method may be implemented by means of software plus a necessary general hardware platform, but of course may also be implemented by means of hardware, but in many cases the former is a preferred embodiment. Based on such understanding, the technical solution of the present application may be embodied essentially or in a part contributing to the prior art in the form of a software product stored in a storage medium (e.g. ROM/RAM, magnetic disk, optical disk) comprising several instructions for causing a terminal (which may be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to perform the method of the embodiments of the present application.

The embodiments of the present application have been described above with reference to the accompanying drawings, but the present application is not limited to the above-described embodiments, which are merely illustrative and not restrictive, and many forms may be made by those having ordinary skill in the art without departing from the spirit of the present application and the scope of the claims, which are to be protected by the present application.

Claims

1. A method for detecting defects in a PCB, the method comprising:

Inputting the PCB dataset into the DSPD_ YOLOv10 model for training, and evaluating a training result;

detecting a sample to be detected according to the trained DSPD_ YOLOv10 model to obtain a detection result;

The space-to-depth convolution SPD_conv module comprises a space-to-depth layer and a non-strided convolution layer, wherein the space-to-depth convolution SPD_conv module is optimized to be a DSPD module, and the DSPD module is configured into the YOLOv model, specifically:

Maintaining the step length of the first convolution of the backbone network in the YOLOv model unchanged, and configuring the DSPD module after the rest convolution layers;

the DSPD module processes the image as follows,

Inputting a feature map for slicing processing to obtain a plurality of feature subgraphs, wherein the specific formula is as follows:

;

2. The method for detecting defects of a PCB according to claim 1, wherein the steps of obtaining a PCB data set, performing image preprocessing and defect labeling on the PCB data set are as follows:

3. The method for detecting a PCB defect according to claim 1, wherein the process of performing the light-weight processing on the YOLOv model is as follows:

4. The method for detecting the defects of the PCB according to claim 1, wherein the bi-directional feature pyramid network BiFPN is configured into the neck network of the dspd_ YOLOv10 model, specifically:

5. The method of claim 1, wherein in evaluating the training results, the evaluation index includes recall, accuracy, average accuracy mean, and frame rate;

the recall rate is calculated by the following formula:

;

the accuracy is calculated by the following formula:

;

The average accuracy is calculated by the following formula:

;

The average accuracy mean value is calculated by the following formula:

;

the frame rate is calculated by the following formula:

;

6. A PCB defect detection system capable of implementing a PCB defect detection method according to any one of claims 1-5, the system comprising:

The first improvement module optimizes a space-to-depth convolution SPD_conv module to be a DSPD module, and configures the DSPD module into the YOLOv model after light weight processing to obtain a DSPD_ YOLOv model;

7. An electronic device comprising a processor, a memory and a program or instruction stored on the memory and executable on the processor, which program or instruction when executed by the processor implements the steps of a method of PCB defect detection as claimed in claims 1-5.

8. A readable storage medium, characterized in that the readable storage medium has stored thereon a program or instructions which, when executed by a processor, implement the steps of a PCB defect detection method as claimed in claims 1-5.