CN113465529B - Chip strain measurement method and system based on visual identification - Google Patents
Chip strain measurement method and system based on visual identification Download PDFInfo
- Publication number
- CN113465529B CN113465529B CN202111010025.6A CN202111010025A CN113465529B CN 113465529 B CN113465529 B CN 113465529B CN 202111010025 A CN202111010025 A CN 202111010025A CN 113465529 B CN113465529 B CN 113465529B
- Authority
- CN
- China
- Prior art keywords
- chip
- crack
- strain
- image
- cracks
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G01B11/00—Measuring arrangements characterised by the use of optical techniques
- G01B11/16—Measuring arrangements characterised by the use of optical techniques for measuring the deformation in a solid, e.g. optical strain gauge
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30108—Industrial image inspection
- G06T2207/30148—Semiconductor; IC; Wafer
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Quality & Reliability (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Theoretical Computer Science (AREA)
- Image Analysis (AREA)
Abstract
本发明涉及基于视觉识别的芯片应变测量方法及系统，其方法包括：获取待检测芯片在不同放大倍数和不同视角的多张图像；利用多尺度目标识别网络识别每张图像中的芯片、引线、焊点和键合点及其形状、大小、坐标和裂纹类型；计算每张图像中相邻焊点、芯片和引线之间的间距，并根据每段间距和对应的裂纹类型确定待检测芯片发生应变的多个参考点；根据所述多个参考点，确定并计算待检测芯片的应变大小。本发明利用芯片的多尺度图像，从多视角和多分辨率条件下，并结合裂纹类型对芯片的应变进行计算，提高了芯片应变的识别准确率和芯片良品率，降低生产成本。
The invention relates to a chip strain measurement method and system based on visual recognition. The method includes: acquiring multiple images of a chip to be detected at different magnifications and different viewing angles; using a multi-scale target recognition network to identify chips, leads, Solder joints and bonding points and their shapes, sizes, coordinates and crack types; calculate the spacing between adjacent solder joints, chips and leads in each image, and determine the strain on the chip to be tested based on the spacing and the corresponding crack type of each segment multiple reference points; according to the multiple reference points, determine and calculate the strain size of the chip to be detected. The invention utilizes the multi-scale image of the chip, calculates the strain of the chip under the conditions of multiple viewing angles and multiple resolutions, and combines the types of cracks, improves the identification accuracy of the chip strain and the chip yield, and reduces the production cost.
Description
Technical Field
The invention belongs to the field of visual identification and chip packaging measurement, and particularly relates to a chip strain measurement method and system based on visual identification.
Background
With the development of intellectualization, high performance and high complexity of electronic products, corresponding chips and sensors are developed towards the direction of light weight, multiple functions and high integration density (NEMS), and microelectronic packaging technology becomes the mainstream trend of the current electronic packaging technology due to the characteristics of high density and high performance. Microelectronic packaging refers to a process of fixedly attaching chips and other components to a substrate and wrapping lead-out pins or terminals with a housing to form an integral structure. The protective action of the protective circuit on the internal integrated circuit is summarized mainly by the following points: firstly, the device plays the roles of mechanical support, environmental protection and stress relaxation, and reduces the influence of external environment and stress on an internal chip; secondly, the functions of signal transmission and power distribution are achieved; and thirdly, a heat dissipation channel is provided to lead out heat generated by devices in the package.
The conventional packaging defects of the chip can be usually identified through CCD image detection, but some chip fine defects (fracture between a welding spot and a metal layer interface, failure of a welding spot micropore structure and the like) caused by external stress (strain) are not easy to identify, so that the yield of the chip is reduced, and even the production cost is improved.
Disclosure of Invention
In order to improve the identification accuracy and the yield of chip strain and reduce the production cost, the invention provides a chip strain measurement method based on visual identification in a first aspect, which comprises the following steps: acquiring a plurality of images of a chip to be detected at different magnification factors and different visual angles; recognizing chips, leads, welding spots and bonding points in each image and the shapes, sizes, coordinates and crack types of the chips, the leads, the welding spots and the bonding points by using a multi-scale target recognition network; calculating the distance between adjacent welding spots, chips and leads in each image, and determining a plurality of reference points of the chips to be detected for generating strain according to the distance and the corresponding crack type; and determining and calculating the strain of the chip to be detected according to the plurality of reference points.
In some embodiments of the present invention, the multi-scale target recognition network comprises a first multi-scale target recognition network and a second multi-scale target recognition network, the first multi-scale target recognition network is used for recognizing chips, leads, welding points and bonding points in images and shapes, sizes and coordinates thereof; and the second multi-scale target identification network is used for identifying the crack types in the chip, the lead, the welding spot and the bonding spot in the image.
Further, the second multi-scale target recognition network is trained by the following steps: acquiring a plurality of crack images of different chips, leads, welding spots and bonding points; labeling each crack image according to the crack type in the crack image to obtain a label of each crack image; constructing a crack image data set according to the multiple crack images and the labels corresponding to the multiple crack images; and training the second multi-scale target recognition network by using the crack image data set until the error of the second multi-scale target recognition network is stable and is lower than a threshold value.
In some embodiments of the present invention, the calculating the distances between adjacent solder joints, chips and leads in each image, and determining a plurality of reference points where the chip to be detected is strained according to each distance and the corresponding crack type includes the following steps:
if the identified crack is a smooth crack, taking a transverse or longitudinal starting point and an end point of the crack as reference points; and if the identified cracks are fractal cracks, taking the tips of the main cracks and the tips of the branch cracks as reference points according to the singularity of the strain.
Further, the step of determining and calculating the strain of the chip to be detected according to the plurality of reference points includes the following steps:
if the crack on the chip to be detected is smooth, calculating the strain of the chip to be detected according to the relative distance between the transverse or longitudinal starting point and the end point of the crack in the image with the minimum resolution; and if the crack on the chip to be detected is a fractal crack, calculating the strain of the chip to be detected according to the reference point and the HRR singularity of the crack.
In the above embodiment, the acquiring a plurality of images of the chip to be detected with different magnifications and different viewing angles includes: fusing images with the same resolution and different viewing angles in the acquired multiple images; and fusing images with the same visual angle but different resolutions in the acquired multiple images.
In a second aspect of the present invention, a chip strain measurement system based on visual recognition is provided, which includes: the acquisition module is used for acquiring a plurality of images of the chip to be detected at different magnification factors and different visual angles; the identification module is used for identifying the chip, the lead, the welding spot and the bonding point in each image and the shape, the size, the coordinate and the crack type of the chip, the lead, the welding spot and the bonding point by using the multi-scale target identification network; the determining module is used for calculating the distance between adjacent welding spots, chips and leads in each image and determining a plurality of reference points of the chips to be detected for generating strain according to the distance and the corresponding crack type; and the calculation module is used for determining and calculating the strain of the chip to be detected according to the plurality of reference points.
Further, the determination module includes a first determination unit for identifying a smooth crack in each image and taking a start point and an end point in a transverse direction or a longitudinal direction of the crack as reference points;
and the second determining unit is used for identifying the fractal cracks in each image and taking the tips of the main cracks and the tips of the branch cracks as reference points according to the singularity of the strain.
In a third aspect of the present invention, there is provided an electronic device comprising: one or more processors; storage means for storing one or more programs which, when executed by the one or more processors, cause the one or more processors to carry out the method provided by the first aspect of the invention.
In a fourth aspect of the invention, a computer-readable medium is provided, on which a computer program is stored, wherein the computer program, when executed by a processor, implements the method provided in the first aspect of the invention.
The invention has the beneficial effects that:
1. the multi-scale target identification network can be used for quickly identifying related chips, leads, welding spots, bonding points and other key targets in chip images under different resolutions, and taking the key targets as reference points, replacing the reference points of a single target with the reference points of a plurality of targets in the prior art, thereby improving the detection accuracy;
2. different calculation methods are determined according to different scales and different crack types, so that the accuracy of strain calculation is further improved;
3. a plurality of images with different magnification factors and different visual angles can improve the definition of the images through image fusion and image enhancement technologies, and reduce the adverse effect on measurement caused by measurement conditions or measurement environments.
Drawings
FIG. 1 is a schematic flow chart of a method for measuring strain of a chip based on visual recognition according to some embodiments of the present invention;
FIG. 2a is a schematic image of a chip at a conventional viewing angle;
FIG. 2b is a schematic view of a solder joint under microscopic viewing angles;
FIG. 2c is a schematic view of a solder joint and a solder wire at a microscopic viewing angle;
FIG. 2d is a schematic image of a chip undergoing cracking;
FIG. 3 is a schematic structural diagram of a chip strain measurement system based on visual identification;
FIG. 4 is a basic block diagram of an electronic device in some embodiments of the invention.
Detailed Description
The principles and features of this invention are described below in conjunction with the following drawings, which are set forth by way of illustration only and are not intended to limit the scope of the invention.
Referring to fig. 1, 2a to 2d, in a first aspect of the present invention, a chip strain measurement method based on visual recognition is provided, including the following steps: s100, acquiring a plurality of images of a chip to be detected at different magnification factors and different visual angles; s200, identifying chips, leads, welding spots and bonding points in each image and the shapes, sizes, coordinates and crack types of the chips, the leads, the welding spots and the bonding points by using a multi-scale target identification network; s300, calculating the distance between adjacent welding spots, chips and leads in each image, and determining a plurality of reference points of strain of the chip to be detected according to the distance and the corresponding crack type; and S400, determining and calculating the strain of the chip to be detected according to the plurality of reference points.
It is understood that the different viewing angles include, but are not limited to, images of the chip taken under a CCD line camera, microscopic viewing angle, ultrasonic scanning, electron scanning microscope at different viewing angles; different resolutions include, but are not limited to, regular magnification (1X), other high magnification images; the cracks caused by the fracture of the welding spot comprise the fracture of a sphere inside the welding spot, the fracture of a micropore structure and the fracture of a PCB reinforcing layer, wherein the fracture of IMC between the welding spot and a metal layer interface is the most, the fracture of the inside of the welding spot is the second, and the fracture of a micropore structure welding disc and the PCB reinforcing layer is the least.
In some embodiments of the present invention, considering that there are multiple targets in each picture, in order to improve the recognition efficiency and accuracy, the targets to be recognized are divided: one category includes chips, leads, pads and bond sites and the other category includes crack types. The multi-scale target recognition network comprises a first multi-scale target recognition network and a second multi-scale target recognition network, wherein the first multi-scale target recognition network is used for recognizing chips, leads, welding spots and bonding points in images as well as shapes, sizes and coordinates of the chips, the leads, the welding spots and the bonding points; and the second multi-scale target identification network is used for identifying the crack types in the chip, the lead, the welding spot and the bonding spot in the image. Because the first multi-scale target recognition network and the second multi-scale target recognition network can adopt the same training set (from the same chip image), the full connection layers of the first multi-scale target recognition network and the second multi-scale target recognition network are connected with each other, and the sharing of characteristic information between the first multi-scale target recognition network and the second multi-scale target recognition network is realized through model fusion means such as transfer learning or federal learning, so that the accuracy and the robustness of respective recognition are improved.
Further, the second multi-scale target recognition network is trained by the following steps: acquiring a plurality of crack images of different chips, leads, welding spots and bonding points; labeling each crack image according to the crack type in the crack image to obtain a label of each crack image; constructing a crack image data set according to the multiple crack images and the labels corresponding to the multiple crack images; and training the second multi-scale target recognition network by using the crack image data set until the error of the second multi-scale target recognition network is stable and is lower than a threshold value.
Optionally, the first multi-scale Object recognition network or the second multi-scale Object recognition network is usually implemented by a multi-scale Object Detection (MOD) model, which includes, but is not limited to, transforming an input image into a Feature map, and performing fusion of features of different scales on the Feature map by using different methods, so as to implement multi-scale Detection, for example, one or more of FPN (Feature Pyramid network), ssd (single Shot multi box detector), fast RCNN, and YOLO series networks.
In step S300 of some embodiments of the present invention, the calculating the distances between adjacent pads, chips and leads in each image, and determining a plurality of reference points where the chip to be detected is strained according to each distance and the corresponding crack type includes the following steps:
if the identified crack is a smooth crack, taking a transverse or longitudinal starting point and an end point of the crack as reference points;
and if the identified cracks are fractal cracks, taking the tips of the main cracks and the tips of the branch cracks as reference points according to the singularity of the strain.
The crack types include longitudinal cracks, transverse cracks, web cracks and block cracks, wherein the web cracks and the block cracks can be regarded as the combination of the longitudinal cracks and the transverse cracks, for convenience of calculation, the cracks are divided into smooth cracks and fractal cracks in the invention, and no cracks are regarded as smooth cracks. When the identified cracks are not cracks, calculating the strain of the chip to be detected according to the relative displacement of the reference points before and after welding or packaging; or, the strain is ignored, and the chip to be tested is regarded as normal.
In step S400 of some embodiments of the present invention, the determining and calculating the magnitude of the strain of the chip to be detected according to the plurality of reference points includes the following steps:
if the crack on the chip to be detected is smooth, calculating the strain of the chip to be detected according to the relative distance between the starting point and the end point of the crack in the transverse direction or the longitudinal direction in the image with the minimum resolution, namely according to the strain relation (RO model for short) in ramberg-Osgood materials:
here, theεAndσrespectively, the strain and the stress are indicated,ε 0andσ 0is a characteristic constant associated with the material,αis the coefficient of hardening of the resin,nis the hardening index;
and if the crack on the chip to be detected is a fractal crack, calculating the strain of the chip to be detected according to the reference point and HRR (Hutchinson, Rice and Rosegren) singularity of the crack.
Specifically, on the basis of the above R-O model, the HRR singularity is expressed as:
whereinε、σAndurespectively represent strain, stress, displacement,ε 0andσ 0is a property of the material in that,αis the coefficient of hardening of the resin,nis the index of the hardening of the steel,randθare both the polar diameter and polar angle relative to the crack tip polar coordinate system,iandjthe abscissa and the ordinate of the stress point are represented;I n 、
、
、
is a tabulated function of the corresponding parameters. It can be seen that the singularities of both strain and stress are equalrIs related to the index (c).
Preferably, the crack tip stress field is expressed as:
whereinσIn order to be the stress,iandjthe abscissa and the ordinate of the stress point are represented;randθthe polar radius and the polar angle are relative to a polar coordinate system of the crack tip;Kas a factor of the intensity of the stress,f ij （θ) As a function of the angular distribution. In which the stress intensity factorKCan be given by:
whereinFThe shape parameters related to geometrical factors such as crack type, size and position are as follows:σnominal stress (stress at crack location calculated as no crack):athe crack size (crack length or depth). Factor of stress intensityKThe breaking criterion is:K=K IC ，K IC is a material constant used for characterizing the fracture toughness of the material.
In chip packaging, because defect measurement or detection of a solder joint, a lead, and a bonding point is easier than that of a chip itself and influence caused by the defect of the chip is larger, the strain of the chip is usually detected or identified by using the defect of the chip or a crack on the chip.
In the above embodiment, in order to reduce image noise in the crack and enhance the sharpness of the image, two or more images are synthesized into a new image. The fusion result can utilize the correlation of two (or more) images in space-time and information complementarity, and the image obtained after fusion has more comprehensive and clear description for the scene, so that the method is more favorable for human eye identification and automatic machine detection, and therefore, the acquisition of a plurality of images with different resolutions and different visual angles of the chip to be detected comprises the following steps: fusing images with the same resolution and different viewing angles in the acquired multiple images; and fusing images with the same visual angle but different resolutions in the acquired multiple images.
The fusion method comprises signal level fusion, data level fusion, feature level fusion and decision level fusion; preferably, the invention adopts pixel-level fusion in the input image and decision-level fusion in the output layer, wherein the pixel-level fusion adopts an image fusion method based on pyramid transformation to fuse the characteristics of texture details, edges and the like in different images;
optionally, the image denoising method includes: the method comprises a field average method, a median filtering method and the like, image enhancement comprises low-pass filtering, high-pass filtering, band-stop filtering and the like, and decision-level fusion adopts a decision tree algorithm to fuse the features in each image.
Referring to fig. 3, in a second aspect of the present invention, there is provided a chip strain measurement system 1 based on visual recognition, comprising: the acquisition module 11 is used for acquiring a plurality of images of the chip to be detected at different magnification factors and different viewing angles; the identification module 12 is used for identifying the chip, the lead, the welding spot and the bonding point in each image and the shape, the size, the coordinate and the crack type of the chip, the lead, the welding spot and the bonding point by using the multi-scale target identification network; the determining module 13 is used for calculating the distances among adjacent welding spots, chips and leads in each image, and determining a plurality of reference points of the chips to be detected, which are subjected to strain, according to each distance and the corresponding crack type; and the calculation module 14 is used for determining and calculating the strain of the chip to be detected according to the plurality of reference points.
Further, the determining module 13 comprises a first determining unit and a second determining unit,
the first determining unit is used for identifying smooth cracks in each image and taking a starting point and an end point of the cracks in the transverse direction or the longitudinal direction as reference points;
and the second determining unit is used for identifying the fractal cracks in each image and taking the tips of the main cracks and the tips of the branch cracks as reference points according to the singularity of the strain.
Referring to fig. 4, in a third aspect of the present invention, there is provided an electronic apparatus comprising: one or more processors; a storage device, configured to store one or more programs, which when executed by the one or more processors, cause the one or more processors to implement the chip strain measurement method based on visual identification provided by the first aspect of the present invention.
The electronic device 500 may include a processing means (e.g., central processing unit, graphics processor, etc.) 501 that may perform various appropriate actions and processes in accordance with a program stored in a Read Only Memory (ROM)502 or a program loaded from a storage means 508 into a Random Access Memory (RAM) 503. In the RAM 503, various programs and data necessary for the operation of the electronic apparatus 500 are also stored. The processing device 501, the ROM 502, and the RAM 503 are connected to each other through a bus 504. An input/output (I/O) interface 505 is also connected to bus 504.
The following devices may be connected to the I/O interface 505 in general: input devices 506 including, for example, a touch screen, touch pad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; output devices 507 including, for example, a Liquid Crystal Display (LCD), speakers, vibrators, and the like; a storage device 508 including, for example, a hard disk; and a communication device 509. The communication means 509 may allow the electronic device 500 to communicate with other devices wirelessly or by wire to exchange data. While fig. 4 illustrates an electronic device 500 having various means, it is to be understood that not all illustrated means are required to be implemented or provided. More or fewer devices may alternatively be implemented or provided. Each block shown in the figures may represent one device or a plurality of devices as desired.
In particular, according to an embodiment of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network via the communication means 509, or installed from the storage means 508, or installed from the ROM 502. The computer program, when executed by the processing device 501, performs the above-described functions defined in the methods of embodiments of the present disclosure. It should be noted that the computer readable medium described in the embodiments of the present disclosure may be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In embodiments of the disclosure, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In embodiments of the present disclosure, however, a computer readable signal medium may comprise a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: electrical wires, optical cables, RF (radio frequency), etc., or any suitable combination of the foregoing.
The computer readable medium may be embodied in the electronic device; or may exist separately without being assembled into the electronic device. The computer readable medium carries one or more computer programs which, when executed by the electronic device, cause the electronic device to:
computer program code for carrying out operations for embodiments of the present disclosure may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + +, Python, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like that fall within the spirit and principle of the present invention are intended to be included therein.
Claims (9)
1. A chip strain measurement method based on visual identification is characterized by comprising the following steps:
acquiring a plurality of images of a chip to be detected at different magnification factors and different visual angles;
recognizing chips, leads, welding spots and bonding points in each image, and the shapes, sizes, coordinates and crack types of the chips, the leads, the welding spots and the bonding points by using a multi-scale target recognition network;
calculating the distance between adjacent welding spots, chips and leads in each image, and determining a plurality of reference points of the strain of the chip to be detected according to the distance and the corresponding crack type; if the identified cracks are fractal cracks, taking the tips of the main cracks and the tips of all the branch cracks as reference points according to the singularity of the strain;
and determining and calculating the strain of the chip to be detected according to the plurality of reference points.
2. The chip strain measurement method based on visual identification according to claim 1, wherein the multi-scale target identification network comprises a first multi-scale target identification network and a second multi-scale target identification network, the first multi-scale target identification network is used for identifying chips, leads, welding points and bonding points in the image, and the shapes, sizes and coordinates thereof;
and the second multi-scale target identification network is used for identifying the crack types in the chip, the lead, the welding spot and the bonding spot in the image.
3. The chip strain measurement method based on visual identification according to claim 2, wherein the second multi-scale target identification network is trained by the following steps:
acquiring a plurality of crack images of different chips, leads, welding spots and bonding points;
labeling each crack image according to the crack type in the crack image to obtain a label of each crack image;
constructing a crack image data set according to the multiple crack images and the labels corresponding to the multiple crack images;
and training the second multi-scale target recognition network by using the crack image data set until the error of the second multi-scale target recognition network is stable and is lower than a threshold value.
4. The chip strain measurement method based on visual identification according to claim 1, wherein the determining and calculating the strain magnitude of the chip to be detected according to the plurality of reference points comprises the following steps:
if the crack on the chip to be detected is smooth, calculating the strain of the chip to be detected according to the relative distance between the transverse or longitudinal starting point and the end point of the crack in the image with the minimum resolution;
and if the crack on the chip to be detected is a fractal crack, calculating the strain of the chip to be detected according to the reference point and the HRR singularity of the crack.
5. The chip strain measurement method based on visual recognition according to any one of claims 1 to 4, wherein the acquiring of the plurality of images of the chip to be detected at different magnifications and different viewing angles comprises:
fusing images with the same resolution and different viewing angles in the acquired multiple images;
and fusing images with the same visual angle but different resolutions in the acquired multiple images.
6. A chip strain measurement system based on visual identification is characterized by comprising:
the acquisition module is used for acquiring a plurality of images of the chip to be detected at different magnification factors and different visual angles;
the identification module is used for identifying the chip, the lead, the welding spot and the bonding point in each image, and the shape, the size, the coordinate and the crack type of the chip, the lead, the welding spot and the bonding point by using the multi-scale target identification network;
the determining module is used for calculating the distance between adjacent welding spots, chips and leads in each image and determining a plurality of reference points of strain of the chip to be detected according to the distance between each section and the corresponding crack type; if the identified cracks are fractal cracks, taking the tips of the main cracks and the tips of all the branch cracks as reference points according to the singularity of the strain;
and the calculation module is used for determining and calculating the strain of the chip to be detected according to the plurality of reference points.
7. The chip strain measurement system based on visual identification according to claim 6, wherein the determination module comprises a first determination unit and a second determination unit,
the first determining unit is used for identifying smooth cracks in each image and taking a starting point and an end point of the cracks in the transverse direction or the longitudinal direction as reference points;
and the second determining unit is used for identifying the fractal cracks in each image and taking the tips of the main cracks and the tips of the branch cracks as reference points according to the singularity of the strain.
8. An electronic device, comprising: one or more processors; a storage device to store one or more programs that, when executed by the one or more processors, cause the one or more processors to implement the visual identification-based chip strain measurement method of any one of claims 1-5.
9. A computer-readable medium, on which a computer program is stored, wherein the computer program, when being executed by a processor, implements the chip strain measurement method based on visual identification according to any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111010025.6A CN113465529B (en) | 2021-08-31 | 2021-08-31 | Chip strain measurement method and system based on visual identification |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111010025.6A CN113465529B (en) | 2021-08-31 | 2021-08-31 | Chip strain measurement method and system based on visual identification |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113465529A CN113465529A (en) | 2021-10-01 |
| CN113465529B true CN113465529B (en) | 2021-11-23 |
Family
ID=77867116
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111010025.6A Active CN113465529B (en) | 2021-08-31 | 2021-08-31 | Chip strain measurement method and system based on visual identification |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113465529B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114531788B (en) * | 2022-02-21 | 2023-03-24 | 深圳眼千里科技有限公司 | Surface mounting method, system, terminal and storage medium for camera module chip |
| CN115355839A (en) * | 2022-07-19 | 2022-11-18 | 中国电子科技集团公司第十一研究所 | An infrared detector focal plane detection system and method |
| CN116577195A (en) * | 2023-04-28 | 2023-08-11 | 广东科信电子有限公司 | A welding wire detection control method, device, equipment and readable storage medium |
| CN116609359A (en) * | 2023-06-06 | 2023-08-18 | 广东科信电子有限公司 | Chip bonding wire detection method and device, electronic equipment and storage medium |
Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5642158A (en) * | 1994-05-02 | 1997-06-24 | Cognex Corporation | Method and apparatus to detect capillary indentations |
| JP2000329536A (en) * | 1999-05-19 | 2000-11-30 | Nec Corp | Apparatus and method for visual inspection |
| JP2001013202A (en) * | 1999-07-02 | 2001-01-19 | Nec Corp | IC device inspection equipment |
| JP2009015735A (en) * | 2007-07-07 | 2009-01-22 | Keyence Corp | Edge connection method for edge image, contour extraction method, contour extraction device, contour extraction program, and computer readable recording medium |
| CN108987296A (en) * | 2018-08-14 | 2018-12-11 | 长江存储科技有限责任公司 | Wafer elastic strain measuring device, measurement method and wafer bonding method |
| CN110689011A (en) * | 2019-09-29 | 2020-01-14 | 河北工业大学 | Solar cell panel defect detection method of multi-scale combined convolution neural network |
| CN111542855A (en) * | 2017-12-31 | 2020-08-14 | Asml荷兰有限公司 | Method and system for defect inspection and re-inspection |
| CN111598856A (en) * | 2020-05-08 | 2020-08-28 | 浙江工商大学 | Method and system for automatic detection of chip surface defects based on defect-oriented multi-point localization neural network |
| CN111928771A (en) * | 2020-08-06 | 2020-11-13 | 广东高标电子科技有限公司 | Strain detection device and strain detection method |
| CN111968119A (en) * | 2020-10-21 | 2020-11-20 | 季华实验室 | Image processing method, device, equipment and medium based on semiconductor defect detection |
| CN112697063A (en) * | 2021-03-23 | 2021-04-23 | 四川省人民医院 | Chip strain measurement method based on microscopic vision |
| CN113218952A (en) * | 2021-04-25 | 2021-08-06 | 华南理工大学 | Method, device, equipment and medium for detecting multi-scale appearance defects of IC packaging carrier plate |
-
2021
- 2021-08-31 CN CN202111010025.6A patent/CN113465529B/en active Active
Patent Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5642158A (en) * | 1994-05-02 | 1997-06-24 | Cognex Corporation | Method and apparatus to detect capillary indentations |
| JP2000329536A (en) * | 1999-05-19 | 2000-11-30 | Nec Corp | Apparatus and method for visual inspection |
| JP2001013202A (en) * | 1999-07-02 | 2001-01-19 | Nec Corp | IC device inspection equipment |
| JP2009015735A (en) * | 2007-07-07 | 2009-01-22 | Keyence Corp | Edge connection method for edge image, contour extraction method, contour extraction device, contour extraction program, and computer readable recording medium |
| CN111542855A (en) * | 2017-12-31 | 2020-08-14 | Asml荷兰有限公司 | Method and system for defect inspection and re-inspection |
| CN108987296A (en) * | 2018-08-14 | 2018-12-11 | 长江存储科技有限责任公司 | Wafer elastic strain measuring device, measurement method and wafer bonding method |
| CN110689011A (en) * | 2019-09-29 | 2020-01-14 | 河北工业大学 | Solar cell panel defect detection method of multi-scale combined convolution neural network |
| CN111598856A (en) * | 2020-05-08 | 2020-08-28 | 浙江工商大学 | Method and system for automatic detection of chip surface defects based on defect-oriented multi-point localization neural network |
| CN111928771A (en) * | 2020-08-06 | 2020-11-13 | 广东高标电子科技有限公司 | Strain detection device and strain detection method |
| CN111968119A (en) * | 2020-10-21 | 2020-11-20 | 季华实验室 | Image processing method, device, equipment and medium based on semiconductor defect detection |
| CN112697063A (en) * | 2021-03-23 | 2021-04-23 | 四川省人民医院 | Chip strain measurement method based on microscopic vision |
| CN113218952A (en) * | 2021-04-25 | 2021-08-06 | 华南理工大学 | Method, device, equipment and medium for detecting multi-scale appearance defects of IC packaging carrier plate |
Non-Patent Citations (3)
| Title |
|---|
| Automatic defect classification for semiconductor manufacturing;Paul B. Chou 等;《Machine Vision and Applications》;19971231;第201-213页 * |
| 基于神经网络的IC芯片图形缺陷检测技术研究;魏鹏;《电子工业专业设备》;20210630;第35-41页 * |
| 电子器件封装中引线键合质量的检测方法;齐翊 等;《电子工艺技术》;20160131;第37卷(第1期);第4-10页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113465529A (en) | 2021-10-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113465529B (en) | Chip strain measurement method and system based on visual identification | |
| CN114550021B (en) | Surface defect detection method and device based on feature fusion | |
| CN110763706B (en) | System and method for analyzing crystal defects | |
| CN111260615A (en) | Detection method for apparent damage of UAV bridge based on fusion of laser and machine vision | |
| CN112597788B (en) | Target measuring method, target measuring device, electronic apparatus, and computer-readable medium | |
| JP2016090548A (en) | Crack information collection method and crack information collection program | |
| CN110222641A (en) | The method and apparatus of image for identification | |
| TW201320231A (en) | Method and system for positioning round holes of motherboard | |
| CN113506327A (en) | A kind of elongation at break calculation method, device and electronic equipment | |
| CN110163900B (en) | Method and apparatus for adjusting point cloud data | |
| CN115861403A (en) | Non-contact object volume measurement method and device, electronic equipment and medium | |
| CN111598097B (en) | Instrument position and reading identification method and system based on robot vision | |
| WO2024139756A1 (en) | Method and apparatus for interpolation of feature map, device, and storage medium | |
| CN114565721B (en) | Object determination method, device, apparatus, storage medium, and program product | |
| CN109902695B (en) | A Line Feature Correction and Purification Method Oriented to Image-to-Line Feature Matching | |
| CN115932780A (en) | Feature extraction method, device, equipment and storage medium of lidar point cloud | |
| CN115153632A (en) | An ultrasonic imaging positioning system, method, device, equipment and storage medium | |
| CN111914784B (en) | Method and device for detecting intrusion of trackside obstacle in real time and electronic equipment | |
| CN114596336B (en) | Complex scene laser tracker target ball center positioning method and device based on YOLOv3 | |
| CN113112551B (en) | Methods, devices, roadside equipment and cloud control platform for determining camera parameters | |
| JPWO2008032375A1 (en) | Image correction apparatus and method, and computer program | |
| CN113936203A (en) | Oil tank layout information generation method, oil tank layout information generation device, electronic apparatus, and medium | |
| CN120070458B (en) | Vibration measurement method, device, equipment and medium for cable structure | |
| CN112991179A (en) | Method, apparatus, device and storage medium for outputting information | |
| CN113542800B (en) | Video picture scaling method, device and terminal equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CP03 | Change of name, title or address | ||
| CP03 | Change of name, title or address |
Address after: No.2, 3 / F, building 12, zone B, Gaoke medical equipment Park, 818 Gaoxin Avenue, Donghu New Technology Development Zone, Wuhan City, Hubei Province (Wuhan area of free trade zone) Patentee after: Wuhan Fenwei Electronic Co., Ltd. Country or region after: China Address before: No.2, 3 / F, building 12, zone B, Gaoke medical equipment Park, 818 Gaoxin Avenue, Donghu New Technology Development Zone, Wuhan City, Hubei Province (Wuhan area of free trade zone) Patentee before: WUHAN FINE MEMS Inc. Country or region before: China |