JP2023003627A - Computer device, method, and computer program - Google Patents

Abstract

To increase accuracy of detection of a mismatch between two images.SOLUTION: A computer device receives a first image and a second image including at least a pattern arrangement from an image-capturing device having an optical sensor and calculates a set of first brightness values and a set of second brightness values for the pattern arrangement from the first image and the second image, respectively. The computer device also calculates a set of difference values between the set of first brightness values and the set of second brightness values and compares the set of difference values with a predetermined threshold value to thereby detect a mismatch between the first image and the second image. The optical sensor has a pixel resolution of 23.6 micrometers/pixel or less.SELECTED DRAWING: Figure 8

Description

The present disclosure relates to an image inspection method and an image inspection apparatus. In particular, the present disclosure relates to an image inspection method and apparatus for detecting discrepancies resulting from differences in imaging conditions when comparing two images.

Conventionally, in order to safely provide consumers with articles such as food and medicine, there is a need to control individual articles during their manufacturing process. In order to manage individual items, it is necessary to individually identify the items during the manufacturing process. An identification number is usually attached to an article or its packaging container in order to identify the article. However, even if an identification number is attached to a container in which a certain number of items are packaged, the identification number may not be attached to each individual item.

For example, medicines that can be harmful to humans depending on how they are used may need to be strictly controlled in units of individual tablets so that they are not taken outside. Commercially available drugs are often sold in a form in which a plurality of tablets are stored in a packaging container. A plurality of tablets stored in a packaging container are each packaged using a PTP (press through pack) labeled with the same design, and each packaging label is not attached with an identification number.

In recent years, in order to identify individual goods, technology to identify goods by accurately distinguishing minute color shading on the surface of goods and/or packaging labels (for example, printing unevenness or fading of packaging labels). is being developed. The identification technology described above can recognize differences in specific areas that are indistinguishable to the human eye, even if the packaging labels have the same design, and can identify individual articles.

In order to individually identify articles (medicine tablets) in the actual manufacturing process, it is necessary to previously generate a reference image obtained by photographing each tablet using an imaging device such as a camera. When identifying individual tablets, a comparative image obtained by photographing each tablet is generated, the comparative image and the reference image are compared, and individual tablets are identified by determining whether they match. Identify.

JP 2016-018340 A

In the identification technique described above, the luminance values of both the reference image and the comparative image are calculated for each pixel, and it is determined whether the calculated luminance values are within a certain range or not, thereby determining that the two match each other. In a case where a plurality of tablets are each packaged with packaging labels of the same design, the brightness value is calculated from the color shading of the packaging labels (which cannot be distinguished by human eyes). Therefore, it is calculated that the position and/or tilt of the imaging device do not match when the tablet is photographed to generate the comparison image and when the tablet is photographed to generate the reference image. The brightness values are also different, and as a result, even the same tablet may be judged to be inconsistent.

Patent Literature 1 discloses a technique of obtaining an image of an object, extracting the edges of the image, and calculating the edge blur amount based on the luminance difference of pixels in a predetermined area including the extracted edges. there is According to the technique disclosed in Patent Literature 1, tilting of the imaging device occurs when the tablet is photographed to generate the comparative image, compared to when the tablet is photographed to generate the reference image. can be detected.

However, the technique disclosed in Japanese Patent Application Laid-Open No. 2002-200010 can detect blurring in edge portions of a generated image, but cannot detect blurring in other portions. Further, the accuracy of identifying an article depends on the pixel resolution of the imaging device, but Patent Document 1 makes no mention of the pixel resolution.

An image inspection apparatus according to one embodiment is a computer device connected to an imaging device having an optical sensor, and receives a first image including at least a pattern array from the imaging device, the first image being captured. calculating a first set of luminance values for the pattern array from the first image generated by photographing the object; receiving a second image including at least the pattern array from the imaging device; The image is generated by photographing the same photographing object as the photographing object, calculating a second set of luminance values for the pattern array from the second image, and combining the first set of luminance values with the second set of luminance values. detecting a mismatch between the first image and the second image by calculating a set of difference values between the set of luminance values and comparing the set of difference values to a predetermined threshold; The optical sensor has a pixel resolution of 23.6 micrometers/pixel or less.

Further, an image inspection apparatus according to another embodiment is a computer device connected to an imaging device having an optical sensor, receiving a first image including at least a pattern array from the imaging device, and obtaining the first image is generated by photographing an object to be photographed, calculates a first coordinate position for the pattern array from the first image, receives a second image including at least the pattern array from the imaging device, and a second is generated by photographing the same photographing object as the photographing object, the second coordinate position for the pattern array is calculated from the first image, and the first coordinate position and the second coordinate position are calculated from the first image. The optical sensor detects a discrepancy between the first image and the second image by calculating a difference value between and comparing the difference value with a predetermined threshold, the optical sensor detecting a 23.6 micron It has a pixel resolution of meters/pixel or less.

According to the image inspection apparatus according to the embodiment, by adopting the optimum image resolution of the imaging device, it is possible to improve the accuracy of detecting a mismatch between two images.

1 is a block diagram showing an example of the configuration of an image inspection system; FIG. FIG. 4 is a block diagram showing an example of a pattern sheet; 6 is a flowchart showing an example of reference image analysis processing; FIG. 10 is a diagram showing an example of an area inside a dot; FIG. 4 is a diagram showing an example of inter-dot regions; FIG. 4 is a diagram showing an example of approximate straight lines of dot rows and dot columns; FIG. 10 is a diagram showing an example of a state in which an object is photographed using a pattern sheet; 9 is a flowchart showing an example of comparative image analysis processing; (a) is a diagram showing an example of pattern arrangement in a reference image, and (b) is a diagram showing an example of pattern arrangement in a comparison image. FIG. 10 is a diagram showing an example of a state in which dot strings are compared; (a) is a diagram showing an example of pattern arrangement in a reference image, and (b) is a diagram showing an example of pattern arrangement in a comparison image. (a) is a diagram showing an example of inter-dot areas between adjacent dots in a reference image, and (b) is a diagram showing an example of inter-dot areas between adjacent dots in a comparison image.

Hereinafter, an image inspection system according to one embodiment will be described in detail with reference to the attached drawings. Imaging inspection systems, for example, in the process of identifying an object based on two images generated by photographing the object, detect discrepancies between the two images resulting from differences in photographing conditions. Hereinafter, in this embodiment, an arbitrary object to be photographed will be referred to as a "photographing target".

For example, in the manufacturing process of pharmaceutical tablets, in order to identify each tablet individually, an image generated in advance of each tablet (reference image (first image)) and an image generated in the actual manufacturing process (comparative image) (second image)) are compared. Since the two images generated by photographing the same object (tablet) match, the imaging device photographs an area at the same relative position within the object when generating the two images. There is a need.

In addition, the imaging device needs to shoot at the same position and inclination with respect to the shooting target when generating two images. If the above conditions are not met, two images generated by photographing the same subject may not match. Furthermore, even if the above-mentioned conditions are met, the two images generated by photographing the same subject may differ due to factors such as differences in the brightness of the photographing location when generating the two images. Images may not match.

The image inspection system, for example, detects the above-mentioned discrepancies in the previous stages of the drug manufacturing process, and therefrom the differences in the imaging conditions. A mismatch in the two images is detected based on, for example, luminance values in the pattern arrays in the images.

Differences in imaging conditions cause two images generated by imaging the same subject to be different. As described above, the difference in the shooting conditions includes the difference in the position of the shooting area, the position and/or the tilt of the imaging device, etc. when shooting the shooting target when generating the two images (the reference image and the comparison image). differences in color, as well as differences in the brightness of the shooting location.

First, the configuration of an image inspection system 100 will be described with reference to FIG. The image inspection system 100 shown in FIG. 1 includes a computer device 1 , an imaging device 2 , an input device 3 and an output device 4 . A computer device 1 is connected to an imaging device 2, an input device 3, and an output device 4, respectively.

The computer device 1 is an information processing device including at least a processor, memory, storage device, and input/output device. Computer device 1 may be a general-purpose computer such as a personal computer. A processor reads a program stored in a storage device into a memory and executes the program. The program is a computer program including computer-readable instructions implementing the image inspection method according to the present embodiment. The input/output device inputs and outputs data to and from an imaging device 2, an input device 3, and an output device 4, which will be described later.

The imaging device 2 is a camera for capturing an image of an object, including one or more CCD image sensors or CMOS image sensors (optical sensors). The imaging device 2 is coupled with the computer device 1 and transmits to the computer device 1 an image generated by photographing an object to be photographed. It is desirable that the imaging device 2 be arranged with more image sensors in order to generate a high-pixel image. The imaging device 2 also includes an array of LEDs (not shown). An array of LEDs illuminates the object. By providing the imaging device 2 with an array in which more LEDs are evenly arranged, the entire object is illuminated with light, and a clearer image can be generated.

In this embodiment, an object is identified by determining whether or not two images generated by photographing the same object match. For example, if the object to be imaged is a tablet (packaging label) and the packaging label is printed with characters or the like by offset printing, the image inspection system 100 according to the present embodiment may reduce the accuracy of the determination described above. . Offset printing is a printing method in which ink adhering to a printing plate is transferred to a transfer roller and transferred to the printing surface via the transfer roller, so that the printing plate and the printing surface do not contact each other. In offset printing, there is a tendency that the characteristic of the transfer roller is a characteristic point as it is, and an error may occur in the determination as described above.

The imaging device 2 adopts the optimum pixel resolution in order to increase the accuracy of the match/mismatch determination described above. The pixel resolution refers to the size of the field of view per pixel of the optical sensor of the imaging device 2 . Although there are variations depending on the combination of camera and lens used, the imaging device 2 adopts the optimum working distance in order to increase the accuracy of the match/mismatch determination described above. The working distance refers to the distance from the tip of the lens of the imaging device 2 to the imaging target.

Input device 3 assists the user in entering information as the user interacts with computing device 1 . The input device 3 is implemented by a mouse, keyboard, and the like. Output device 4 likewise assists in outputting information to the user as the user interacts with computing device 1 . The output device 4 is implemented by a display screen or the like.

In this embodiment, an example in which the computer device 1 and the imaging device 2 are directly coupled is shown, but the present invention is not limited to such an example. The computer device 1 and imaging device 2 may be connected through a network.

Although not shown, the image inspection system 100 may include a support member (holder) for supporting the imaging device 2 . The support member supports the image pickup device 2 so that the image pickup device 2 is focused on the object to be photographed. The image inspection system 100 may also include a jig for fixing the object to be imaged. The jig fixes the object to be photographed so that the focus of the imaging device 2 is aligned with a specific region within the object to be photographed.

Also, the imaging device 2 may include an acceleration sensor and a gyro sensor. The acceleration sensor and the gyro sensor detect the angular velocity when the imaging device 2 is tilted, and transmit it to the computer device 1 as an angular velocity signal.

In addition, image inspection system 100 may include a brightness sensor. The brightness sensor is installed at an arbitrary position in the environment where the imaging device 2 captures an image of an object. The brightness sensor detects the brightness in the environment and sends it to the computer device 1 as a brightness signal.

Next, an example of the pattern sheet PS will be described with reference to FIG. The pattern sheet PS is photographed by the imaging device 2 in order to detect discrepancies between the two images. The pattern sheet PS includes pattern arrays PA. The pattern array PA includes a set of dots formed at regular intervals. The dots in the pattern array PA are desirably densely arranged in order to improve the accuracy of detecting differences in imaging conditions. In this embodiment, the dots are arranged at regular intervals of 1 mm.

In this embodiment, the pattern array PA corresponds to a set of dots, but may be a set of patterns of arbitrary shapes other than dots. For example, the pattern array PA may be a set of patterns having arbitrary shapes such as squares, triangles, or rhombuses.

The imaging device 2 generates a reference image and a comparison image by photographing the pattern sheet PS. For example, when the image inspection system 100 is applied to the manufacturing process of articles (to identify individual articles), the image inspection system 100 generates a reference image in advance by photographing the pattern sheet PS. Then, the image inspection system 100 generates a comparison image by photographing the same pattern sheet PS as that used when generating the reference image, in a stage prior to the actual manufacturing process. A discrepancy between the two images is detected by comparing the luminance values, etc., of the two images generated by photographing the same pattern sheet PS. Details will be described later.

Next, an example of reference image analysis processing executed by the image inspection system 100 will be described with reference to the flowchart shown in FIG. In the reference image analysis process, the imaging device 2 photographs a specific area of the pattern sheet PS to generate a reference image. A specific area of the pattern sheet PS corresponds to an area including at least the pattern array PA. Hereinafter, this specific area will be referred to as an imaging area IA. Also, the pattern sheet PS corresponds to the object O to be photographed.

As described above, the imaging device 2 employs the optimum pixel resolution in order to increase the accuracy of determining match/mismatch between two images. As a result of an experiment in which a specific pixel resolution is applied to the image inspection system 100 according to the present embodiment, the above-described accuracy is improved when the pixel resolution is reduced. Specifically, when the pixel resolution is high, the probability of determining that two images generated by photographing different objects match each other is high. This is because when the pixel resolution is high, the feature points appear at the same positions in any imaging target, and the matching rate of the feature points increases.

In this embodiment, for example, the number of horizontal pixels of the optical sensor of the imaging device 2 is 1456, the number of vertical pixels is 1088, the pixel size is 3.45 micrometers, and the optical magnification of the lens is 0.44 or 0.88. provided that the optical sensor of imaging device 2 employs a pixel resolution of 23.6 micrometers/pixel or less. Employing a pixel resolution of 23.6 micrometers/pixel or less results in the increased accuracy described above. Note that the above conditions such as the number of pixels and pixel size are merely examples, and the above pixel resolution is applied to all conditions.

In addition, the imaging device 2 employs an optimum working distance in order to increase the accuracy of determining match/mismatch between two images. From the results of an experiment in which a specific working distance was applied to the image inspection system 100 according to the present embodiment, the above-described accuracy is improved when the working distance is shortened. This is because the pixel resolution of the lens is reduced by shortening the working distance.

In the present embodiment, for example, under the condition that the focal length of the lens of the imaging device 2 is 35 mm, the lens of the imaging device 2 is 250 mm (for example, pixel resolution of 23.6 micrometers/pixel) or less (the lower limit is 187 mm) working distance. Employing a working distance of 250 millimeters or less results in the increased accuracy described above. Note that the conditions such as the focal length described above are merely examples, and the above working distance is applied to all conditions.

First, the imaging device 2 photographs the imaging area IA of the imaging target O (pattern sheet PS) (step 301). In this embodiment, the entire area of the pattern array PA shown in FIG. 2 corresponds to the imaging area IA.

Next, the imaging device 2 generates an image corresponding to the imaging area IA (step 302). This image serves as a reference image RI to be compared with a later-described comparison image. When the reference image RI is generated, the imaging device 2 transmits the reference image RI to the computer device 1, and the control device of the computer device 1 receives the reference image RI via the input/output device.

Next, the control device of the computer device 1 stores the reference image RI in the storage device (step 303). The controller then reads the reference image RI and identifies individual dots in the pattern array PA (step 304). The dots may be identified, for example, by binarizing the reference image RI and labeling the binarized image. Through this process, circular portions of all dots in the pattern array PA are recognized. That is, a set of coordinate positions of pixels corresponding to the outer frame portion of the dot is calculated.

Next, the control device calculates the central coordinates of each identified dot (step S75). The central coordinates are derived, for example, by calculating the diameter of the dot from the coordinate position of the outer frame portion of the dot identified in step 304 and calculating the central coordinates of the diameter.

Next, the control device calculates an area within a predetermined range within the dot from the central coordinates calculated in step 305 (step 306). In this embodiment, this area corresponds to a square area having a predetermined size centered on the central coordinates. This area is called an "intra-dot area (intra-pattern area) DA _intra ". Through this processing, a set of coordinate positions of pixels corresponding to the intra-dot area DA _intra is calculated.

FIG. 4 shows an example of the in-dot area DA _intra . As shown in FIG. 4, the in-dot area DA _intra corresponds to a square area having a predetermined size centered on the center coordinates (in-dot center coordinates (in-pattern center coordinates) CC _intra ) in the dot D. do. In this embodiment, the intra-dot area DA _intra corresponds to a square, but may have any shape other than the square.

Next, the control device calculates luminance values for all pixels in the intra-dot area DA _intra , and calculates the average (in-dot average luminance value (in-pattern average luminance value)) (step 307). A luminance value is calculated by taking a weighted average of each pixel RGB value according to equation (1). The coefficients used in equation (1) (0.298912 for R, 0.586611 for G, and 0.114478 for B) are derived from the rule of thumb that the human eye sees that way.
Luminance value = 0.298912 x R + 0.586611 x G + 0.114478 x B (1)

Next, the control device calculates an area within a predetermined range between adjacent dots from the coordinates identified in step 304 and the central coordinates calculated in step 305 (step 308). In the present embodiment, this area is formed by calculating inter-dot central coordinates (inter-pattern central coordinates) between the respective central coordinates of two adjacent dots, and determining a predetermined size centered on the inter-dot central coordinates. corresponds to a square area with This area is called an "inter-dot area (inter-pattern area) DA _inter ". Through this processing, a set of coordinate positions of pixels corresponding to the inter-dot area DA _inter is calculated.

FIG. 5 shows an example of the dot-to-dot area DA _inter . As shown in FIG. 5, the inter-dot area DA _inter is the center between the intra-dot center coordinates (in-dot center coordinates CC _intra1 and intra-dot center coordinates CC _intra2 ) of adjacent dots (dots D ₁ and D ₂ ). It corresponds to a square area having a predetermined size centered on coordinates (inter-dot center coordinates CC _inter ). In this embodiment, the inter-dot area DA _inter corresponds to a square, but may have any shape other than the square.

Next, the control device calculates luminance values for all pixels in the inter-dot area DA _inter and calculates the average (inter-dot average luminance value (inter-pattern average luminance value)) (step 309). The luminance value is calculated according to Equation (1) described above.

Next, the control device repeats the processing of steps 305 to 307 for all dots in the pattern array PA in the reference image RI recognized in step 304 (step 310). Through this processing, a set of in-dot average brightness values for all dots in the pattern array PA is calculated. Each of the calculated in-dot average brightness values is stored in the storage device in association with a value (for example, dot identification number (pattern identification number)) indicating each dot in the pattern array PA.

The control device also repeats the processing of steps 308 and 309 for all adjacent dots within the pattern array PA in the reference image RI recognized in step 304 (step 311). Through this process, a set of dot-to-dot average brightness values for all adjacent dots in the pattern array PA is calculated. Each of the calculated inter-dot average brightness values is stored in the storage device in association with a value (for example, an inter-dot group identification number (inter-pattern group identification number)) indicating a group between dots in the pattern array PA.

Further, the control device identifies a set of dots (dot row (pattern row)) at the X coordinate in the pattern array PA, and calculates an approximate straight line connecting the center coordinates of each dot in the dot row (step 312). . FIG. 6 shows an example of an approximate straight line for a particular dot row.

As described above, the dots are arranged evenly in the pattern array PA. Therefore, as shown in FIG. 6, in the dot row in the pattern array PA, the dots are arranged so as to draw a straight line in the direction of the arrow X. As shown in FIG. This straight line corresponds to the approximate straight line.

This process is repeated for all rows in the pattern array PA. Through this processing, a set of approximate straight lines for all dot rows in the pattern array PA is calculated. The calculated approximate straight line is stored in the storage device in association with a value (for example, dot row identification number (pattern row identification number)) indicating each dot row in the pattern array PA.

Similarly, the control device identifies a set of dots (dot row (pattern row)) at the Y coordinate in the pattern array PA, and calculates an approximate straight line connecting the center coordinates of each dot in the dot row (step 313). ). FIG. 6 shows an example of an approximate straight line for a particular dot row.

As described above, the dots are arranged evenly in the pattern array PA. Therefore, as shown in FIG. 6, in the dot row in the pattern array PA, the dots are arranged so as to draw a straight line in the arrow Y direction. This straight line corresponds to the approximate straight line.

This process is repeated for all columns in the pattern array PA. Through this processing, a set of approximate straight lines for all dot rows in the pattern array PA is calculated. The calculated approximate straight line is stored in the storage device in association with a value (for example, dot row identification number (pattern row identification number)) indicating each dot row in the pattern array PA.

By the reference image analysis processing described above, the in-dot average brightness value, the inter-dot average brightness value, and the approximate straight line for all the dots of the pattern array PA in the reference image RI are stored in the storage device of the computer device 1 . In the comparison image analysis process, which will be described later, the in-dot average brightness value, the inter-dot average brightness value, and the approximate straight line are similarly calculated for all the dots in the array PA for the comparison image. Then, the in-dot average brightness value, the inter-dot average brightness value, and the approximate straight line for the comparative image are compared with the intra-dot average brightness value, the inter-dot average brightness value, and the approximate straight line for the reference image.

In the reference image analysis process, an example in which the imaging device 2 photographs the pattern sheet PS as the imaging target O has been described, but the present invention is not limited to such an example. For example, the pattern sheet PS may exist separately from the imaging target O (medicine tablet, etc.), and may overlap the imaging target O and be photographed. The same applies to the comparative image analysis processing to be described later. In this case, the pattern sheet PS is composed of a transparent sheet.

FIG. 7 shows an example of a state in which the photographing object O is photographed using the pattern sheet PS. As shown in FIG. 7, the pattern sheet PS is arranged between the imaging device 2 and the object O to be photographed. The light L emitted from the image pickup device 2 passes through the pattern sheet PS and reaches the object O to be photographed. Images (reference image and comparative image) generated by photographing the object O in this way include the pattern array PA and the package label, for example, when the object O is a drug tablet (package label). It will be.

Also, instead of using the pattern sheet PS, a pattern array may be formed in advance on the object O to be photographed. In this case, if the imaging target O is a drug tablet (packaging label), a pattern array is formed on the packaging sheet.

Next, the comparison image analysis processing executed by the image inspection system 100 will be described with reference to the flowchart shown in FIG. The comparative image analysis processing is performed, for example, in the pre-stage of the manufacturing process of medicines. In the comparative image analysis process, the imaging device 2 photographs the same photographing area IA of the pattern sheet PS as the reference image RI to generate a comparative image.

In principle, a comparative image generated by photographing the same pattern sheet PS as the original pattern sheet PS of the reference image RI matches the reference image RI. However, when the pattern sheet PS is photographed to generate the reference image RI (hereinafter referred to as the first photographing) and when the pattern sheet PS is photographed to generate the comparative image (hereinafter referred to as the second photographing), The two images may not match due to the difference in shooting conditions between the two.

In the processing shown in FIG. 8, a mismatch between the comparison image and the reference image is detected by comparing the luminance values of the comparison image and the reference image. By executing this process in the pre-stage of the drug manufacturing process, differences in imaging conditions can be detected. By detecting the difference in the photographing conditions in the previous stage of the manufacturing process, it is possible to take measures such as correcting the imaging device 2 to an appropriate position.

First, the imaging device 2 photographs the imaging area IA of the imaging target O (pattern sheet PS) (step 801). The imaging area IA corresponds to the same area as the imaging area IA in the reference image RI described with reference to FIG.

Next, the imaging device 2 generates an image corresponding to the imaging area IA (step 802). This image corresponds to the comparison image CI to be compared with the reference image RI. After the comparison image CI is generated, the imaging device 2 transmits the comparison image CI to the computer device 1, and the control device of the computer device 1 receives the reference image RI via the input/output device.

Next, in the computer device 1, the processing of steps 803 to 813 is performed on the comparison image CI. omitted. That is, the control device of the computer device 1 calculates the in-dot average brightness value, the inter-dot average brightness value, and the approximate straight line for all the dots of the pattern array PA in the comparison image CI.

Next, the control device compares the approximate straight line for the specific dot row calculated in step 812 with the approximate straight line for the corresponding dot row calculated for the reference image, and calculates the difference value between the two. . It is also determined whether this difference value is within a predetermined threshold (step 814). The difference value of the approximate straight line for the two images (reference image RI and comparison image CI) is calculated for all dot rows and determined whether it is within the threshold.

Similarly, the control device compares the approximate straight line for the specific dot row calculated in step 810 with the approximate straight line for the corresponding dot row calculated for the reference image, and calculates the difference value between the two. . It is also determined whether this difference value is within a predetermined threshold (step 815). The difference values of the approximate straight lines for the two images are calculated for all dot rows and determined whether they are within the threshold.

An example of the difference in the approximate straight line between the dot line in the reference image RI and the dot line in the comparison image CI will be described with reference to FIGS. 9 and 10. FIG.

FIG. 9(a) shows an example of the pattern array PA in the state in the first imaging and the reference image RI generated as a result. Arrows in the pattern array PA indicate approximate straight lines for specific dot rows. As shown in FIG. 9(a), the imaging device 2 faces the pattern sheet PS, which is an object to be photographed, in the first photographing. As a result, in the generated reference image RI, the pattern array PA is positioned substantially in the center within the reference image RI.

FIG. 9B shows an example of the pattern array PA in the state in the first imaging and the comparative image CI generated as a result. Arrows in the pattern array PA indicate approximate straight lines for specific dot rows. This approximate straight line corresponds to the approximate straight line shown in FIG. As shown in FIG. 9B, the imaging device 2 is tilted with respect to the pattern sheet PS to be photographed at the time of the first photographing. As a result, in the generated comparative image CI, the pattern array PA is laterally displaced in the comparative image CI as compared with the pattern array in the comparative image RI.

FIG. 10 shows an approximate straight line for a specific dot row (reference image approximate straight line RA) calculated for the reference image RI and an approximate straight line for the corresponding dot row calculated for the comparative image CI (comparative An example of a state of comparison with the image approximation straight line CA) is shown. In FIG. 10, the reference image approximating straight line RA and the comparative image approximating straight line CA are shown in the same plane in order to clearly show that the comparative image approximating straight line CA is laterally shifted. In order to distinguish between the two, the dot row in the reference image RI is blacked out and the dot row in the comparative image CI is shaded.

As shown in FIG. 10, when the reference image approximation straight line RA and the comparison image approximation straight line CA are compared, they are separated by the difference Di. Since the reference image approximation straight line RA is the reference, the difference Di means that the comparison image approximation straight line CA is laterally shifted by Di. Difference values indicating this difference are calculated for all dot rows and dot columns in steps 814 and 815 .

Note that in this embodiment, the approximate straight lines for the dot rows and dot columns in the pattern array are compared, but the present invention is not limited to such an example. For example, for both the reference image RI and the comparison image CI, an approximate straight line connecting any two points in the pattern array PA (for example, a straight line inclined at approximately 45 degrees) may be calculated and compared. .

Further, for example, the center coordinates of each dot in the reference image RI and the center coordinates of each dot in the comparison image CI may be compared. In this case, although the computational load increases, it is possible to detect the occurrence of a mismatch only in a specific area within the image. That is, in the processing of steps 814 and 815, the coordinate positions between the dots in the reference image RI and the dots in the comparison image CI are compared.

Through the processing of steps 814 and 815, image mismatch between the reference image RI and the comparison image CI can be detected. As described above, this discrepancy is due to the fact that the imaging device 2 is tilted with respect to the pattern sheet PS in the second photographing as compared to the first photographing. Thus, by detecting discrepancies between images, differences in imaging conditions can be determined.

Returning to the description of FIG. 8, the control unit converts the dot-to-dot average brightness value and the dot-to-dot average brightness value for the specific dot calculated in step 808 to the dot-to-dot average brightness value for the corresponding dot calculated for the reference image. It is compared with the inter-dot average brightness value and the inter-dot average brightness value, and the difference value between the two is calculated. It is also determined whether this difference value is within a predetermined threshold (step 816). The dot-to-dot average brightness value and the dot-to-dot average brightness value difference value for the two images (the reference image RI and the comparison image CI) are calculated for all dots, and it is determined whether or not they are within the threshold.

11 and 12, an example of the difference in the in-dot average brightness value and the inter-dot average brightness value between the dots in the pattern array PA in the reference image RI and the dots in the pattern array PA in the comparison image CI. explain. Note that the examples shown in FIGS. 11 and 12 are different from the examples shown in FIGS. 9 and 10 .

FIG. 11(a) shows an example of the pattern array PA in the reference image RI. As shown in FIG. 11(a), each dot in the pattern array PA has substantially the same size.

FIG. 11(b) shows an example of the pattern array PA in the comparison image CI. As shown in FIG. 11(b), each dot in the pattern array PA has a larger dot size in the portion surrounded by the dashed line than the dots in the reference image RI. This is because, for example, the distance between the imaging device 2 and the pattern sheet PS at the time of the first photography is different from the distance at the location at the time of the second photography.

FIG. 12(a) shows an example of the reference image inter-dot area DA _RDinter between adjacent dots in the reference image RI. FIG. 12(b) shows an example of the comparison image inter-dot area DA _CDinter between adjacent dots in the comparison image CI.

As shown in FIG. 12, the reference image inter-dot area DA _RDinter does not overlap with any of the two dots (reference image dots RD ₁ and RD ₂ ) in the reference image RI. In other words, there is no black portion in the reference image inter-dot area DA _RDinter .

On the other hand, the comparative image inter-dot area DA _CDinter partially overlaps both of the two dots (comparative image dots CD ₁ and CD ₂ ) in the comparative image CI. In other words, a black portion exists in the comparison image inter-dot area DA _CDinter .

Therefore, the dot-to-dot average brightness value for the reference image dot-to-dot area DA _{RDinter differs} from the dot-to-dot average brightness value for the comparison image dot-to-dot area DA _CDinter by a constant difference value. By setting a threshold within this difference value range, the difference between the two exceeds the threshold.

By setting the threshold value as described above, both the reference image dot _{RD 1} and the comparison image dot CD ₁ have substantially the same center coordinates. does not exceed the threshold. _The same is true for the reference image dot _RD2 and the comparative image dot CD2.

On the other hand, when the comparison image inter-dot area DA _CDinter and the reference image inter-dot area DA _RDinter are compared, the difference between the average luminance values of the two exceeds the threshold value. As a result, discrepancies between the reference image and the comparison image will be detected.

As described above, the process of step 816 enables the image discrepancy between the reference image RI and the comparison image CI to be corrected even if the center coordinates of the dots of the two images are substantially the same and the sizes of the dots of both images are different. can be detected. As described above, this discrepancy is due to the difference in distance between the imaging device 2 and the pattern sheet PS between the first photographing and the second photographing. Thus, by detecting discrepancies between images, differences in imaging conditions can be determined.

In this embodiment, the in-dot average brightness value and the inter-dot average brightness value are compared with respect to all the dots in the pattern array PA for the reference image RI and the comparative image CI. No need to compare. For example, in both the reference image RI and the comparison image CI, comparison may be made only for dots within a predetermined area.

In addition, in the present embodiment, the approximate straight lines, the in-dot average brightness values, and the inter-dot average brightness values for the dot rows and dot columns of the reference image RI and the comparative image CI are compared. do not have to. Any one or more of the approximate straight line for the dot row and dot column, the intra-dot average brightness value, and the inter-dot average brightness value may be compared.

As described above, according to the present embodiment, which describes the image inspection system according to the embodiment, it is possible to detect mismatches that occur in all regions within the two images.

The approximate straight line, the in-dot average brightness value, and the difference value for the inter-dot average brightness value may be output to the output device 4 coupled to the computer device 1 . By outputting the difference ground, the user can, for example, grasp how much the imaging device 2 is tilted, and can take this into consideration when performing the second photographing again.

Also, at the time of the first photographing or the second photographing, the tilt of the imaging device 2 and/or the brightness of the photographing location are detected, and the approximate straight line, the intra-dot average luminance value, and the difference of the inter-dot average luminance value are detected. It may be stored in the storage device of the computer device 1 in association with the value. By doing so, the inclination of the imaging device 2 corresponding to the difference value can be output to the output device 4 and fed back to the user.

Note that the hardware components described in the above embodiments are exemplary only, and that other configurations are possible. Also, the order of the processes described in the above embodiment does not necessarily have to be executed in the order described, and may be executed in any order. Moreover, additional steps may be newly added without departing from the basic concept of the invention.

Also, the image inspection system 100 according to one embodiment of the present invention is implemented by a computer program executed by the computer device 1, but the computer program may be stored in a non-temporary storage medium. Examples of non-transitory storage media include read-only memory (ROM), random access memory (RAM), registers, cache memory, semiconductor memory devices, magnetic media such as internal hard disks and removable disk devices, magneto-optical media, and Including optical media such as CD-ROM discs and Digital Versatile Discs (DVDs).

The present disclosure relates to computer devices, methods and computer programs . In particular, the present disclosure relates to computer devices, methods and computer programs for detecting discrepancies resulting from differences in imaging conditions when comparing two images.

A computer device according to one embodiment is a computer device connected to an imaging device having an optical sensor, and receives a first image including at least a pattern array from the imaging device, the first image being an object to be photographed. calculating from the first image a first set of luminance values for the pattern array; receiving from the imaging device a second image including at least the pattern array; is generated by photographing the same photographing object as the photographing object, calculates a second set of luminance values for the pattern array from the second image, and calculates the first set of luminance values and the second luminance value detecting a mismatch between the first image and the second image by calculating a set of difference values between the set of values and comparing the set of difference values to a predetermined threshold; detecting a difference in the capture of the first image and the second image, the optical sensor having a pixel resolution of 23.6 micrometers/pixel or less, the difference being a difference in the position of the imaging area, the position of the imaging device; and/or at least one of differences in tilt and differences in brightness of the location where the picture was taken, the computing device comprising an output device presenting information regarding the differences.

Further, a computing device according to another embodiment is a computing device connected to an imaging device having an optical sensor, receiving a first image including at least a pattern array from the imaging device, the first image being , calculating a first coordinate position for the pattern array from the first image generated by photographing an object to be photographed; receiving a second image including at least the pattern array from the imaging device; The image is generated by photographing the same photographing object as the photographing object, calculating the second coordinate position for the pattern array from the second image, and calculating the distance between the first coordinate position and the second coordinate position. and comparing the difference value with a predetermined threshold value to detect a mismatch between the first image and the second image to determine the difference between the first image and the second image. Detecting differences in imaging conditions, the optical sensor having a pixel resolution of 23.6 micrometers/pixel or less, the differences being differences in the position of the imaging region, differences in the position and/or tilt of the imaging device, and At least one of the difference in lightness of the photographing location is included, and the computing device comprises an output device for presenting information regarding the difference.
A method according to an embodiment is also a method performed by a computing device connected to an imaging device having an optical sensor, the method comprising receiving from the imaging device a first image comprising at least a pattern array. wherein the first image is generated by photographing a subject; calculating from the first image a first set of luminance values for the pattern array; receiving a second image comprising the pattern array, the second image being generated by photographing the same subject as the subject; calculating a second set of luminance values; calculating a set of difference values between the first set of luminance values and the second set of luminance values; detecting a discrepancy between the first image and the second image to detect a difference in the capture conditions of the first image and the second image by comparing with a threshold value, the difference comprises at least one of a difference in the position of the imaging region, a difference in the position and/or tilt of the imaging device, and a difference in the brightness of the location to be photographed; and presenting information about the difference; The optical sensor has a pixel resolution of 23.6 micrometers/pixel or less.
In accordance with another embodiment, a method is performed by a computing device connected to an imaging device having an optical sensor, the method comprising receiving from the imaging device a first image comprising at least a pattern array. wherein the first image is generated by photographing a subject; calculating a first coordinate position for the pattern array from the first image; receiving a second image containing the array, wherein the second image is generated by imaging the same imaging subject as the imaging subject; calculating a second coordinate position; calculating a difference value between the first coordinate position and the second coordinate position; a step of detecting a mismatch between the image and the second image to detect a difference in shooting conditions of the first image and the second image, the difference being a difference in the position of the shooting area, the imaging device and presenting information about the difference, including at least one of a difference in the position and/or tilt of the photographing location and a difference in the brightness of the location to be photographed, wherein the optical sensor is 23.6 micrometers/pixel or less has a pixel resolution of
Also, a computer program according to one embodiment is a computer program comprising computer-executable instructions that, when executed by a computing device, cause the computing device to perform the method described above.
Also, a computing device of another embodiment is a computing device connected to an imaging device having an optical sensor, receiving a first image including at least a pattern array from the imaging device, the first image comprising: calculating a first set of luminance values for the pattern array from the first image generated by photographing a subject; receiving a second image including at least the pattern array from the imaging device; is generated by photographing the same photographing object as the photographing object, a second set of luminance values for the pattern array is calculated from the second image, and the first set of luminance values and the second set of luminance values are calculated from the second image. detecting a mismatch between the first image and the second image by computing a set of difference values between the set of luminance values and comparing the set of difference values with a predetermined threshold , the optical sensor has a pixel resolution of 23.6 micrometers/pixel or less, and the first set of luminance values is obtained from the first image at a predetermined first distance between adjacent patterns in the pattern array. A second set of intensity values is obtained from the second image for a predetermined second inter-pattern area between adjacent patterns in the pattern array. is performed by calculating the luminance value for .

Computer devices, methods, and computer program products according to embodiments can increase the accuracy of detecting discrepancies between two images by employing an optimal image resolution in an imaging device.

A computing device according to one embodiment is a computing device connected to an imaging device having an optical sensor, receiving from the imaging device a first image comprising at least a pattern array, the first image comprising: calculating a first set of luminance values for the pattern array and a first coordinate position for the pattern array from the first image generated by photographing an object to be photographed in advance ; receiving a second image including at least a pattern array, the second image being generated by photographing a subject identical to the subject; from the second image, a second image of the pattern array; and a second coordinate position for the pattern array, and a set of luminance difference values between the first set of luminance values and the second set of luminance values. , calculating a coordinate difference value between the first coordinate position and the second coordinate position, and comparing the set of luminance difference values with a predetermined threshold value ; or detecting discrepancies between the first image and the second image by comparing to a defined threshold to detect differences in the capture of the first image and the second image; The optical sensor has a pixel resolution of 23.6 micrometers/pixel or less, and the differences include differences in the position of the imaging region, differences in the position and/or tilt of the imaging device, and imaging associated with the differences . At least one of the brightness differences detected by a brightness sensor located anywhere in the location, the computing device comprising an output device for presenting information about the differences.

Also according to an embodiment is a method performed by a computing device connected to an imaging device having an optical sensor, the method comprising receiving from the imaging device a first image comprising at least a pattern array. wherein the first image is generated by photographing an object to be photographed in advance ; and from the first image, a set of first luminance values for the pattern array and the pattern array a step of calculating a first coordinate position ; and a step of receiving a second image including at least a pattern array from the imaging device, wherein the second image is the same imaging target as the imaging target. calculating from the second image a second set of luminance values for the pattern array and a second coordinate position for the pattern array; and the first calculating a set of luminance difference values between the set of luminance values and the second set of luminance values; and calculating the coordinate difference values between the first coordinate position and the second coordinate position and comparing the set of luminance difference values with a predetermined threshold value , or comparing the coordinate difference values with a predetermined threshold value, to obtain the first image and the second image. a step of detecting a difference in photographing conditions between the first image and the second image by detecting a mismatch between the images of the first image and the second image, wherein the difference is a difference in the position of the photographing area, the position of the imaging device at least one of a difference in position and/or tilt, and a difference in brightness detected by a brightness sensor placed at any position of the shooting location associated with said difference; and C., wherein the optical sensor has a pixel resolution of 23.6 micrometers/pixel or less.
Also, a computer program according to one embodiment is a computer program comprising computer-executable instructions that, when executed by a computing device, cause the computing device to perform the method described above.
Also, a computing device of another embodiment is a computing device connected to an imaging device having an optical sensor, receiving a first image including at least a pattern array from the imaging device, the first image comprising: calculating a first set of luminance values for the pattern array from the first image generated by photographing a subject; receiving a second image including at least the pattern array from the imaging device; is generated by photographing the same photographing object as the photographing object, a second set of luminance values for the pattern array is calculated from the second image, and the first set of luminance values and the second set of luminance values are calculated from the second image. Detect a mismatch between the first image and the second image by computing a set of difference values between the set of luminance values and comparing the set of difference values with a predetermined threshold value and the optical sensor has a pixel resolution of 23.6 micrometers/pixel or less, and the first set of luminance values are obtained from the first image at a predetermined number between adjacent patterns in the pattern array. A second set of luminance values is obtained from the second image by calculating luminance values for one inter-pattern region, and a second set of luminance values for predetermined second pattern inter-pattern regions between adjacent patterns in the pattern array. This is done by calculating the luminance value for the region.

Claims (13)

A computing device connected to an imaging device having an optical sensor,
receiving a first image including at least a pattern array from the imaging device, the first image being generated by photographing an object to be photographed;
calculating a first set of luminance values for the pattern array from the first image;
receiving a second image including at least a pattern array from the imaging device, the second image being generated by photographing the same subject as the subject;
calculating a second set of luminance values for the pattern array from the second image;
calculating a set of difference values between the first set of luminance values and the second set of luminance values;
detecting a mismatch between the first image and the second image by comparing the set of difference values to a predetermined threshold;
the optical sensor has a pixel resolution of 23.6 micrometers/pixel or less;
A computing device characterized by: 2. The computer device of claim 1, wherein the imaging device further comprises a lens, and wherein the distance from the lens to the subject is 250 millimeters or less. calculating the first set of luminance values by calculating luminance values for predetermined first pattern inner regions in each pattern in the pattern array from the first image;
calculating the second set of luminance values by calculating luminance values for predetermined second pattern inner regions in each pattern in the pattern array from the second image;
3. A computing device according to claim 1 or 2, characterized in that: The first pattern inner region includes a plurality of pixels, and the first pattern inner region calculates a luminance value for each pixel in the first pattern inner region, and calculates an average of the calculated luminance values. compute a set of luminance values of
The second pattern inner area includes a plurality of pixels, and the second pattern inner area is calculated by calculating a luminance value of each pixel in the second pattern inner area and calculating an average of the calculated luminance values. compute a set of luminance values for
4. A computer device as recited in claim 3, wherein: calculating from the first image the first set of luminance values by calculating luminance values for a first predetermined inter-pattern region between adjacent patterns in the pattern array;
calculating from the second image the second set of luminance values by calculating luminance values for a second predetermined inter-pattern region between adjacent patterns in the pattern array;
5. A computer device as claimed in any one of claims 1 to 4, characterized in that: The first inter-pattern area includes a plurality of pixels, and the first inter-pattern area is calculated by calculating a luminance value for each of the pixels in the first inter-pattern area and calculating an average of the calculated luminance values. compute a set of luminance values of
The second inter-pattern area includes a plurality of pixels, and the second inter-pattern area is calculated by calculating a luminance value for each of the pixels in the second inter-pattern area and calculating an average of the calculated luminance values. compute a set of luminance values for
6. A computer device as recited in claim 5, wherein: A computing device connected to an imaging device having an optical sensor,
receiving a first image including at least a pattern array from the imaging device, the first image being generated by photographing an object to be photographed;
calculating a first coordinate position for the pattern array from the first image;
receiving a second image including at least a pattern array from the imaging device, the second image being generated by photographing the same subject as the subject;
calculating a second coordinate position for the pattern array from the first image;
calculating a difference value between the first coordinate position and the second coordinate position;
detecting a mismatch between the first image and the second image by comparing the difference value to a predetermined threshold;
the optical sensor has a pixel resolution of 23.6 micrometers/pixel or less;
A computing device characterized by: 8. The computing device of claim 7, wherein the imaging device further comprises a lens, and wherein the distance from the lens to the subject is 250 millimeters or less. calculating the first coordinate position by calculating a set of center coordinates within each pattern in the pattern array from the first image;
calculating the second coordinate position by calculating a set of central coordinates within each pattern in the pattern array from the second image;
9. A computing device according to claim 7 or 8, characterized in that: calculating the first coordinate position by calculating a first approximate straight line connecting a plurality of patterns in the pattern array from the first image based on the set of central coordinates;
calculating the second coordinate position by calculating a second approximate straight line connecting a plurality of patterns in the pattern array from the second image based on the set of central coordinates;
10. A computer device as recited in claim 9. A method performed by a computing device connected to an imaging device having an optical sensor, comprising:
receiving from the imaging device a first image comprising at least a pattern array, the first image being generated by photographing a subject;
calculating from the first image a first set of luminance values for the pattern array;
receiving from the imaging device a second image including at least a pattern array, wherein the second image is generated by photographing a subject identical to the subject;
calculating a second set of luminance values for the pattern array from the second image;
calculating a set of difference values between the first set of luminance values and the second set of luminance values;
detecting a mismatch between the first image and the second image by comparing the set of difference values to a predetermined threshold;
the optical sensor has a pixel resolution of 23.6 micrometers/pixel or less;
A method comprising: A method performed by a computing device connected to an imaging device having an optical sensor, comprising:
receiving from the imaging device a first image comprising at least a pattern array, the first image being generated by photographing a subject;
calculating a first coordinate position for the pattern array from the first image;
receiving from the imaging device a second image including at least a pattern array, wherein the second image is generated by photographing a subject identical to the subject;
calculating a second coordinate position for the pattern array from the first image;
calculating a difference value between the first coordinate position and the second coordinate position;
detecting a mismatch between the first image and the second image by comparing the difference value to a predetermined threshold;
the optical sensor has a pixel resolution of 23.6 micrometers/pixel or less;
A method comprising: A computer program comprising computer-executable instructions, said computer-executable instructions, when executed by a computer device, causing said computer device to perform the method of claim 11 or 12. program.

Images