KR102538080B1 - Apparatus and method for learning iris image - Google Patents

Abstract

An iris image learning apparatus and method are disclosed. An iris image learning apparatus according to an embodiment of the present invention includes an input unit for receiving an iris image in which a pupil including an iris of a target is photographed; A data generator that generates time-series data by sampling brightness levels on a plurality of virtual straight lines extending radially from the center of the pupil to the boundary between the pupil and the white of the iris in the iris image, and converts the time-series data into a feature image (FEATURE IMAGE). and a learning unit for converting and learning the feature image.

Description

Iris image learning device and method {APPARATUS AND METHOD FOR LEARNING IRIS IMAGE}

The present invention relates to iris recognition technology, and more particularly, to iris recognition technology using machine learning.

Recently, due to COVID-19, people are increasingly interested in personal health.

In the case of smart watches, health management applications that can check personal health are provided. The health management application of the smart watch provides various diagnostic functions such as electrocardiogram, blood sugar management, and exercise management, and these diagnosis results are used as remote medical treatment data and are expanding to the personal health record (PHR) market. .

In addition, fingerprint recognition is being phased out due to COVID-19, and iris recognition technology is attracting attention as a new biometric technology in the market due to the technical limitations of face recognition for wearing a mask.

However, there is a disadvantage in that it is difficult to apply the iris recognition algorithm to lightweight devices such as smart watches (smartphones, wearable smart IoT devices, etc.).

For example, the conventional general-purpose iris recognition algorithm (Neurotechnology's VeriEye algorithm) costs €16 for a lump sum purchase of 8,000-copy and €76 for 1-copy purchase, which requires too much cost to apply to a lightweight device. .

Therefore, it is essential to develop a lightweight iris recognition algorithm that can be easily used in lightweight devices such as smart watches (smartphones, Wearable Smart IoT devices, etc.).

Meanwhile, Korean Registered Patent No. 10-1969766 “Iris Recognition Apparatus and Method” discloses error improvement and optimization of an image sensor acquiring an iris image.

The present invention can develop an iris recognition algorithm based on machine learning that can be used in a lightweight device.

In addition, the present invention can improve the accuracy of iris recognition without losing iris image information.

An iris image learning apparatus according to an embodiment of the present invention for achieving the above object includes an input unit for receiving an iris image in which a pupil including an iris of a target is photographed; A data generator that generates time-series data by sampling brightness levels on a plurality of virtual straight lines extending radially from the center of the pupil to the boundary between the pupil and the white of the iris in the iris image, and converts the time-series data into a feature image (FEATURE IMAGE). and a learning unit for converting and learning the feature image.

In this case, the data generating unit may set a point at which a change in the brightness level exceeds a predetermined value on the plurality of virtual straight lines as a feature point.

In this case, the data generation unit may generate frequency data from a change in brightness at the feature point using a FFT (FAST FOURIER TRANSFORM) algorithm.

In this case, the data generator may compare the time-series data and the frequency data to correct an error in the brightness level.

In this case, the learning unit may convert the time-series data into the feature image using a recall plot (RECURRENCE PLOT) technique.

In this case, the feature image may be generated using only the brightness of the iris section in the time-series data.

In this case, the number of the plurality of virtual straight lines may be set to 360/N in advance (N is an integer of 2 or more), and the angle between the plurality of virtual straight lines may be set equal to N degrees.

In addition, an iris image learning method according to an embodiment of the present invention for achieving the above object is an iris image learning method of an iris image learning device, comprising the steps of receiving an iris image in which a pupil including an iris of a target is photographed. ; Generating time-series data by sampling brightness levels on a plurality of virtual straight lines extending radially from the center of the pupil to the boundary between the pupil and the white of the iris in the iris image, and converting the time-series data into a feature image (FEATURE IMAGE), , learning the feature image.

In this case, in the generating of the time-series data, a point at which a change in the brightness level occurs at a predetermined value or more on the plurality of virtual straight lines may be set as a feature point.

In this case, in the generating of the time series data, frequency data may be generated from a change in brightness at the feature point using a FFT (FAST FOURIER TRANSFORM) algorithm.

In this case, the generating of the time-series data may compare the time-series data with the frequency data to correct an error in the brightness level.

In this case, the learning step may convert the time-series data into the feature image using a recall plot (RECURRENCE PLOT) technique.

In this case, the feature image may be generated using only the brightness of the iris section in the time series data.

The number of the plurality of virtual straight lines may be previously set to 360/N (where N is an integer greater than or equal to 2), and an angle between the plurality of virtual straight lines may be set equal to N degrees.

The present invention can develop an iris recognition algorithm based on machine learning that can be used in a lightweight device.

In addition, the present invention can improve the accuracy of iris recognition without losing iris image information.

1 is a block diagram illustrating an iris image learning apparatus according to an embodiment of the present invention.
2 and 3 are diagrams illustrating a process of detecting a brightness level of an iris image according to an embodiment of the present invention.
4 and 5 are graphs showing results of applying the FFT algorithm to time series data according to an embodiment of the present invention.
6 to 11 are diagrams illustrating results of feature image conversion of time-series data according to an embodiment of the present invention.
12 is a diagram showing a result of converting time-series data into a two-dimensional spatial trajectory according to an embodiment of the present invention.
FIG. 13 is a diagram showing a result of generating a feature image from the 2D space trajectory shown in FIG. 12 .
14 to 21 are diagrams illustrating results of converting time-series data into feature images using a recall plot technique according to an embodiment of the present invention.
22 is a diagram illustrating a process of generating a feature image from the brightness level of an iris section of time-series data according to an embodiment of the present invention.
23 is a diagram showing a deep learning structure according to an embodiment of the present invention.
24 and 25 are diagrams illustrating feature maps generated using the deep learning structure of FIG. 23 .
26 is an operational flowchart illustrating a method for learning an iris image according to an embodiment of the present invention.
27 is an operational flowchart illustrating a method for recognizing an iris image through iris image learning according to an embodiment of the present invention.
28 is a diagram showing a computer system according to an embodiment of the present invention.

The present invention will be described in detail with reference to the accompanying drawings. Here, repeated descriptions, well-known functions that may unnecessarily obscure the subject matter of the present invention, and detailed descriptions of configurations are omitted. Embodiments of the present invention are provided to more completely explain the present invention to those skilled in the art. Accordingly, the shapes and sizes of elements in the drawings may be exaggerated for clarity.

Throughout the specification, when a certain component is said to "include", it means that it may further include other components without excluding other components unless otherwise stated.

Hereinafter, preferred embodiments according to the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram illustrating an iris image learning apparatus according to an embodiment of the present invention.

Referring to FIG. 1 , an iris image learning apparatus according to an embodiment of the present invention includes an input unit 110, a data generator 120, a learning unit 130, and an analysis unit 140.

The input unit 110 may receive an iris image in which a pupil including the subject's iris is photographed.

In this case, the number of the plurality of virtual straight lines may be set to 360/N in advance (N is an integer of 2 or more), and the angle between the plurality of virtual straight lines may be set equal to N degrees.

The data generator 120 may generate time-series data by sampling brightness levels on a plurality of virtual straight lines extending radially from the center of the pupil to the boundary between the pupil and the white of the iris image.

At this time, the data generator 120 may set a point where the change in brightness exceeds a predetermined value on the plurality of virtual straight lines as a feature point.

At this time, the data generating unit 120 may generate frequency data from a change in brightness at the feature point using a FFT (FAST FOURIER TRANSFORM) algorithm.

At this time, the data generator 120 may correct the error in brightness by comparing the time series data with the frequency data.

At this time, when the data generator 120 receives a clear iris image that is insensitive to external light using the AF 5MP camera module and high-power IR-LED, it more accurately combines the characteristics of the radial inflection point into a feature and a hypothesis factor to increase the brightness. errors can be corrected.

The learning unit 130 may convert the time-series data into a feature image and learn the feature image.

At this time, the learning unit 130 may convert the time-series data into the feature image using a recursive plot (RECURRENCE PLOT) technique.

In this case, the feature image may be generated using only the brightness of the iris section in the time-series data.

At this time, the learning unit 130 may generate a deep learning model for iris recognition by learning the feature image.

At this time, the learning unit 130 may develop a pre-processor using the machine learning-based VGG16 model to capture a clear and accurate iris image, review various deep learning techniques, and finally create a deep learning model. there is.

At this time, the learning unit 130 may learn and update a deep learning model for iris recognition by learning the feature image.

At this time, the learning unit 130 may also learn user information of the target person for the feature image.

The analysis unit 140 recognizes the target of the iris image for recognition using the feature image generated from the iris image for recognition input for iris recognition and the deep learning model for iris recognition generated in the learning unit 130 and uses the user information can be obtained.

At this time, the analysis unit 140 may find an inflection point on a virtual straight line on each radial line by defining feature maps of the kernel filter (3x3) from time-series data generated from the iris image for recognition.

2 and 3 are diagrams illustrating a process of detecting a brightness level of an iris image according to an embodiment of the present invention.

Referring to FIG. 2 , it can be seen that virtual straight lines are radially set from an iris image, and an inflection point, which is a point where a change in brightness on the virtual straight lines is greater than a predetermined value, is extracted as a feature point.

Referring to FIG. 3 , it can be seen that the magnitude of brightness on a virtual straight line from the center of the pupil to the outside is expressed as time-series data.

At this time, it can be seen that a point where the brightness size changes is extracted through a time series analysis technique, and the extracted point is set as a feature point.

At this time, the FFT modeling of Equation 1 may be applied by interpreting the inflection point appearing for each position on the straight line as a signal.

[Equation 1]

1/2 frequency components of the sampling number may be extracted.

4 and 5 are graphs showing results of applying the FFT algorithm to time series data according to an embodiment of the present invention.

Referring to FIG. 4 , it can be seen that the result of applying the FFT algorithm to time series data is expressed as a frequency spectrum (frequency, amplitude).

Referring to FIG. 5 , it can be seen that the result of applying the FFT algorithm of time series data is shown on a complex plane.

One of the important reasons for using the iris recognition using the FFT algorithm is that the same amplitude occurs even for the original signal shifted with respect to the x-axis or the time-axis.

For example, when people approach the camera to take a picture of the iris, the angle of the face may change slightly. At this time, the iris data is slightly distorted and the brightness changes, but when the FFT algorithm is applied, mathematics According to the characteristics of Equation 2, the same amplitude result can be obtained for the same iris.

[Equation 2]

That is, since the feature point of the inflection point with the largest brightness change for each person's iris pattern can have a minimum difference in size in the frequency domain, time series data can be corrected from frequency data generated using the FFT algorithm.

If the location of the inflection point is shifted by finding the center point of the pupil incorrectly, there may be cases where some inflection points are buried in the surrounding gray value due to the influence of ambient lighting. In this case, if a hypothesis is defined that corrects the error by finding the amplitude of the time axis through FFT for the signal at the inflection point, A.I. An iris pattern recognition (matching) algorithm can be created.

In addition, when the user attempts iris recognition, the present invention induces the user to always look at the front of the LCD (mirror) so that the angle of the iris is only slightly distorted.

6 to 11 are diagrams illustrating results of feature image conversion of time-series data according to an embodiment of the present invention.

Referring to FIGS. 6 and 7 , it can be seen that a feature image as shown in FIG. 7 is generated by applying a recall plot technique to the time series data shown in FIG. 6 .

Referring to FIGS. 8 and 9 , it can be seen that a feature image as shown in FIG. 9 is generated by applying a recurrence plot technique to the time series data shown in FIG. 8 .

Referring to FIGS. 10 and 11 , it can be seen that a feature image as shown in FIG. 11 is generated by applying a recurrence plot technique to the time series data shown in FIG. 10 .

12 is a diagram showing a result of converting time-series data into a two-dimensional spatial trajectory according to an embodiment of the present invention. FIG. 13 is a diagram showing a result of generating a feature image from the 2D spatial trajectory shown in FIG. 12 .

The feature image generation process shown in FIGS. 12 and 13 is disclosed in the paper "Classification of Time-Series Images Using Deep Convolutional Neural Networks" published in 2017, and generates a two-dimensional spatial trajectory by applying a recall plot technique to time-series data And it can be seen that the feature image is expressed from the two-dimensional space trajectory.

14 to 21 are diagrams illustrating results of converting time-series data into feature images using a recall plot technique according to an embodiment of the present invention.

Referring to FIGS. 14 and 15 , it can be seen that the result of converting the time-series data of the white noise of FIG. 14 to the feature image of FIG. 15 using a recall plot technique is shown.

Referring to FIGS. 16 and 17 , it can be seen that the result of converting the time-series data of the harmonic vibration of FIG. 16 into the feature image of FIG. 17 using a recall plot technique.

Referring to FIGS. 18 and 19 , it can be seen that the result of converting the linear trend chaotic time series data of FIG. 18 into the feature image of FIG. 19 using a recall plot technique is shown.

Referring to FIGS. 20 and 21 , it can be seen that the result of converting the time-series data of the auto-regressive process of FIG. 20 into the feature image of FIG. 21 using a recall plot technique is shown.

At this time, straight lines of time series data can be expressed as an 11x11 matrix of simple images.

In this case, when the feature image is a matrix R, Ri,j can be expressed as in Equation 3.

[Equation 3]

In Equation 3, Ri,j means the distance of Si, and the distance between the two S can be displayed as a matrix only by rounding the minimum reference value (e) to the distance.

For example, the value of (1,1) of R in the matrix is that the distance between S1(0.1) and S1 is 0, but for R(1,2), the distance between S1(0,1) and S2(1,2) is simple. When calculated, it can be expressed as in Equation 4.

[Equation 4]

As described above, a recursive plot matrix can be generated from recording the distance between S of all combinations. At this time, if the feature image is expressed as a binary value of 1 and 0, it can be expressed as in Equation 5.

[Equation 5]

22 is a diagram illustrating a process of generating a feature image from the brightness level of an iris section of time-series data according to an embodiment of the present invention. 23 is a diagram showing a deep learning model according to an embodiment of the present invention. 24 and 25 are diagrams illustrating feature maps generated using the deep learning model of FIG. 23 .

Referring to FIG. 22, a feature image (70x70) is created using only time-series data corresponding to an iris section from time-series data obtained from an iris image,

It can be seen that a process of learning a feature image and generating a learning model using the deep learning model of the CNN (Convolution Neural Network) structure shown in FIG. 23 is shown.

Referring to FIGS. 24 and 25 , it can be seen that a feature map generated from a deep learning model that has learned the feature map is shown.

26 is an operational flowchart illustrating a method for learning an iris image according to an embodiment of the present invention.

Referring to FIG. 26 , the iris image learning method according to an embodiment of the present invention may receive an iris image (S210).

That is, in step S210, an iris image in which a pupil including an iris of the target is photographed may be input.

In this case, the number of the plurality of virtual straight lines may be set to 360/N in advance (N is an integer of 2 or more), and the angle between the plurality of virtual straight lines may be set equal to N degrees.

In addition, the iris image learning method according to an embodiment of the present invention may generate time-series data (S220).

That is, in step S220, time-series data may be generated by sampling brightness levels on a plurality of virtual straight lines extending radially from the center of the pupil to the boundary between the pupil and the white of the iris in the iris image.

At this time, in step S220, a point at which a change in the brightness level occurs at a predetermined value or more on the plurality of virtual straight lines may be set as a feature point.

At this time, in step S220, frequency data may be generated from a change in brightness at the feature point using a FFT (FAST FOURIER TRANSFORM) algorithm.

At this time, in step S220, the error in brightness may be corrected by comparing the time series data with the frequency data.

At this time, step S220 receives a clear iris image that is insensitive to external light using the AF 5MP camera module and high-power IR-LED, and more accurately combines the characteristics of the radial inflection point with a feature and a hypothesis factor to reduce the error in brightness. can be corrected.

In addition, the iris image learning method according to an embodiment of the present invention may learn feature images (S230).

That is, in step S230, the time-series data may be converted into a feature image, and the feature image may be learned.

At this time, in step S230, the time-series data may be converted into the feature image using a recursive plot technique.

In this case, the feature image may be generated using only the brightness of the iris section in the time-series data.

At this time, in step S230, a deep learning model for iris recognition may be generated by learning the feature image.

At this time, in step S230, a deep learning model for iris recognition may be learned and updated by learning the feature image.

At this time, in step S230, user information of the target person for the feature image may be received and learned together.

At this time, in step S230, a preprocessor is developed using the machine learning-based VGG16 model to capture a clear and accurate iris image, and after reviewing various deep learning techniques, a deep learning model may be finally generated. .

27 is an operational flowchart illustrating a method for recognizing an iris image through iris image learning according to an embodiment of the present invention.

Referring to FIG. 27 , the method for recognizing an iris image through iris image learning according to an embodiment of the present invention may receive an iris image for recognition (S310).

That is, in step S310, an iris image for recognition in which a pupil including the subject's iris is photographed may be input.

In addition, the iris image recognition method through iris image learning according to an embodiment of the present invention may generate time-series data (S320).

That is, in step S320, time-series data may be generated from the iris image for recognition in the same manner as in step S220.

In addition, the iris image recognition method through iris image learning according to an embodiment of the present invention may recognize the iris of the target (S330).

That is, in step S330, the time-series data may be converted into a feature image (FEATURE IMAGE) in the same manner as in step S230.

At this time, in step S330, if the feature maps of the kernel filter (3x3) are defined from the time-series data generated from the iris image for recognition, an inflection point on a virtual straight line on each radial shape can be found.

At this time, in step S330, an object of an iris image for recognition may be recognized using a pre-generated deep learning model for iris recognition, and user information may be acquired.

28 is a diagram showing a computer system according to an embodiment of the present invention.

Referring to FIG. 28 , an iris image learning apparatus according to an embodiment of the present invention may be implemented in a computer system 1100 such as a computer-readable recording medium. As shown in FIG. 28, the computer system 1100 includes one or more processors 1110, memory 1130, user interface input device 1140, and user interface output device 1150 communicating with each other through a bus 1120. and storage 1160 . In addition, computer system 1100 may further include a network interface 1170 coupled to network 1180 . The processor 1110 may be a central processing unit or a semiconductor device that executes processing instructions stored in the memory 1130 or the storage 1160 . The memory 1130 and the storage 1160 may be various types of volatile or non-volatile storage media. For example, the memory may include ROM 1131 or RAM 1132 .

An iris image learning apparatus and method according to an embodiment of the present invention displays 180 (=360 degrees/2) radial lines in time series from the center of the pupil, and uses a recurrence plot technique to determine the inflection point as a feature image. can create

In addition, the iris image learning apparatus and method according to an embodiment of the present invention may generate a learning model by learning a feature image using a CNN algorithm.

In addition, the iris image learning apparatus and method according to an embodiment of the present invention uses a CNN algorithm and defines feature maps of a kernel filter (3x3) from time-series data generated from an iris image for recognition on a virtual straight line on each radial. An inflection point can be found.

In addition, the iris image learning apparatus and method according to an embodiment of the present invention can develop a machine learning-based iris recognition algorithm with discrimination of about 1,000 within 1 second in an embedded system using an ARM A72 class process with a reduced amount of calculation.

As described above, in the iris image learning apparatus and method according to an embodiment of the present invention, the configuration and method of the embodiments described above are not limitedly applicable, but the embodiments can be modified in various ways. All or part of each embodiment may be configured by selectively combining them.

110: input unit 120: data generation unit
130: learning unit 140: analysis unit
1100: computer system 1110: processor
1120: bus 1130: memory
1131: Rom 1132: Ram
1140: user interface input device
1150: user interface output device
1160: storage 1170: network interface
1180: Network

Claims (14)

an input unit for receiving an iris image in which a pupil including an iris of a subject is photographed;
a data generation unit generating time-series data by extracting brightness levels on a plurality of virtual straight lines radially extending from the center of the pupil to the boundary between the pupil and the white of the iris in the iris image for each predetermined unit; and
A learning unit that converts the time series data into a feature image and learns the feature image.
including,
The time series data is
The iris image learning device characterized in that it includes values of the brightness size extracted for each predetermined unit on the plurality of virtual straight lines. The method of claim 1,
The data generator
An iris image learning device, characterized in that setting a point where a change in the brightness size exceeds a predetermined value on the plurality of virtual straight lines as a feature point. The method of claim 2,
The data generator
An iris image learning device characterized in that for generating frequency data from a change in brightness at the feature point using a FFT (FAST FOURIER TRANSFORM) algorithm. The method of claim 3,
The data generator
The iris image learning device, characterized in that for correcting the error of the brightness size by comparing the time series data with the frequency data. The method of claim 4,
The learning department
An iris image learning device, characterized in that for converting the time-series data into the feature image using a recall plot (RECURRENCE PLOT) technique. The method of claim 5,
The feature image is
An iris image learning device characterized in that it is generated using only the brightness size of the iris section in time series data. The method of claim 1,
The number of the plurality of virtual straight lines is
An iris image learning device, characterized in that it is set to 360/N in advance (N is an integer of 2 or more), and the angle between the plurality of virtual straight lines is set equal to N degrees. In the iris image learning method of the iris image learning device,
receiving an iris image in which a pupil including an iris of a subject is photographed;
generating time-series data by extracting brightness levels on a plurality of virtual straight lines radially extending from the center of the pupil to the boundary between the pupil and the white of the iris in the iris image for each predetermined unit; and
Converting the time series data into a feature image and learning the feature image
including,
The time series data is
An iris image learning method comprising values of brightness magnitudes extracted for each predetermined unit on the plurality of virtual straight lines. The method of claim 8,
Generating the time series data
The iris image learning method, characterized in that setting a point on the plurality of virtual straight lines at which the change in the brightness size exceeds a predetermined value occurs as a feature point. The method of claim 9,
Generating the time series data
An iris image learning method characterized by generating frequency data from a change in brightness at the feature point using a FFT (FAST FOURIER TRANSFORM) algorithm. The method of claim 10,
Generating the time series data
The iris image learning method, characterized in that by comparing the time series data and the frequency data to correct the error of the brightness size. The method of claim 11,
The learning step is
An iris image learning method, characterized in that for converting the time-series data into the feature image using a recall plot (RECURRENCE PLOT) technique. The method of claim 12,
The feature image is
An iris image learning method characterized in that it is generated using only the brightness size of the iris section in time series data. The method of claim 8,
The number of the plurality of virtual straight lines is
An iris image learning method, characterized in that it is set to 360/N in advance (N is an integer of 2 or more), and the angle between the plurality of virtual straight lines is set equal to N degrees.

Images