JP2001353127A - Optical equipment and camera with gaze detection function - Google Patents

Abstract

(57) [Summary] [Problem] To reduce the troublesome operation of the calibration work required by the user and quickly acquire correction data for calculating an individual difference correction coefficient for performing accurate gaze detection, Using this, it is possible to detect the optimal gaze information. SOLUTION: A storage means for storing a correction coefficient relating to an individual difference for correcting a shift between an optical axis of an eyeball and a line of sight, and an arithmetic means for correcting an optical axis of the eyeball using the individual difference correction coefficient and detecting a line of sight. S01) the coordinate instructing means for instructing an arbitrary coordinate on the observation screen by an external operation member; and assuming that the coordinates of the line of sight calculated by the arithmetic means are different from the coordinates watched by the user, (S04 → S07), the updating means (S04) updates the individual difference correction coefficient using the gaze coordinates.
08).

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an improvement in an optical device with a line-of-sight detection function for detecting an axis in the direction of a gazing point observed by a user, that is, a line of sight, and a camera.

[0002]

2. Description of the Related Art Hitherto, various devices have been proposed which detect what direction a user is gazing in an observation space, that is, what is called a line of sight. In Japanese Patent Application Laid-Open No. Hei 2-264632, gaze detection is performed by irradiating a user's eyeball with infrared light and using a corneal reflection image formed by reflected light from the cornea of the eyeball and an image formation position of a pupil to obtain a gaze of the user. An apparatus is disclosed.

FIG. 8 is a diagram for explaining the principle of a known gaze detection method.

In FIG. 1, reference numerals 13a and 13b denote light sources such as light-emitting diodes which emit infrared light insensitive to a user, and each of the light sources 13a and 13b is in the x direction with respect to the optical axis of the light receiving lens 12. , And are arranged so as to illuminate the user's eyeball 17 from below (a position offset in the y direction). Part of the illumination light reflected by the eyeball 17 is transmitted through the light receiving lens 12 to the image sensor 1.
Focus on 4 17b is a cornea, and 17c is an iris.

FIG. 9A is a schematic diagram of an eyeball image projected on the image sensor 14, and FIG. 9B is a diagram showing an intensity distribution of a signal from an output line of the image sensor 14. is there.

Hereinafter, a method of detecting a line of sight will be described with reference to FIGS. 8 and 9.

The infrared light emitted from the light source 13b irradiates the cornea 17b of the user's eyeball 17. At this time, the cornea 1
The reflected corneal image d (virtual image) formed by a part of the infrared light reflected on the surface of 7b is collected by the light receiving lens 12,
An image is formed at a position d 'on the image sensor 14. Similarly,
The infrared light emitted by the light source 13a irradiates the cornea 17b of the eyeball 17. At this time, a corneal reflection image e formed by a part of the infrared light reflected on the surface of the cornea 17b is condensed by the light receiving lens 12, and forms an image at a position e 'on the image sensor 14.

The light beams from the ends a and b of the iris 17c form images of the ends a and b on the image sensor 14 via the light receiving lens 12 at positions a 'and b'. When the rotation angle θ of the optical axis of the eyeball 17 with respect to the optical axis of the light receiving lens 12 is small, the x coordinates of the ends a and b of the iris 17c are xa and xb
Then, the coordinate x of the center position c of the pupil 17d is expressed as xc ≒ (xa + xb) / 2 (1).

The x coordinate of the midpoint between the corneal reflection images d and e substantially coincides with the x coordinate xo of the center of curvature o of the cornea 17b. Therefore, the x-coordinates of the corneal reflection image generation positions d and e are x
Assuming that d, xe, and a standard distance between the center of curvature o of the cornea 17b and the center c of the pupil 17d are Loc, the rotation angle θx of the optical axis 17a of the eyeball 17 is Loc × sinθx ≒ (xd + xe) / 2−xc Satisfies the relational expression (2). Therefore, as shown in FIG. 9A, by detecting the position of each characteristic point (corneal reflection image and the center of the pupil) of the eyeball 17 projected on the image sensor 14, the optical axis of the eyeball 17 is detected. 17a can be determined.

From the equation (2), the rotation angle of the optical axis 17a of the eyeball 17 is given by β × Loc × sin θx ≒ ｛(xp0−δx) -xic｝ × Pt (3) β × Loc × sin θy ≒ ｛(yp0 −δy) −yic｝ × Pt (4) Here, β is an imaging magnification determined by the position of the eyeball 17 with respect to the light receiving lens 12, and is substantially obtained as a function of the interval | xd'-xe '| between the two corneal reflection images. Θx is the rotation angle of the eyeball optical axis in the zx plane, and θy is the rotation angle of the eyeball optical axis in the yz plane. (Xpo, ypo) is the coordinates of the midpoint of the two corneal reflection images on the image sensor 14, and (xic, yic) is the coordinates of the center of the pupil on the image sensor 14. Pt is a pixel pitch of the image sensor 14. δx and δy are correction terms for correcting the coordinates of the midpoint of the corneal reflection image, and correction terms for correcting an error caused by illuminating the user's eyeball with divergent light instead of parallel light, and δy. The correction term includes a correction term for correcting an offset component caused by illuminating the user's eyeball with divergent light from the lower eyelid.

The rotation angle of the optical axis 17a of the user (θx,
When θy) is calculated, the point of gaze (x, y) on the observation surface of the user is x [mm] = mx (θx−bx) / ax (5) y [mm] = m × (θy-by) / ay (6) Here, the x direction indicates the horizontal direction with respect to the observer, and the y direction indicates the vertical direction with respect to the user. m
Is a conversion coefficient for converting the rotation angle of the eyeball 17 into coordinates on the observation surface, and ax, bx, ay, and by are gazing point calibration coefficients, and the rotation angle of the user's eyeball 17 and the gazing point on the observation surface. Is a correction coefficient for matching.

Here, in Japanese Patent Application Laid-Open No. 4-242630, calibration is performed as a function of acquiring correction data for correcting individual differences in the user's line of sight, so that the line of sight can be detected more accurately. Detection devices have been proposed. A method of acquiring the calibration coefficients ax, bx, ay, and by along the “calibration mode” in the embodiment of the proposed device will be described.

[0013] As shown in FIG. 10, the user is gazed at four fixed targets 250 to 253 on the observation surface in order. The gaze of the targets 250 and 251 is for obtaining the calibration coefficients ax and bx in the x direction, and the gaze of the targets 252 and 253 is for obtaining the calibration coefficients ay and by in the y direction.

First, the x direction will be described. X detected when the user gazes at the target 250 (x1, 0)
Assuming that the direction eyeball rotation angle is θx1 and the x-direction eyeball rotation angle detected when gazing at the target 251 (x2, 0) is θx2, the calibration coefficients ax and bx are as follows: ax = m × (θx1−θx2) / (X1−x2) (7) bx = (θx1 × x2−θx2 × x1) / (x1−x2) (8)

Similarly, in the y direction, when the user gazes at the optotype 252 (0, y1), the detected y-direction eyeball rotation angle is θy1, and when the user gazes at the optotype 253 (0, y2). Assuming that the detected y-direction eyeball rotation angle is θy2, the calibration coefficients ay and by are as follows: ay = m × (θy1-θy2) / (y1-y2) (9) by = (θy1 × y2-θy2) × y1) / (y1-y2) (10)

In this manner, the calibration coefficients ax, bx, ay, and by are stored as the individual difference correction coefficients for each of the plurality of users in the calibration operation, and the individual difference correction data adapted to the user is stored. If it can be set, when the user's eyeball rotation angle is determined, it is possible to specify which point on the observation screen the user is gazing at, for example, when applied to a camera, on the user's focus plate looking into the finder of the camera The gaze position can be accurately calculated, and the gaze information can be used for focus adjustment of the photographing lens and the like.

Here, a specific operation for obtaining the individual difference correction coefficient described above is described in Japanese Patent Laid-Open No. Hei 6-3487.
The description will be made in accordance with an example of application to a camera disclosed in Japanese Patent Publication No. 4 (JP-A) No. 4

FIG. 11 is a schematic view of a main part of a single-lens reflex camera.

In FIG. 1, reference numeral 1 denotes a photographing lens for convenience.
Although shown with a single lens, it is actually composed of more lenses. Reference numeral 2 denotes a main mirror, which is obliquely moved to or retracted from the photographing optical path according to the state of observation of the subject image by the finder system and the state of photographing of the subject image. 3 is a submirror,
The light beam transmitted through the main mirror 2 is reflected toward a focus detection device 6 located below the camera body. 4 is a shutter,
Reference numeral 5 denotes a photosensitive member, specifically, a silver halide film or CC.
D, etc. Reference numeral 7 denotes a focusing plate arranged on a predetermined image forming plane of the photographing lens 1, 8 denotes a pentaprism for changing a finder optical path, and 9 and 10 denote image forming lenses and photometric sensors for measuring the luminance of a subject in an observation screen. is there. An eyepiece 11 provided with a spectroscope 11a is disposed behind the exit of the pentaprism 8, and is used for observation of the focus plate 7 by a user's eye 15. The spectroscope 11a is formed of, for example, a dichroic mirror that transmits visible light and reflects infrared light.

Reference numeral 12 denotes a light receiving lens, and reference numeral 14 denotes an image sensor using an image sensor such as a CCD. The image sensor is arranged so as to be conjugate with the vicinity of the pupil of the user's eye 15 at a predetermined position with respect to the light receiving lens 12. . Reference numerals 13a to 13f denote infrared light emitting diodes each serving as an illumination light source. 1
Reference numeral 5 denotes a high-brightness light-emitting diode that can be visually recognized even in a bright subject. Light emitted from the light-emitting diode 5 is reflected by the main mirror 2 via a light projecting prism 16 and vertically reflected by a micro prism 7a provided on a focus plate 7. And is visually recognized by the user's eye 15 through the pentaprism 8 and the eyepiece 11. Actually, by providing a plurality of high-intensity light emitting diodes 15, prisms 16, and micro prisms 7a, it is possible to selectively illuminate a plurality of focus detection areas on the focus plate 7. This selective illumination of the focus detection area is hereinafter referred to as superimposition.

FIGS. 12A and 12B are an upper external view and a rear view of the single-lens reflex camera.

In FIG. 1, reference numeral 18 denotes a release button, 19 denotes an LCD as an external monitor display device, 21 denotes a mode dial for selecting a photographing mode and the like, and 20 denotes an electronic dial for performing various settings. Reference numeral 24 denotes a cross button for manually selecting an arbitrary focus detection area, and includes direction buttons 24a (up selection), 24b (down selection), 24c (left selection), and 24d (right selection).

FIG. 13 shows the eyepiece 11 shown in FIG.
FIG. 4 is a viewfinder view diagram further observed, and a focus detection area 2
00 to 206 are superimposed on the focus plate 7 respectively.

FIG. 14 is a detailed explanatory diagram of the mode dial 21 shown in FIG.

By setting the display of the mode dial 21 to the index 22 marked on the camera body, the photographing mode is set based on the display contents. Reference numeral 21a denotes a lock position for disabling the camera, 21b denotes a position in an automatic shooting mode controlled by a shooting program preset by the camera, and 21c denotes a manual shooting mode in which a user can set shooting contents. Reference numeral 21d denotes a "CAL" position in a calibration mode for calibrating the user's line of sight.

Next, the calibration operation for acquiring the individual difference correction coefficient in the single-lens reflex camera will be described with reference to the flowcharts of FIGS.

First, in step S29, the user rotates the mode dial 21 to move the CAL position 21
Match d with index 22. When the camera is set to the CAL position, the camera switches to the calibration mode.

In the calibration mode, various usage conditions (for example, a state of naked eyes or a state of wearing glasses) to cope with a plurality of users or even the same user.
Therefore, a plurality of individual difference correction coefficients can be held. For example, when five individual difference correction coefficients can be held, CAL
Numbers are assigned, "CAL-1", "CAL-
2 "..." CAL-5 ".

In the next step S30, the user turns the electronic dial 20 to register a CAL number for registering an individual difference correction coefficient according to his / her own use situation, and turns the LCD1.
9 is displayed and specified. Here, the user observes the observation screen from the eyepiece lens 11. Then, the focus detection area 204 located at the right end of the focus detection areas in FIG. 13 blinks due to the superimposition (S31), so that the user pays attention to this point (S32) and the release button 18
Is depressed (S33). When the release button 18 is pressed, the visual axis detection is performed a plurality of times by a CPU (not shown) provided in the camera (S34), and the eyeball rotation angle of the user with respect to the right end focus detection area is acquired.

In the next step S35, the CPU emits an electronic sound several times using a sounding body (not shown) in order to make the user recognize that the visual line detection for the focus detection area 204 at the right end has been completed. Focus detection area 2 at the same time
04 is fully lit for a predetermined time. Then, step S3
At 6, the focus detection area 204 is turned off, and at the same time, the focus detection area 200 at the left end starts blinking. The user gazes at the blinking focus detection area 200 (S3).
7) Pressing the release button 18 (S38) causes the CPU to execute line-of-sight detection (S39). The eye gaze detection is performed a plurality of times in the same manner as in the right end focus detection area, and the eyeball rotation angle of the user with respect to the left end focus detection area is acquired.

In step S40, the CPU emits an electronic sound several times using a sounding body (not shown) so that the user can recognize that the visual line detection for the focus detection area 200 has been completed. At the same time, the focus detection area 200 is fully lit.

By this series of operations, an eyeball rotation angle corresponding to the left and right focus detection areas is obtained.

In the next step S41, the CPU determines whether or not the obtained eyeball rotation angle is appropriate. The determination is performed by calculating the rotation angle of the eyeball, which is the return number from the gaze detection subroutine, and the reliability of the pupil diameter acquired at the same time. That is, when the rotation angle of the eyeball and the pupil diameter detected in the gaze detection subroutine are not reliable, it is determined that the calculated calibration data of the gaze is also unreliable. Also, if there is reliability of the rotation angle of the eyeball and the pupil diameter detected in the gaze detection subroutine, if the calculated gaze calibration data falls within the range of general individual differences, it is determined to be appropriate, On the other hand, if the calculated line-of-sight calibration data greatly deviates from the range of general individual differences, the calculated line-of-sight calibration data is determined to be inappropriate.

If it is determined that the calculated line-of-sight calibration data is inappropriate, the process proceeds to step S42, in which the CP
U turns off the lighting focus detection areas 204 and 200, emits an electronic sound using a sounding body (not shown) for a predetermined time, and warns that the visual line calibration has failed. At the same time, a warning is issued by blinking the “CAL” display on the LCD 19. After the warning sound from the sounding body and the warning display on the LCD 19 have been performed for a predetermined time, the process proceeds to the initial step of the calibration routine, and the visual axis calibration in the x direction is again performed.

If the calculated line-of-sight calibration data is proper, the flow advances to step S43 to notify the end of the line-of-sight calibration in the x-direction several times in the focus detection areas 204 and 200 by superimposed display. Blinks to let the user know.

Following the calibration in the x direction,
Calibration in the y direction is performed in a similar manner.

At step S44 in FIG. 16, the CPU
Blinks the focus detection area mark 205 located at the upper end of the focus detection area in FIG. 13 by superimposing. When the user presses down the release button 18 while watching the focus detection area where blinking has started (S45) (S45).
46), the calibration operation is started by the CPU. In the next step S47, the CPU performs eye gaze detection a plurality of times simultaneously with the start of the operation, and obtains the eyeball rotation angle of the user with respect to the upper end focus detection area.
Then, in step S48, the CPU causes the sound to be emitted several times using a sounding body (not shown) in order to make the user recognize that the line-of-sight detection for the focus detection area 205 at the upper end has been completed. At the same time, the focus detection area 205 is fully lit for a predetermined time.

Subsequently, in step S49, the CPU turns off the focus detection area 205, and at the same time, the focus detection area 206 at the lower end starts blinking. The user gazes at the blinking focus detection area 206 (S50), and presses the release button 1
8 is depressed (S51) to cause the CPU to execute the line-of-sight detection (S52). The eye gaze detection is performed a plurality of times in the same manner as the focus detection area at the upper end, and the eyeball rotation angle of the user with respect to the focus detection area at the lower end is acquired.

In step S53, the CPU causes the sound to be emitted several times using a sounding body (not shown) in order to make the user recognize that the line-of-sight detection for the focus detection area 206 has been completed. At the same time, the focus detection area 206 is fully lit.

Through this series of operations, an eyeball rotation angle corresponding to the vertical focus detection area is obtained.

Next, in step S54, the CPU determines whether or not the obtained eyeball rotation angle is appropriate.
If it is determined that the calculated line-of-sight calibration data is inappropriate, the process proceeds to step S55, and the lighting focus detection area 2
05 and 206 are turned off, an electronic sound is emitted for a predetermined time using a sounding body (not shown), and a warning is issued that the line-of-sight calibration has failed. At the same time, a warning is issued by blinking the “CAL” display on the LCD 19. After the warning sound from the sounding body and the warning display on the LCD 19 are performed for a predetermined time, the process proceeds to the initial step of the calibration routine, and the camera is set in a state where the visual axis calibration in the y direction can be executed again. If the calculated line-of-sight calibration data is appropriate, step S56
And the completion of the eye-gaze calibration in the y direction is determined by the superimposition to the focus detection area 20.
5,206 blinks several times to make the user recognize.

After a series of line-of-sight calibration operations are completed, the CPU waits until the user operates the electronic dial 20 or the mode dial 21. If the user rotates the electronic dial 20 to select another calibration number, the change of the calibration number is detected, and the process proceeds to the initial step of the line-of-sight calibration routine. In addition, the user can use the mode dial 2
When 1 is rotated to select another shooting mode, the process returns to the main routine.

After the above-described calibration operation, the individual difference correction coefficients ax, b in a specific use state of the user are obtained.
x, ay, and by can be obtained.

In practice, however, as shown in FIG. 17, even when the same gazing point is observed, the pupil diameter of the user changes in response to the change in the surrounding brightness, and the pupil diameter changes in accordance with the change in the pupil diameter. The calculated coordinate values also change.

Therefore, while observing the same gazing point,
Japanese Patent Application Laid-Open No. 06-034874 discloses that a correction value considering a change in pupil diameter is used in calculating a rotation angle so that the same coordinates are calculated irrespective of a change in ambient brightness. That is, (7), (8), (9), (1
Rotation angles θx1, θx2, θy1, θy2 in equation (0)
Are assumed to be functions of the pupil diameter Rpp, respectively, and θx1 (Rpp), θx2 (Rpp), θy1 (R
pp), θy2 (Rpp), θx1 (Rpp) = kx1 × Rpp + mx1 (11) θx2 (Rpp) = kx2 × Rpp + mx2 (12) θy1 (Rpp) = ky1 × Rpp + my1 ... (13) θy2 (Rpp) = ky2 × Rpp + my2 (14) It can be expressed by a linear function. The above (11), (12),
In the equations (13) and (14), the coefficients kx1, mx1,
In order to obtain kx2, mx2, ky1, my1, ky2, and my2, it is necessary to perform the above-described calibration operation at least twice under different pupil diameters Rpp, that is, under different brightness environments. Further, in order to obtain the above coefficient more accurately, it is preferable to perform calibration a plurality of times under various brightness environments and calculate using the least squares method or the like.

By doing so, the calibration coefficients ax, bx, ay, and by are functions depending on the pupil diameter, and ax = gx (Rpp) (15) bx = hx (Rpp) (16) ay = gy (Rpp) (17) by = hy (Rpp) (18) (Specifically, (11),
Equation (12) is converted to equations (7) and (8), and (13) and (14)
Expressions obtained by substituting the expressions into Expressions (9) and (10), respectively). When this is reflected in the equations (5) and (6), the line-of-sight coordinates (x, y) finally become the pupil diameter Rpp and the eyeball rotation angle (θx, θ).
It can be seen that the function of y) can be obtained as x = fx (Rpp, θx) (19) y = fy (Rpp, θy) (20)

As described above, the gazing point on the observation screen of the user can be more accurately identified by repeating the calibration operation a plurality of times. For example, when applied to the single-lens reflex camera, the user looks into the finder system of the camera. When the position of the user's line of sight on the focus plate is calculated, the line of sight information can be used for focus adjustment or exposure control of the lens.

[0048]

In the above conventional example, in order to detect the line of sight with high accuracy, each time the observation state of the user, that is, the brightness of the surrounding environment changes, FIGS. The user must perform a series of calibration operations described in accordance with the flowchart of the above, and as a result, there is a problem that the operability of the camera is impaired.

In particular, in the act of calibration work, as described above, a plurality of operating members must be operated in accordance with a predetermined procedure, and eye movements required by the camera must be sequentially performed in accordance with the demands. That is, it is an independent act unrelated to the original photographing act, and there is a problem that forcing the user to perform this act repeatedly causes pain to the user.

(Object of the Invention) An object of the present invention is to reduce the troublesome operation of the calibration work required by the user, and to provide correction data for calculating an individual difference correction coefficient for performing accurate gaze detection. An object of the present invention is to provide an optical device with a line-of-sight detection function and a camera that can quickly acquire and detect optimal line-of-sight information using this.

[0051]

SUMMARY OF THE INVENTION In order to achieve the above object, the present invention provides a correction coefficient relating to an individual difference for correcting a deviation between an optical axis of an eyeball and a line of sight obtained from a rotation angle of an eyeball and a pupil diameter of a user. , An arithmetic unit that corrects the optical axis of the eyeball using the individual difference correction coefficient, and detects a line of sight, and a display unit that displays any visually recognizable target on the observation screen at arbitrary coordinates. In an optical device with a line-of-sight detection function having, coordinate specifying means for specifying an arbitrary coordinate on the observation screen by an external operation member, and coordinates of the line of sight calculated by the arithmetic means, coordinates with which the user gazes, When the gaze coordinates are input by the coordinate instructing means, the gaze coordinates are inputted, and the gaze detection function is updated by the gaze coordinates. That.

[0052]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, the present invention will be described in detail based on illustrated embodiments.

FIG. 11 is a schematic view showing an essential part of a single-lens reflex camera according to an embodiment of the present invention, FIG. 12 (a) and FIG. 12 (b) are a top view and a rear view, and FIG. Is the same as FIG. 13 described above, and the explanatory diagram of the mode dial is the same as FIG.
4, respectively, and details thereof are omitted.

FIG. 1 is a main part electric circuit diagram incorporated in the single-lens reflex camera having the above-mentioned configuration according to the embodiment of the present invention, and the same parts as those in FIG. 1 are denoted by the same reference numerals.

A microcomputer (hereinafter, referred to as CPU) 300 built in the camera body includes a visual line detection circuit 301, a photometry circuit 302, an automatic focus detection circuit 303, a signal input circuit 304, an LCD drive circuit 305, and an LED drive circuit. 306, IRED drive circuit 307, shutter control circuit 308, motor control circuit 309, focus adjustment circuit 31
0, an aperture driving circuit 311 is connected. CPU 30
The EEPROM 300a attached to 0 has a storage function of a line of sight correction data for correcting an individual difference of the line of sight as storage means.

The line-of-sight detection circuit 301 converts an eyeball image signal from the image sensor 14 (CCD-EYE) into an A / D signal.
After the conversion, the image information is transmitted to the CPU 300. The line sensor 6a includes seven sets of line sensors CCD-L2, CCD-L1, CCD-C, corresponding to the seven focus detection areas 200 to 206 in the finder visual field shown in FIG.
CCD-R1, CCD-R2, CCD-U, CCD-D
Is a well-known CCD line sensor. The automatic focus detection circuit 303 performs A / D conversion of the voltage obtained from the line sensor 6a and sends the voltage to the CPU 300.

SW1 is a switch which is turned on by the first stroke of the release button 18 to start photometry, AF, line of sight detection, and the like, and SW2 is a switch which is turned on by the second stroke of the release button 18. Electronic dial 20
Is a switch for a user to input a predetermined setting by rotation, and the rotation amount is transmitted to a signal input circuit via an up / down counter. The directions 24a to 24d are
Up (SW-U), down (SW-D) left, (S
WL) Right switch for detecting the input of (SW-R).

The LCD drive circuit 305 is provided for the LCD 19
The LED drive circuit 306 controls the lighting / flashing of the superimposed LED 15 and the like. IRE
The D drive circuit 307 includes an infrared light emitting diode (IRED1).
6) 13a to 13f are selectively turned on according to the situation.

The operation of the single-lens reflex camera having the above configuration will be described with reference to the flowchart of FIG. This flowchart operates in a state where the mode dial 21 is set to an index in a mode in which the line-of-sight information is valid among the shooting modes 21b and 21c.

When the switch SW1 is turned on (S00),
The CPU 300 proceeds to step S01, and executes a user's line of sight detection. Specifically, the coordinates on the focus plate 7 are calculated as shown in the conventional example. However, in the coordinate calculation, if the individual difference correction coefficient already exists, the effective correction coefficients ax and b are calculated based on the above-described equations (5) and (6).
This is performed using x, ay, and by. This is the data a
The determination may be made based on whether x, bx, ay, and by have been calculated once in the past, or even if it is determined that the data is valid only when it can be calculated twice or three or more times. good. Note that the eyeball rotation angle cannot be corrected for the change in the pupil diameter by only one calculation. If there is no individual difference correction coefficient, the EEPROM 30
Detection is performed using a default value stored as 0a as a reference correction coefficient.

In the next step S02, the closest focus detection area among the focus detection areas 200 to 206 is determined as the focus detection area intended by the user with respect to the line-of-sight coordinates on the focus plate calculated as described above. select. Then, the selected focus detection area is superimposed (S0
3) Then, the user is notified of the selected focus detection area.

In the focus detection area displayed by the superimposition, when a default value is used for the individual difference correction coefficient or when the correction for the influence of the surrounding brightness is not sufficient, the actual focus of the user is often used. It differs from the intended focus detection area. In this case, the user uses the cross button 24
Thus, a desired focus detection area can be manually reselected.

In the next step S04, it is a step of determining whether or not the focus detection area selected by the camera by gaze calculation is an area intended by the user. After the superimposition (S03), a predetermined time has elapsed. During this time, it is detected whether or not there is an input of the cross button 24 from the user.
If there is no input to any of the direction buttons 24a to 24d of the cross button 24, it is assumed that the focus detection area intended by the user matches the focus detection area selected by the camera, that is, the calculation of the line-of-sight coordinates is performed correctly. It is determined that the focus has been shifted to step S05, and the focus of the lens is adjusted in the focus detection area. Thereafter, when the selected focus detection area is in focus, the process proceeds to step S06, in which the switch SW2 that is turned on by the second stroke of the release button is detected. If the switch SW2 is turned on, a known series of release operations is performed, and shooting is completed. .

On the other hand, if there is an input of pressing any one of the direction buttons of the cross button 24, step S04 to step S0
Go to 7. Here, the focus detection area selected by the line of sight detection is recognized as being different from the focus detection area intended by the user and is canceled, and the focus detection area designated by the user is selected again. At the same time, this focus detection area is superimposed to inform the user. Subsequently, the process proceeds to step S08, where a subroutine for obtaining calibration data obtained by reselecting the focus detection area is performed.

Before describing this subroutine, a method of correcting calibration data in the x-axis direction will be described.

FIG. 3A shows the eyeball rotation angle θx when the calibration operation shown in the conventional example is performed twice or more.
FIG. 7 is a relationship diagram between the pupil diameter Rpp.

In the figure, reference numeral 51 denotes a function θx1 (Rpp) obtained when the user observes the coordinate x1 (Equation (11)).
52) is a function θx2 (Rpp) obtained when the user observes the coordinate x2 (same as the linear function of the equation (12)). If these two straight lines exist, as shown in the conventional example, the correction coefficient a for an arbitrary pupil diameter Rpp
x and bx can be obtained. However, even in this case, if the number of calibrations is small, it is not always possible to obtain a highly accurate correction coefficient.

Therefore, it is assumed that there exists a pupil diameter Rppc and an eyeball rotation angle θxc obtained when a certain coordinate xc is gazed at a point 53 in FIG. 3B. Since (Rppc, θx1 (Rppc)) can be obtained from the straight line 51, (x1, θx1 (Rppc)) and (xc,
θxc) into Equations (7) and (8), FIG.
As shown in (a) and (b), the correction coefficients ax (Rppc) and bx (Rppc) for the pupil diameter Rppc can be obtained as point coordinates 54 and 55. (However,
Instead of obtaining (Rppc, θx1 (Rppc)) from the straight line 51, instead of obtaining (Rppc, θx2 (Rpp
c)). According to the above operation, if the x coordinate xc of the gazing point is known, and the pupil diameter Rppc and the x-axis direction eyeball rotation angle θxc can be detected when gazing at that point, the correction coefficient ax (Rppc) for this pupil diameter Rppc ), Bx (Rppc) can be plotted. Therefore, by repeating this operation, a large amount of plot data on Rpp-ax and Rpp-bx can be obtained. When the number of plot data increases, the approximation line is updated each time, so that the correction coefficient ax = gx (Rp
p) and bx = hx (Rpp) (from equations (15) and (16)) can be calculated, and it is possible to obtain highly accurate line-of-sight coordinates.

Next, a method of obtaining the correction coefficients ax and bx when the straight lines 51 and 52 do not exist, that is, when the calibration has not been performed a plurality of times, will be described with reference to FIG.

For example, the pupil diameter when the user gazes at the known coordinates of x = x0 for the first time is Rpp0, and the eyeball rotation angle in the x-axis direction is θx0 (point 56). Assuming that the pupil diameter when the same coordinate x = x0 is gazed at another occasion is Rpp0 ′ and the eyeball rotation angle is θx0 ′ (point 57), the pupil diameter and the eyeball when gazing at x = x0 are obtained from these two points. The relational expression 58 of the rotation angle can be obtained. The plot points increase each time the known coordinates of x = x0 are repeatedly observed, and by applying an approximate straight line to those points, the relational expression between the pupil diameter and the eyeball rotation angle becomes accurate.

As described above, if there is a relational expression of pupil diameter-eyeball rotation angle by gaze at x = x0, pupil diameter Rppc when gazing at a known coordinate x = xc different from x = x0 is obtained. When the eyeball rotation angle θxc can be detected, as described above, the correction coefficient ax (Rppc) in Rppc,
bx (Rppc) can be obtained.

According to the above method, if a large data group of the known x coordinate, pupil diameter and eyeball rotation angle is obtained,
As shown in (a) and (b), ax = gx (Rpp) and bx = hx (Rpp) in equations (15) and (16) can be accurately identified, and the user's line of sight coordinates can be correctly determined. You can ask for it.

The correction coefficient updating method described so far has been described based on the x-axis direction. However, the correction coefficient can be updated in the y-axis direction in the same manner. That is, if x ← y, the method of updating the correction coefficients ay and by in the y-axis direction will be described.

Returning again to the subroutine "acquisition of calibration data" (S08), this subroutine will be described with reference to FIG.

First, as a premise, the gaze detection before reselection (S
The coordinates calculated according to (01) are those for which the user's line of sight has not been correctly obtained, and the actual coordinates watched by the user at this time are the coordinates of the focus detection area reselected. . Proceeding to step S09, the pupil diameter Rpp of the user detected at the time of detecting the line of sight in step S01.
c, Reacquire the eyeball rotation angle (θxc, θyc). Then, the process proceeds to step S10, and the focus detection area (xc, yc) specified by the user by reselection is acquired.
From these data groups, it is checked in the next step S11 whether the correction coefficient can be updated. Specifically, the correction coefficient updating method described above is used.

If it is not possible to update the correction coefficient, the process proceeds to step S12, and this data group is held as calibration data. If renewal is possible,
The process proceeds to step S13, in which a new correction coefficient is calculated so that it can be used for subsequent gaze coordinate calculation.

When the above subroutine is completed, the process returns to the main routine of FIG. 2, and the lens driving / focusing operation is performed in the focus detection area designated by the reselection (S0).
5), detecting that the switch SW2 is turned on, performing a known release operation, and terminating the photographing.

According to the above-described embodiment, in the routine for detecting the line of sight of the user, if the actual line of sight of the user is different from the calculated line of sight, the actual coordinates of the user are determined. Is input, the individual difference correction coefficient in the line-of-sight coordinate calculation process is updated. Therefore, it is possible to update the individual difference correction coefficient without independently forcing the user to operate a plurality of operation members to perform the calibration operation and to gazing at specific coordinates independently. It is possible to provide a camera having a line-of-sight detection unit with high accuracy in accordance with the condition.

More specifically, when the camera selects a focus detection area not intended by the user during the photographing operation, the user reselects the focus detection area using the cross button 24 corresponding to the coordinate indicating means. When coordinates on the observation screen are input by the cross button 24, the input coordinates and the user's pupil diameter and eyeball rotation angle obtained by the line-of-sight detection are used as input parameters, and pupil diameter-individual difference. Obtained as point data on the correction coefficient coordinate space. When a lot of this point data is obtained, it becomes possible to treat the individual difference correction coefficient as a function of the pupil diameter. Specifically, by applying an approximate straight line, an accurate individual difference correction coefficient can be obtained.
It is possible to select one focus detection area intended by the user from the plurality of focus detection areas existing on the focus plate by detecting the user's line of sight, and can select one focus detection area from a plurality of focus detection areas existing on the focusing screen. Becomes

The above is the configuration of the embodiment of the present invention, and the correspondence with the present invention. The present invention is not limited to these modes and configurations, and any configuration may be used as long as the functions described in the claims can be achieved. Further, the present invention can be provided not only to a camera but also to other optical devices having a visual line detection function.

[0081]

As described above, according to the present invention,
In addition to reducing the cumbersome operation of the calibration work that is forced by the user, the correction data for calculating the individual difference correction coefficient for accurately performing the gaze detection is quickly obtained, and the optimum gaze information is obtained using the correction data. An object of the present invention is to provide an optical device with a line-of-sight detection function and a camera capable of detection.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a main part of an electric configuration of a single-lens reflex camera according to an embodiment of the present invention.

FIG. 2 is a flowchart showing a shooting operation of the single-lens reflex camera of FIG. 1;

FIG. 3 is a diagram illustrating a relationship between an eyeball rotation angle and a pupil diameter in the embodiment of the present invention.

FIG. 4 is a plot diagram of a pupil diameter and an individual difference correction coefficient in the embodiment of the present invention.

FIG. 5 is a diagram for explaining the principle of data acquisition according to the embodiment of the present invention.

FIG. 6 is an approximate straight line correction diagram of a pupil diameter and an individual difference correction coefficient in the embodiment of the present invention.

FIG. 7 is a flowchart illustrating an operation of acquiring calibration data in the embodiment of the present invention.

FIG. 8 is a diagram schematically illustrating a main part of a general eyeball image.

FIG. 9 is a diagram illustrating a general eyeball image and extracted feature points.

FIG. 10 is an eye chart for general calibration.

FIG. 11 is a diagram schematically showing main parts when applied to a single-lens reflex camera in the conventional and the present invention.

FIG. 12 is an external view of a main part of the single-lens reflex camera of FIG. 11;

FIG. 13 is a view of the viewfinder of the single-lens reflex camera of FIG. 11;

FIG. 14 is a diagram for explaining a mode dial of the single-lens reflex camera of FIG. 11;

FIG. 15 is a flowchart showing a part of a conventional calibration operation.

FIG. 16 is a flowchart showing the operation following FIG.

FIG. 17 is a diagram for explaining a relationship between a line of sight, a pupil diameter, and peripheral brightness.

[Explanation of symbols]

REFERENCE SIGNS LIST 1 photographing lens 6 focus detection device 7 focus plate 11 eyepiece 13 infrared light emitting diode (IRED) 14 image sensor (CCD-EYE) 15 LED for superimpose 17 eyeball 24 cross button 200 to 206 focus detection point mark (calibration) 300 CPU 301 Eye-gaze detection circuit

Claims (7)

[Claims] 1. A storage means for storing a correction coefficient relating to an individual difference for correcting a deviation between an optical axis of an eyeball and a line of sight obtained from an eyeball rotation angle and a pupil diameter of a user, and an eyeball using the individual difference correction coefficient. An optical device with a line-of-sight detection function, comprising: an arithmetic unit that corrects an optical axis and detects a line of sight, and a display unit that displays an arbitrary visible target at arbitrary coordinates on the observation screen. Gaze coordinates are input by the coordinate designating means, assuming that coordinate designating means for designating arbitrary coordinates by an external operation member and coordinates of a line of sight calculated by the arithmetic means are different from coordinates gazed by a user. And updating means for updating the individual difference correction coefficient by using the gaze coordinates. 2. The apparatus according to claim 1, wherein the updating means uses input coordinate values input by the coordinate designating means, an eyeball rotation angle and a pupil diameter of the user as input parameters for updating. An optical device with a line-of-sight detection function according to item 1. 3. The apparatus according to claim 1, wherein the updating unit obtains the individual difference correction coefficient as a function of the pupil diameter in order to cope with a change in the pupil diameter due to a user's use of the optical device. Or the optical device with a line-of-sight detection function according to 2. 4. The method according to claim 1, wherein the updating means includes: an eyeball rotation angle of a user;
From the pupil diameter and the input coordinate values, an individual difference correction coefficient corresponding to the pupil diameter is obtained as point data on a pupil diameter-individual difference correction coefficient coordinate space, and an approximate straight line is obtained from a plurality of the point data to obtain the function. 4. The method according to claim 3, wherein
An optical device with a line-of-sight detection function according to item 1. 5. The updating means obtains a plurality of point data on a pupil diameter-eyeball rotation angle coordinate space when the same observation screen coordinate is gazed a plurality of times in different use situations, and obtains an approximate straight line from the point data. 5. The method according to claim 1, wherein
The optical device with a line-of-sight detection function according to any one of the above. 6. A camera comprising the optical device with a visual line detection function according to claim 1. 7. The camera according to claim 6, wherein the updating unit is executed during a series of photographing operations of the camera.