KR20250113193A - A method of measuring a head up display ghost image and an apparatus of measuring a head up display ghost image - Google Patents

Abstract

A method for measuring optical characteristics according to embodiments further comprises the steps of: generating images including points in each pattern for a virtual image plane using one or more optical measurement devices, each image captured based on the one or more optical measurement devices, each image corresponding to at least one of a left image, a center image, and a right image; and generating positions of the points based on the one or more optical measurement devices and each pattern; and generating a level for a ghost image for the virtual image plane based on the generated positions, wherein the positions are obtained based on a field of view of the one or more optical measurement devices, a gap between the left optical measurement device and the right optical measurement device.

Description

{A METHOD OF MEASURING A HEAD UP DISPLAY GHOST IMAGE AND AN APPARATUS OF MEASURING A HEAD UP DISPLAY GHOST IMAGE}

The present invention relates to a method and device for measuring a HUD ghost image, and more specifically, to a method and device for measuring the optical characteristics of a three-dimensional virtual image generated by an augmented reality device.

Augmented reality (AR) is a branch of virtual reality (AR) that is frequently used in digital media as a computer graphics technique that synthesizes virtual objects or information into the real environment to make them appear as if they existed in the original environment.

Augmented reality is also called mixed reality (MR) because it combines the real world with a virtual world with additional information in real time and displays it as a single image. It is a hybrid VR system that combines real and virtual environments, and research and development have been conducted mainly in the United States since the late 1990s.

For example, augmented reality can be used in remote medical diagnosis, broadcasting, architectural design, manufacturing process management, etc. In addition, with the recent widespread use of smartphones, it has entered the full-scale commercialization phase, and various products are being developed in the gaming and mobile solution industries and in the education sector.

Meanwhile, wearable computers can be used to realize augmented reality outdoors. In particular, head mounted displays (HMDs) enable augmented reality by overlaying computer graphics and text on the user's actual environment in real time. In addition, head up displays (HUDs) enable augmented reality by showing various information necessary for driving the vehicle on the outside of the vehicle's windshield.

For example, a head-up display can implement augmented reality by projecting light from inside the vehicle to the outside of the windshield on a virtual plane located outside the windshield of the vehicle, thereby allowing the driver to obtain information necessary for driving the vehicle on that virtual plane without moving his or her gaze while driving.

At this time, the geometric characteristics, including the position of the virtual plane formed by the augmented reality device, can be determined according to the optical characteristics of the individual augmented reality device, such as HMD and HUD.

Therefore, there is a growing need for a method and device capable of measuring optical characteristics of the output of an augmented reality device.

Related prior art includes Korean Patent Publication No. 10-2017-0114375 (Title of invention: Virtual reality content display method and device, publication date: October 16, 2017).

The present invention seeks to provide a method and device for measuring optical characteristics of a virtual image generated by an augmented reality device.

In addition, the present invention seeks to provide a method and device for calculating, based on a user of an augmented reality device, a virtual image distance, a look down/up angle, a horizontal/vertical field of view, static distortion, a ghosting level, etc. of the virtual image, by utilizing the optical characteristics of a virtual image generated by an augmented reality device.

The problems to be solved by the present invention are not limited to the problem(s) mentioned above, and other problem(s) not mentioned will be clearly understood by those skilled in the art from the description below.

In order to achieve the above object, the method for measuring optical characteristics of an augmented reality device provided in the present invention includes the steps of: capturing a test image including a plurality of patterns output on a virtual plane by an augmented reality device using a plurality of cameras arranged around a predetermined measurement reference position; obtaining angle information including information regarding angles of view of the plurality of cameras and arrangement information including information regarding arrangements of the plurality of cameras; and calculating coordinates of the plurality of patterns based on the plurality of captured images captured by the plurality of cameras, the angle information, and the arrangement information.

Preferably, when the plurality of cameras are a central camera positioned at the measurement reference position, a left camera and a right camera positioned symmetrically with respect to the measurement reference position, and the plurality of patterns are arranged in a horizontal and vertical alignment on the test image, the step of calculating the coordinates of the plurality of patterns may calculate the coordinates of the plurality of patterns by using the horizontal pixel number of the plurality of captured images, the coordinates of the plurality of patterns in the plurality of captured images, the angles of view of the plurality of cameras included in the angle of view information, and the distance between the left camera and the right camera included in the arrangement information.

Preferably, the step of calculating the coordinates of the plurality of patterns can calculate the coordinates of the plurality of patterns using mathematical expression 1.

[Mathematical Formula 1]

Here, x _ij , y _ij , z _ij are the x, y, z-axis coordinates of the ith horizontal and jth vertical pattern based on the measurement reference position, α is the distance between the left camera and the right camera, M is the horizontal pixel number of the plurality of captured images, θ is the angle of view of the plurality of cameras, m ^L _ij is the horizontal direction coordinate of the ith horizontal and jth vertical pattern in the captured image of the left camera, m ^R _ij is the horizontal direction coordinate of the ith horizontal and jth vertical pattern in the captured image of the right camera, and m ^C _ij is the horizontal direction coordinate of the ith horizontal and jth vertical pattern in the captured image of the center camera.

Preferably, the method may further include a step of calculating a virtual image distance between the measurement reference position and the virtual plane by using coordinates of the measurement reference position and coordinates of at least one of the plurality of patterns on the virtual plane.

Preferably, the step of calculating the virtual image distance can calculate the virtual image distance using mathematical expression 2.

[Mathematical formula 2]

Here, D _VI is the virtual image distance, and x ₂₂ , y ₂₂ , and z ₂₂ are coordinates of one pattern among the plurality of patterns.

Preferably, the method may further include a step of calculating a look down/up angle from the measurement reference position to the virtual plane using coordinates of the measurement reference position and coordinates of at least one of the plurality of patterns on the virtual plane.

Preferably, the step of calculating the lookdown/up angle can calculate the lookdown/up angle using mathematical expression 3.

[Mathematical Formula 3]

Here, θ _down/up is the lookdown/up angle, and x ₂₂ , y ₂₂ , z ₂₂ are coordinates of one pattern among the plurality of patterns.

Preferably, the method may further include a step of calculating a horizontal field of view of the measurement reference position by using coordinates of the measurement reference position and coordinates of two patterns located at opposite ends in the horizontal direction among the plurality of patterns on the virtual plane.

Preferably, the step of calculating the horizontal viewing angle can calculate the horizontal viewing angle using mathematical expression 4.

[Mathematical formula 4]

Here, θ ^H _FOV is the horizontal field of view, O is the coordinate of the measurement reference position, and P ₂₁ and P ₂₃ are the coordinates of two patterns located at both ends in the horizontal direction.

Preferably, the method may further include a step of calculating a vertical field of view of the measurement reference position by using coordinates of the measurement reference position and coordinates of two patterns located at opposite ends in the vertical direction among the plurality of patterns on the virtual plane.

Preferably, the step of calculating the vertical viewing angle can calculate the vertical viewing angle using mathematical expression 5.

[Mathematical Formula 5]

Here, θ ^V _FOV is the vertical field of view, O is the coordinate of the measurement reference position, and P ₁₂ and P ₃₂ are the coordinates of two patterns located at both ends in the vertical direction.

Preferably, the method may further include a step of calculating static distortion for each of three axes based on the measurement reference position, based on the coordinates of the plurality of patterns on the virtual plane.

Preferably, the step of calculating coordinates of the plurality of patterns may further include the step of calculating coordinates of a plurality of ghost patterns corresponding to each of the plurality of patterns, and calculating a ghosting level based on the coordinates of the plurality of patterns and the coordinates of the plurality of ghost patterns.

In addition, in order to achieve the above object, the optical characteristic measuring device of the augmented reality device provided in the present invention includes a photographing unit which photographs a test image including a plurality of patterns output on a virtual plane by the augmented reality device using a plurality of cameras arranged around a predetermined measurement reference position; an obtaining unit which obtains angle information including information regarding angles of view of the plurality of cameras and arrangement information including information regarding arrangement of the plurality of cameras; and a calculating unit which calculates coordinates of the plurality of patterns based on the plurality of photographed images photographed by the plurality of cameras, the angle information, and the arrangement information.

Preferably, when the plurality of cameras are a central camera positioned at the measurement reference position, a left camera and a right camera positioned symmetrically with respect to the measurement reference position, and the plurality of patterns are arranged in a horizontal and vertical alignment on the test image, the calculation unit can calculate the coordinates of the plurality of patterns by using the horizontal pixel number of the plurality of captured images, the coordinates of the plurality of patterns in the plurality of captured images, the angles of view of the plurality of cameras included in the angle of view information, and the distance between the left camera and the right camera included in the arrangement information.

Preferably, the output unit can output the coordinates of the plurality of patterns using mathematical expression 6.

[Mathematical Formula 6]

Here, x _ij , y _ij , z _ij are the x, y, z-axis coordinates of the ith horizontal and jth vertical pattern based on the measurement reference position, α is the distance between the left camera and the right camera, M is the horizontal pixel number of the plurality of captured images, θ is the angle of view of the plurality of cameras, m ^L _ij is the horizontal direction coordinate of the ith horizontal and jth vertical pattern in the captured image of the left camera, m ^R _ij is the horizontal direction coordinate of the ith horizontal and jth vertical pattern in the captured image of the right camera, and m ^C _ij is the horizontal direction coordinate of the ith horizontal and jth vertical pattern in the captured image of the center camera.

Preferably, the calculation unit can further calculate a virtual image distance between the measurement reference position and the virtual plane by using the coordinates of the measurement reference position and the coordinates of at least one of the plurality of patterns on the virtual plane.

Preferably, the output unit can calculate the virtual image distance using mathematical expression 7.

[Mathematical formula 7]

Here, D _VI is the virtual image distance, and x ₂₂ , y ₂₂ , and z ₂₂ are coordinates of one pattern among the plurality of patterns.

Preferably, the calculation unit can further calculate a lookdown/up angle for the virtual plane from the measurement reference position by using the coordinates of the measurement reference position and the coordinates of at least one of the plurality of patterns on the virtual plane.

Preferably, the output unit can calculate the lookdown/up angle using mathematical expression 8.

[Mathematical formula 8]

Here, θ _down/up is the lookdown/up angle, and x ₂₂ , y ₂₂ , z ₂₂ are coordinates of one pattern among the plurality of patterns.

Preferably, the calculation unit can further calculate the horizontal viewing angle of the measurement reference position by using the coordinates of the measurement reference position and the coordinates of two patterns located at both ends in the horizontal direction among the plurality of patterns on the virtual plane.

Preferably, the output unit can calculate the horizontal viewing angle using mathematical expression 9.

[Mathematical formula 9]

Here, θ ^H _FOV is the horizontal field of view, O is the coordinate of the measurement reference position, and P ₂₁ and P ₂₃ are the coordinates of two patterns located at both ends in the horizontal direction.

Preferably, the calculation unit can further calculate the vertical viewing angle of the measurement reference position by using the coordinates of the measurement reference position and the coordinates of two patterns located at both ends in the vertical direction among the plurality of patterns on the virtual plane.

Preferably, the output unit can calculate the vertical viewing angle using mathematical expression 10.

[Mathematical Formula 10]

Here, θ ^V _FOV is the vertical field of view, O is the coordinate of the measurement reference position, and P ₁₂ and P ₃₂ are the coordinates of two patterns located at both ends in the vertical direction.

Preferably, the output unit can further output static distortion for each of three axes based on the measurement reference position, based on the coordinates of the plurality of patterns on the virtual plane.

Preferably, the calculating unit may further calculate coordinates of a plurality of ghost patterns corresponding to each of the plurality of patterns based on the plurality of photographed images, the angle information, and the arrangement information, and may further calculate a ghosting level based on the coordinates of the plurality of patterns and the coordinates of the plurality of ghost patterns.

A method for measuring optical characteristics according to embodiments further comprises the steps of: generating images including points in each pattern for a virtual image plane using one or more optical measurement devices, each image captured based on the one or more optical measurement devices, each image corresponding to at least one of a left image, a center image, and a right image; and generating positions of the points based on the one or more optical measurement devices and each pattern; and generating a level for a ghost image for the virtual image plane based on the generated positions, wherein the positions are obtained based on a field of view of the one or more optical measurement devices, a gap between the left optical measurement device and the right optical measurement device.

The present invention has the effect of easily measuring the optical characteristics of a virtual image generated by an augmented reality device by using a plurality of cameras.

In addition, the present invention has the effect of being able to calculate, based on a user of the augmented reality device, a virtual image distance, a look down/up angle, a horizontal/vertical field of view, static distortion, a ghosting level, etc. of the virtual image by utilizing the optical characteristics of a virtual image generated by the augmented reality device.

FIG. 1 is a flowchart illustrating a method for measuring optical characteristics of an augmented reality device according to one embodiment of the present invention.
FIG. 2 is a flowchart illustrating a method for calculating a virtual image distance according to one embodiment of the present invention.
FIG. 3 is a flowchart illustrating a lookdown/up angle calculation method according to one embodiment of the present invention.
Figure 4 is a flowchart showing a method for calculating a horizontal viewing angle according to one embodiment of the present invention.
Figure 5 is a flowchart showing a method for calculating a vertical viewing angle according to one embodiment of the present invention.
Figure 6 is a flowchart illustrating a static distortion calculation method according to one embodiment of the present invention.
Figure 7 is a flowchart illustrating a ghosting level calculation method according to one embodiment of the present invention.
FIG. 8 is a block diagram showing an optical characteristic measuring device of an augmented reality device according to one embodiment of the present invention.
FIGS. 9A and 9B are drawings for explaining an environment for measuring optical characteristics of an augmented reality device according to one embodiment of the present invention.
FIG. 10 is a drawing for explaining the result of capturing a test image on a virtual plane using multiple cameras according to one embodiment of the present invention.
FIGS. 11A and 11B are drawings for explaining coordinates of multiple patterns included in a photographed image captured using multiple cameras according to one embodiment of the present invention.
FIGS. 12a and 12b are drawings for explaining a method for calculating coordinates of a plurality of patterns according to one embodiment of the present invention.
FIG. 13 is a drawing for explaining a method for calculating a virtual image distance according to one embodiment of the present invention.
FIGS. 14a and 14b are diagrams for explaining a method for calculating a lookdown/up angle according to one embodiment of the present invention.
FIGS. 15a and 15b are diagrams for explaining a method for calculating a horizontal viewing angle and a vertical viewing angle according to one embodiment of the present invention.
FIG. 16 is a drawing for explaining a method for calculating static distortion according to one embodiment of the present invention.
FIG. 17 is a diagram for explaining a method for calculating a ghosting level according to one embodiment of the present invention.
FIG. 18 shows a measurement configuration for image quality characteristics of a virtual image type 3D display such as a 3D HUD according to embodiments.
Figure 19 shows a measurement method for ghost images according to embodiments.
Figure 20 shows a test image with nine measurement points according to embodiments and three corresponding images captured by LMDs.
Fig. 21 illustrates a method for obtaining a ghost level for a ghost image according to embodiments.
Figure 22 shows an optical measurement method according to embodiments.
Figure 23 illustrates a setup configuration for ghost images and binocular misalignment according to embodiments.
Fig. 24 shows an optical measuring device according to embodiments.

The present invention can have various modifications and various embodiments, and specific embodiments are illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the present invention to specific embodiments, but should be understood to include all modifications, equivalents, or substitutes included in the spirit and technical scope of the present invention. In describing each drawing, similar reference numerals are used for similar components.

The terms first, second, A, B, etc. may be used to describe various components, but the components should not be limited by the terms. The terms are only used to distinguish one component from another. For example, without departing from the scope of the present invention, the first component may be referred to as the second component, and similarly, the second component may also be referred to as the first component. The term and/or includes any combination of a plurality of related described items or any item among a plurality of related described items.

When it is said that a component is "connected" or "connected" to another component, it should be understood that it may be directly connected or connected to that other component, but that there may be other components in between. On the other hand, when it is said that a component is "directly connected" or "directly connected" to another component, it should be understood that there are no other components in between.

The terminology used in this application is only used to describe specific embodiments and is not intended to limit the present invention. The singular expression includes the plural expression unless the context clearly indicates otherwise. In this application, it should be understood that the terms "comprises" or "has" and the like are intended to specify the presence of a feature, number, step, operation, component, part or combination thereof described in the specification, but do not exclude in advance the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts or combinations thereof.

Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. Terms defined in commonly used dictionaries, such as those defined in common dictionaries, should be interpreted as having a meaning consistent with the meaning they have in the context of the relevant art, and will not be interpreted in an idealized or overly formal sense unless expressly defined in this application.

Hereinafter, preferred embodiments according to the present invention will be described in detail with reference to the attached drawings.

The present invention relates to a method and apparatus for measuring optical characteristics of a virtual reality device, and the measurement can be performed in the following environment. For example, referring to FIG. 9a, a user's eyes may be positioned in an eye box, and a virtual plane by an output of the virtual reality device may be formed outside a transparent or translucent screen (e.g., a windshield of a vehicle). At this time, the user may view the entire virtual plane by moving only his or her eyes. In addition, referring to FIG. 9b, a plurality of cameras may be positioned in the eye box centered on a measurement reference position. More specifically, cam _C may be positioned at the measurement reference position, and cam _L and cam _R may be positioned at positions symmetrical to both sides thereof. Meanwhile, a plurality of patterns may be positioned in a test image while being aligned horizontally and vertically (e.g., 3x3).

However, the present invention is not limited to being implemented only in such an environment, and can of course be implemented in various different environments. For example, the position and size of the eye box, the number and arrangement of cameras, the number and arrangement of patterns included in the test image, etc. may vary depending on the measurement environment.

FIG. 1 is a flowchart illustrating a method for measuring optical characteristics of an augmented reality device according to one embodiment of the present invention.

In step S110, an optical characteristic measurement device captures a test image including a plurality of patterns output on a virtual plane by an augmented reality device using a plurality of cameras arranged around a predetermined measurement reference position.

For example, referring to Fig. 9b, one camera may be placed at a measurement reference position located at the center of the eye box, and the remaining cameras may be placed symmetrically facing the front at the same height on both sides.

At this time, the optical characteristic measurement device can be connected to multiple cameras wirelessly or via wires to transmit a command to capture a test image on a virtual plane.

In step S120, the optical characteristic measuring device obtains angle information including information about angles of view of the plurality of cameras and arrangement information including information about arrangement of the plurality of cameras.

For example, the optical characteristic measuring device can receive information about the angle of view of the camera and information about the arrangement of the camera from the user, and obtain angle of view information and arrangement information. Preferably, the information about the angle of view of the camera is the horizontal angle of view, and the information about the arrangement of the camera can be the distance between cameras arranged symmetrically on both sides of the measurement reference position.

Finally, in step S130, the optical characteristic measurement device calculates coordinates of a plurality of patterns based on a measurement reference position based on a plurality of photographed images, angle information, and arrangement information captured by the plurality of cameras.

At this time, the optical characteristic measuring device can calculate three-dimensional coordinates of multiple patterns on a virtual plane with the measurement reference position as the origin (0, 0, 0) by using information about the sizes of multiple captured images, information about coordinates of multiple patterns included in the multiple captured images within the images, information about the angles of view of multiple cameras, and information about the arrangement of multiple cameras.

Meanwhile, a detailed method for calculating coordinates of multiple patterns is specifically described in the examples below.

In another embodiment, when the optical characteristic measuring device has a central camera positioned at a measurement reference position, a left camera, and a right camera positioned symmetrically with respect to the measurement reference position, and a plurality of patterns are arranged in a horizontal and vertical alignment on a test image, the coordinates of the plurality of patterns can be calculated using the horizontal pixel count of the plurality of captured images, the coordinates of the plurality of patterns in the plurality of captured images, the angles of view of the plurality of cameras included in the angle of view information, and the distance between the left camera and the right camera included in the arrangement information.

For example, referring to Fig. 9b, multiple cameras may be a central camera (cam _C ) positioned at a measurement reference position, a left camera (cam _L ) and a right camera (cam _R ) positioned symmetrically about the measurement reference position. In addition, nine patterns may be arranged in a horizontal and vertical alignment on the test image.

At this time, the optical characteristic measuring device can calculate three-dimensional coordinates of nine patterns on a virtual plane with the measurement reference position as the origin (0, 0, 0) by using the horizontal pixel count of multiple captured images, the coordinates of multiple patterns in the multiple captured images, the angles of view of the multiple cameras included in the angle of view information, and the distance between the left camera and the right camera included in the arrangement information.

In another embodiment, the optical property measuring device can calculate coordinates of a plurality of patterns using mathematical expression 1.

[Mathematical Formula 1]

Here, x _ij , y _ij , z _ij are the x, y, z-axis coordinates of the ith horizontal and jth vertical pattern based on the measurement reference position, α is the distance between the left camera and the right camera, M is the horizontal pixel number of multiple captured images, θ is the field of view of the multiple cameras, m ^L _ij is the horizontal direction coordinate of the ith horizontal and jth vertical pattern in the captured image of the left camera, m ^R _ij is the horizontal direction coordinate of the ith horizontal and jth vertical pattern in the captured image of the right camera, and m ^C _ij is the horizontal direction coordinate of the ith horizontal and jth vertical pattern in the captured image of the center camera.

At this time, referring to FIG. 10, the central camera (cam _C ) may be placed at the measurement reference position, which is the center of the eye box, and the left camera (cam _L ) and the right camera (cam _R ) may be placed at a distance of α. Then, a test image on a virtual plane may be captured using the central camera (cam _C ), the left camera (cam _L ), and the right camera (cam _R ) arranged so that the optical characteristic measurement device faces the front. As a result, a captured image (captured image by cam _L ) captured using the left camera (cam _L ) may have a test image that is biased to the right, a captured image (captured image by cam _C ) captured using the central camera (cam _C ) may not have a biased test image, and a captured image (captured image by cam _R ) captured using the right camera (cam _R ) may have a test image that is biased to the left.

Meanwhile, referring to Fig. 11a, the three-dimensional coordinates of nine patterns appearing on the virtual plane can be expressed as P _{ij =} (x _ij , y _ij , z _ij ), where i can be the horizontal index of the pattern (i = 1, 2, 3) and j can be the vertical index of the pattern (j = 1, 2, 3). That is, P _ij can be the three-dimensional coordinates of the center of the i-th horizontal and j-th vertical pattern.

Also, referring to FIG. 11b, the pixel coordinates of the nine patterns appearing in the captured images can be represented as P ^L _ij , P ^C _ij , and P ^R _ij , which can mean the coordinates of the patterns appearing in the captured images of the left camera (cam _L ), the center camera (cam _C ), and the right camera (cam _R ), respectively. At this time, P ^L _ij = (m ^L _ij , n ^L _ij ), P ^C _ij = (m ^C _ij , n ^C _ij ), and P ^R _ij = (m ^R _ij , n ^R _ij ). At this time, P ^L _ij , P ^C _ij , and P ^R _ij can be the pixel coordinates of the center of the i-th horizontal and j-th vertical pattern.

Meanwhile, referring to Fig. 12a, it can be seen that a proportional relationship as in mathematical expression 2 below holds true.

[Mathematical formula 2]

Here, z is the distance from the measurement reference position to the virtual plane along the z-axis, θ is the field of view of the camera, α is the distance between the left camera and the right camera, m ^L _ij is the horizontal coordinate of the ith horizontal and jth vertical pattern in the image captured by the left camera, m ^R _ij is the horizontal coordinate of the ith horizontal and jth vertical pattern in the image captured by the right camera, and M is the horizontal pixel number of the captured image.

At this point, it is obvious that mathematical expression 1 can be obtained by transforming mathematical expression 2.

For example, referring to FIG. 12b, the optical characteristic measurement device can calculate x 11 , y 11 , and z 11 using mathematical expression 1 by using the results captured by the central camera (cam _C ), the left camera (cam _L ), and the right camera (cam _R ) regarding _the same pattern (i = ₁ , j = ₁₎ .

FIG. 2 is a flowchart illustrating a method for calculating a virtual image distance according to one embodiment of the present invention.

In step S210, an optical characteristic measurement device captures a test image including a plurality of patterns output on a virtual plane by an augmented reality device using a plurality of cameras arranged around a predetermined measurement reference position.

In step S220, the optical characteristic measurement device obtains angle information including information about angles of view of the plurality of cameras and arrangement information including information about arrangement of the plurality of cameras.

In step S230, the optical characteristic measuring device calculates coordinates of a plurality of patterns based on a measurement reference position based on a plurality of photographed images, angle information, and arrangement information captured by the plurality of cameras.

Finally, in step S240, the optical characteristic measurement device calculates a virtual image distance between the measurement reference position and the virtual plane using the coordinates of the measurement reference position and at least one of the coordinates of a plurality of patterns on the virtual plane.

For example, referring to FIG. 13, the optical characteristic measurement device can calculate the virtual image distance by calculating the distance to the coordinates (x ₂₂ , y ₂₂ , z ₂₂ ) of P ₂₂ based on the measurement reference position (0, 0, 0).

In another embodiment, the optical property measuring device can calculate the virtual image distance using mathematical expression 3.

[Mathematical Formula 3]

Here, D _VI is the virtual image distance, and x ₂₂ , y ₂₂ , z ₂₂ are the 3D coordinates of the pattern where i = 2, j = 2.

FIG. 3 is a flowchart illustrating a lookdown/up angle calculation method according to one embodiment of the present invention.

In step S310, an optical characteristic measurement device captures a test image including a plurality of patterns output on a virtual plane by an augmented reality device using a plurality of cameras arranged around a predetermined measurement reference position.

In step S320, the optical characteristic measurement device obtains angle information including information about angles of view of the plurality of cameras and arrangement information including information about arrangement of the plurality of cameras.

In step S330, the optical characteristic measuring device calculates coordinates of a plurality of patterns based on a measurement reference position based on a plurality of photographed images, angle information, and arrangement information captured by the plurality of cameras.

Finally, in step S340, the optical characteristic measurement device calculates a look down/up angle from the measurement reference position to the virtual plane using the coordinates of the measurement reference position and at least one coordinate of a plurality of patterns on the virtual plane.

At this time, the lookdown/up angle is an angle that represents the difference in height between the eye box and the virtual plane, indicating whether the user is looking up or down at the virtual plane.

For example, when the coordinates (x ₂₂ , y ₂₂ , z ₂₂ ) of P ₂₂ are calculated based on the measurement reference position (0, 0, 0) where the user's eyes are located, if y ₂₂ < 0, it can be a look down situation as in Fig. 14a, and if y ₂₂ > 0, it can be a look up situation as in Fig. 14b.

In another embodiment, the optical property measurement device can calculate the lookdown/up angle using Equation 4.

[Mathematical Formula 4]

Here, θ _down/up is the lookdown/up angle, and x ₂₂ , y ₂₂ , z ₂₂ are the 3D coordinates of the pattern where i=2, j=2.

Figure 4 is a flowchart showing a method for calculating a horizontal viewing angle according to one embodiment of the present invention.

In step S410, an optical characteristic measurement device captures a test image including a plurality of patterns output on a virtual plane by an augmented reality device using a plurality of cameras arranged around a predetermined measurement reference position.

In step S420, the optical characteristic measurement device obtains angle information including information about angles of view of the plurality of cameras and arrangement information including information about arrangement of the plurality of cameras.

In step S430, the optical characteristic measuring device calculates coordinates of a plurality of patterns based on a measurement reference position based on a plurality of photographed images, angle information, and arrangement information captured by the plurality of cameras.

Finally, in step S440, the optical characteristic measuring device calculates a horizontal field of view of the measurement reference position using coordinates of the measurement reference position and coordinates of two patterns located at both ends in the horizontal direction among a plurality of patterns on the virtual plane.

For example, referring to FIG. 15a, the optical characteristic measurement device can calculate the angle ∠P _{21 OP 23 as a horizontal viewing angle by using the three-dimensional coordinates O = (0, 0, 0) of the measurement reference position, and the three-dimensional coordinates of two patterns P 21} = (x ₂₁ , y ₂₁ , z ₂₁ ) and P ₂₃ = (x ₂₃ , y ₂₃ , z ₂₃ ), which are located at opposite ends in the horizontal direction among the _plurality of _patterns on the virtual plane.

In another embodiment, the optical property measuring device can calculate the horizontal viewing angle using Equation 5.

[Mathematical Formula 5]

Here, θ ^H _FOV is the horizontal field of view, O is the coordinate of the measurement reference position, and P ₂₁ and P ₂₃ are the coordinates of two patterns located at both ends in the horizontal direction among multiple patterns.

Figure 5 is a flowchart showing a method for calculating a vertical viewing angle according to one embodiment of the present invention.

In step S510, an optical characteristic measurement device captures a test image including a plurality of patterns output on a virtual plane by an augmented reality device using a plurality of cameras arranged around a predetermined measurement reference position.

In step S520, the optical characteristic measurement device obtains angle information including information about angles of view of the plurality of cameras and arrangement information including information about arrangement of the plurality of cameras.

In step S530, the optical characteristic measuring device calculates coordinates of a plurality of patterns based on a measurement reference position based on a plurality of photographed images, angle information, and arrangement information captured by the plurality of cameras.

Finally, in step S540, the optical characteristic measuring device calculates a vertical field of view of the measurement reference position using coordinates of the measurement reference position and coordinates of two patterns located at both ends in the vertical direction among a plurality of patterns on the virtual plane.

For example, referring to FIG. 15b, the optical characteristic measurement device can calculate the angle ∠P 12 OP 32 as a vertical viewing angle by using the three-dimensional coordinates O = (0, 0, 0) of the measurement reference position, and the three-dimensional coordinates of two patterns P ₁₂ = (x ₁₂ , y ₁₂ , z ₁₂ ) and P ₃₂ = (x ₃₂ , y ₃₂ , z ₃₂ ), which are located at opposite ends in the vertical direction among the plurality _{of patterns} on the virtual plane.

In another embodiment, the optical characteristic measurement device can calculate the vertical viewing angle using Equation 6.

[Mathematical Formula 6]

Here, θ ^V _FOV is the vertical field of view, O is the coordinate of the measurement reference position, and P ₁₂ and P ₃₂ are the coordinates of two patterns located at both ends in the vertical direction.

Figure 6 is a flowchart illustrating a static distortion calculation method according to one embodiment of the present invention.

In step S610, an optical characteristic measurement device captures a test image including a plurality of patterns output on a virtual plane by an augmented reality device using a plurality of cameras arranged around a predetermined measurement reference position.

In step S620, the optical characteristic measurement device obtains angle information including information about angles of view of the plurality of cameras and arrangement information including information about arrangement of the plurality of cameras.

In step S630, the optical characteristic measuring device calculates coordinates of a plurality of patterns based on a measurement reference position based on a plurality of photographed images, angle information, and arrangement information captured by the plurality of cameras.

Finally, in step S640, the optical characteristic measuring device calculates static distortion for each of three axes based on the measurement reference position, based on the coordinates of a plurality of patterns on the virtual plane.

At this time, static distortion is caused by the projection of the virtual reality device, and as shown in Fig. 16, it represents the deviation degree of the three-dimensional coordinates of multiple patterns based on the line corresponding to each of the three axes (x, y, z).

Meanwhile, the optical property measuring device can calculate the static distortion for each of the three axes using mathematical expression 7.

[Mathematical formula 7]

Here, DT ^x _Linearity , DT ^y _Linearity , and DT ^z _Linearity are linear distortion values based on the x, y, and z axes, respectively, and x _ab , y _ab , and z _ab are the x, y, and z coordinates of the a-th (a=1,2,3)th horizontal and b-th (b=1,2,3)th vertical pattern.

Figure 7 is a flowchart illustrating a ghosting level calculation method according to one embodiment of the present invention.

In step S710, an optical characteristic measurement device captures a test image including a plurality of patterns output on a virtual plane by an augmented reality device using a plurality of cameras arranged around a predetermined measurement reference position.

In step S720, the optical characteristic measurement device obtains angle information including information about angles of view of the plurality of cameras and arrangement information including information about arrangement of the plurality of cameras.

In step S730, the optical characteristic measuring device calculates coordinates of a plurality of patterns based on a measurement reference position and coordinates of a plurality of ghost patterns based on a plurality of photographed images, angle information, and arrangement information captured by the plurality of cameras.

For example, a ghost pattern may appear on a windshield of a vehicle that transmits half of the incoming light and reflects the other half. More specifically, referring to FIG. 17, two physical layers of the windshield may cause a ghosting phenomenon, causing the user to see double images of a pattern on a virtual plane and a corresponding ghost pattern, or the pattern may appear blurred.

At this time, the optical characteristic measuring device can calculate the coordinates of multiple ghost patterns corresponding to each of the multiple patterns in the same way as the method of calculating the coordinates of the multiple patterns.

Finally, in step S740, the optical characteristic measuring device can calculate a ghosting level based on the coordinates of the plurality of patterns and the coordinates of the plurality of ghost patterns.

At this time, the optical characteristic measuring device can calculate the ghosting level from the difference (gap) between the original pattern and the corresponding ghost pattern.

More specifically, the optical characteristic measurement device can calculate the ghosting level using mathematical expression 8.

[Mathematical formula 8]

Here, Ghost is the ghosting level, x _ij , y _ij , z _ij are the x, y, z coordinates of the i-th (i=1,2,3)th horizontal and j-th (j=1,2,3)th vertical pattern, and x _Gij , y _Gij , z _Gij are the x, y, z coordinates of the i-th horizontal and j-th vertical ghost pattern.

FIG. 8 is a block diagram showing an optical characteristic measuring device of an augmented reality device according to one embodiment of the present invention.

Referring to FIG. 8, an optical characteristic measuring device (800) of an augmented reality device according to one embodiment of the present invention includes a photographing unit (810), an acquisition unit (820), and a calculation unit (830).

The camera unit (810) uses multiple cameras arranged around a predetermined measurement reference position to capture a test image including multiple patterns output on a virtual plane by an augmented reality device.

The acquisition unit (820) acquires angle information including information about the angles of view of the plurality of cameras and arrangement information including information about the arrangement of the plurality of cameras.

Finally, the output unit (830) calculates coordinates of multiple patterns based on the measurement reference position based on multiple shooting images, angle information, and arrangement information captured by the multiple cameras.

In another embodiment, when a plurality of cameras are a central camera positioned at a measurement reference position, a left camera and a right camera positioned symmetrically about the measurement reference position, and a plurality of patterns are arranged in a horizontal and vertical alignment on a test image, the calculation unit (830) can calculate the coordinates of the plurality of patterns by using the horizontal pixel count of the plurality of captured images, the coordinates of the plurality of patterns in the plurality of captured images, the angles of view of the plurality of cameras included in the angle of view information, and the distance between the left camera and the right camera included in the arrangement information.

In another embodiment, the output unit (830) can output coordinates of multiple patterns using mathematical expression 9.

[Mathematical formula 9]

Here, x _ij , y _ij , z _ij are the x, y, z-axis coordinates of the ith horizontal and jth vertical pattern based on the measurement reference position, α is the distance between the left camera and the right camera, M is the horizontal pixel number of multiple captured images, θ is the field of view of the multiple cameras, m ^L _ij is the horizontal direction coordinate of the ith horizontal and jth vertical pattern in the captured image of the left camera, m ^R _ij is the horizontal direction coordinate of the ith horizontal and jth vertical pattern in the captured image of the right camera, and m ^C _ij is the horizontal direction coordinate of the ith horizontal and jth vertical pattern in the captured image of the center camera.

In another embodiment, the calculation unit (830) can further calculate a virtual image distance between the measurement reference position and the virtual plane by using the coordinates of the measurement reference position and at least one of the coordinates of a plurality of patterns on the virtual plane.

In another embodiment, the calculation unit (830) can calculate the virtual image distance using mathematical expression 10.

[Mathematical formula 10]

Here, D _VI is the virtual image distance, and x ₂₂ , y ₂₂ , and z ₂₂ are the coordinates of one pattern among multiple patterns.

In another embodiment, the calculation unit (830) can further calculate a lookdown/up angle from the measurement reference position to the virtual plane using the coordinates of the measurement reference position and at least one of the coordinates of a plurality of patterns on the virtual plane.

In another embodiment, the calculation unit (830) can calculate the lookdown/up angle using mathematical expression 11.

[Mathematical Formula 11]

Here, θ _down/up is the lookdown/up angle, and x ₂₂ , y ₂₂ , z ₂₂ are the coordinates of one pattern among multiple patterns.

In another embodiment, the calculation unit (830) can further calculate the horizontal viewing angle of the measurement reference position by using the coordinates of the measurement reference position and the coordinates of two patterns located at both ends in the horizontal direction among a plurality of patterns on the virtual plane.

In another embodiment, the calculating unit (830) can calculate the horizontal viewing angle using mathematical expression 12.

[Mathematical formula 12]

Here, θ ^H _FOV is the horizontal field of view, O is the coordinate of the measurement reference position, and P ₂₁ and P ₂₃ are the coordinates of two patterns located at both ends in the horizontal direction.

In another embodiment, the calculation unit (830) can further calculate the vertical viewing angle of the measurement reference position by using the coordinates of the measurement reference position and the coordinates of two patterns located at both ends in the vertical direction among a plurality of patterns on the virtual plane.

In another embodiment, the calculating unit (830) can calculate the vertical viewing angle using mathematical expression 13.

[Mathematical formula 13]

Here, θ ^V _FOV is the vertical field of view, O is the coordinate of the measurement reference position, and P ₁₂ and P ₃₂ are the coordinates of two patterns located at both ends in the vertical direction.

In another embodiment, the generating unit (830) can further generate static distortions for each of three axes based on the measurement reference position, based on coordinates of a plurality of patterns on the virtual plane.

In another embodiment, the calculation unit (830) may further calculate coordinates of a plurality of ghost patterns corresponding to each of a plurality of patterns based on a plurality of captured images, angle information, and arrangement information, and may further calculate a ghosting level based on the coordinates of the plurality of patterns and the coordinates of the plurality of ghost patterns.

Meanwhile, the embodiments of the present invention described above can be written as a program that can be executed on a computer, and can be implemented in a general-purpose digital computer that operates the program using a computer-readable recording medium.

The computer-readable recording medium includes a magnetic storage medium (e.g., ROM, floppy disk, hard disk, etc.) and an optical reading medium (e.g., CD-ROM, DVD, etc.).

The present invention has been described with reference to preferred embodiments thereof. Those skilled in the art will appreciate that the present invention may be implemented in modified forms without departing from the essential characteristics of the present invention. Therefore, the disclosed embodiments should be considered from an illustrative rather than a restrictive perspective. The scope of the present invention is indicated by the claims, not the foregoing description, and all differences within the scope equivalent thereto should be interpreted as being included in the present invention.

Referring to FIGS. 9-10, a camera according to embodiments may correspond to a light measuring device (LMD). The light measuring device according to embodiments may generate a virtual image plane and generate images including patterns at different locations. An optical characteristic measuring method according to embodiments includes a step of generating images including points in each pattern for a virtual image plane using one or more light measuring devices, each image being captured based on the one or more light measuring devices, and each image corresponding to at least one of a left image, a center image, and a right image; and a step of generating positions of points based on the one or more light measuring devices and each pattern; wherein the positions may be obtained based on a field of view of the one or more light measuring devices, a gap between the left light measuring device and the right light measuring device.

Referring to FIG. 10, one LDM can capture three images for a virtual plane at the center, left, and right positions, and multiple LDMs can capture three images for a virtual plane at the center, left, and right positions.

Referring to Fig. 12, the positions of the points can be estimated based on, for example, the capture angle for the left LDM position and the capture angle for the right LDM position.

Based on mathematical expression 1, the coordinate values of the position for the pattern of the image can be calculated.

Referring to Mathematical Formula 1, by the optical characteristic measuring method/device according to the embodiments, a left image including points in a pattern may be captured based on a left optical measuring device of one or more optical measuring devices, a central image including points in a pattern may be captured based on a central optical measuring device of one or more optical measuring devices, and a right image including points in a pattern may be captured based on a right optical measuring device of one or more optical measuring devices. An index according to the embodiments may indicate a number order, and a field of view may correspond to an angle of view.

Referring to FIG. 13 and mathematical expression 2, the coordinate values of the position can be calculated based on the horizontal pixel index of the left light measurement device, the horizontal pixel index of the right light measurement device, the horizontal pixel index of the center light measurement device, and the field of view of the left light measurement device.

Referring to FIG. 14 and mathematical expression 4, the optical characteristic measurement method according to the embodiments may further include a step of measuring a virtual image distance for a virtual image plane based on a position within the pattern at the center and an optical measurement device.

Referring to FIG. 15, the method may further include a step of measuring a look down angle and a look up angle for a virtual image plane based on a position within the pattern and a virtual image distance.

Referring to FIG. 16, the method may further include: a step of measuring a horizontal field of view for the virtual image plane based on a distance to a left point in the center and a distance to a right point in the center; and a step of measuring a vertical field of view for the virtual image plane based on a distance to a top point in the center and a distance to a bottom point in the center.

A method for measuring optical characteristics according to embodiments may include a step of measuring horizontal distortion for a virtual image plane based on a line between a center and a center top point and a center bottom point; and a step of measuring vertical distortion for the virtual image plane based on a line between a center and a center left point and a center right point.

The optical specific measurement method/device according to the embodiments may be referred to as the method/device according to the embodiments.

FIG. 18 shows a measurement configuration for image quality characteristics of a virtual image type 3D display such as a 3D HUD according to embodiments.

The optical characteristic measuring device according to the embodiments of FIGS. 8 and 23 can measure the optical characteristics of a virtual image having a structure such as that of FIG. 18.

The method/device according to the embodiments may include the following configuration for measurement of image quality characteristics.

Referring to Figure 18, the geometric relationship between the eye box, the virtual image, and the white diffuser can be seen.

When the user's eyes are positioned in the eye box, it is assumed that the user can see the entire virtual image with the natural rolling movement of the eyes. The eye box position can be specified by the supplier. Otherwise it can be estimated according to the method provided in IEC 62629-62-11 (see especially 4.2.3). The measuring device of the imaging LMD can be set within the eye box position. The 3D image can be provided in the front or back in the virtual image plane. The 3D coordinate system of xyz as shown in Fig. 18 is defined to determine the position of the 3D image and the virtual image plane from the user's eyes.

Referring to Fig. 18, the designed viewing distance is the distance between the center of the eye box and the position of the half mirror. It can be presented by the supplier. At this distance, the proper view is observed or the image quality characteristics of the virtual image reproduced by the autostereoscopic 3D display are accurately measured. For the measurement, the viewing distance can be applied as the measurement distance. The measurement distance can be fixed when measuring the item to be evaluated.

The configuration and condition measurements for contrast and chromaticity of 3D virtual overlay under ambient conditions are as follows.

Various virtual image contents such as numbers, letters, and other symbols can be played back as 3D displays in the form of virtual images, such as 3D HUDs. In general, clear 3D virtual images can be displayed for the real environment. In order to evaluate image quality from this perspective, contrast and color-related properties should be measured for the 3D virtual images overlapping the real surround.

However, the real environment is very diverse. The background effect, in which the color and brightness characteristics of the 3D virtual image appear different from those of the adjacent real objects compared to the adjacent real objects, is a visual perception problem and is therefore excluded from this criterion. In order to improve the visibility, the brightness of the 3D virtual image can be controlled according to a predetermined algorithm embedded in the 3D HUD according to the change in the illumination level of the actual surround *?*. Therefore, a measurement method is proposed to evaluate the change in contrast (reflecting the change in black and white luminance) and chromaticity of the 3D virtual image under various ambient environmental conditions. The illumination level and the correlated color temperature for the ambient environmental conditions can be referenced.

The room where the measurement is performed is a completely dark room. The white diffuser is positioned behind the virtual image plane with a size larger than the horizontal and vertical FOV of the virtual image plane. Since the entire virtual plane can be observed when the eye is inside the eyebox, the white diffuser screen size can also be large enough to overlap the entire virtual plane if the LMD is inside the eyebox.

The illuminance and correlated color temperature of the ambient conditions are measured at the center of the white diffuser. The luminance reflected by the white diffuser is measured with zero input applied to the display to be measured. This measurement is evaluated for the presence of stray light.

Figure 19 shows a measurement method for a ghost image according to embodiments.

Figure 20 shows a test image with nine measurement points according to embodiments and three corresponding images captured by LMDs.

The optical characteristic measuring device according to the embodiments of FIGS. 8 and 23 can measure optical characteristics based on the above-described embodiments and/or the method described in FIGS. 18 and below. Specifically, it can generate measurement values for a ghost image related to a HUD.

The 3D illusion (virtual image) is generated through a process of reflection and magnification through the half mirror and optical system of the 3D HUD of Fig. 18. A certain amount of light reaching the mirror is reflected and the rest is transmitted outside the mirror within the half mirror, such as the windshield of Fig. 18. The two physical layers of the windshield cause a ghosting image that appears as a double image with shadows and outlines. The ghosting level can be measured from the gap between the original pattern and the second pattern.

The conditions for the measurement method according to the examples are as follows.

a) Test pattern: Test image with 9 circles in Fig. 20

b) Test signals: nine circles with black borders and central crosses against a completely white background;

c) ambient surround dark surround condition; and

d) Test Pattern Image Acquisition: Three imaging LMDs in the eyebox are used to capture three sets of 2D images, namely the left, center and right images in Fig. 19.

The flow chart regarding the measurement method according to the embodiments is as follows.

a) Apply a test signal.

b) Three test pattern images are acquired by three imaging LMDs, LMD(L), LMD(C) and LMD(R) of Fig. 19.

c) For all positions (xij, yij, zij)ij=1,2,3 for P11 to P33 of the original test pattern and for the corresponding positions (xGij, yGij, zGij)ij=1,2,3 for PG11 to PG33, the ghost pattern is determined according to the procedure described above.

d) The level of the ghost image is calculated as the average of the distance or angle between the original and the corresponding location for P11 to P33 according to Fig. 21.

Fig. 21 illustrates a method for obtaining a ghost level for a ghost image according to embodiments.

The method/device according to the embodiments can generate a ghost level (distance) and a ghost level (angle) based on the difference value between the positions of the test pattern and the positions of the ghost patterns of the ghost image, as in FIG. 21, by the flowchart described in FIG. 20.

Optical measurements related to ghost images according to embodiments can be performed based on the levels obtained as described above.

Figure 22 shows an optical measurement method according to embodiments.

The optical measurement method according to the embodiments may include the following flow chart.

An optical measurement method according to embodiments of the S2200 may include a step of generating a test signal.

The optical measurement method according to embodiments of S2201 may further include a step of obtaining a test pattern for a virtual image.

The optical measurement method according to the embodiments of S2202 may further include a step of generating positions of a test pattern and positions of a ghost pattern for a virtual image. The method of generating positions refers to the description of FIGS. 1 to 17 described above.

The optical measurement method according to the S2203 embodiments may further include a step of generating a ghost level. The method of obtaining the ghost level refers to the description of FIGS. 18 to 21 described above.

The step (S2203) of generating a ghost level of the optical measurement method according to the embodiments further includes the following operations: A general measurement method applied to ghost images and binocular misalignment.

Due to binocular misalignment, the position of the 3D virtual image may differ depending on the positions of the left and right eyes. Ghost images appear as shadows or outlines around the original image. The degree of ghosting and binocular misalignment is evaluated using the position information of the 3D virtual test image. Therefore, the embodiments provide a measurement method for determining the position of the 3D virtual test image based on the user's eyes.

The measurement configuration for position estimation of the embodiments is as follows:

Figure 23 illustrates a setup configuration for ghost images and binocular misalignment according to embodiments.

Referring to Fig. 23, there is illustrated a configuration of a three-dimensional imaging LMD for evaluating the level of ghosting and the degree of misalignment between the two eyes, and a test pattern displayed on a virtual image plane in a three-dimensional xyz coordinate system. Except for the test pattern, the computational method for estimating the three-dimensional virtual image geometry is the same in this standard and IEC 62629-62-11. The test pattern in Fig. 22 (a circle 2201 with a central cross on a shaded background 220) is different from the pattern applied to measure the shape of the IEC 62629-62-3D virtual image (a circle with a central cross on a background).

The center of the Eye Box is defined as the origin (x = 0, y = 0, z = 0), which is the location where the center of the entrance pupil of the imaging LMD is located (LMD _C ). The spacing (α) between the left (LMD _L ) and right (LMD _R ) imaging LMDs is assumed to be equal to the user's interpupillary distance (IPD). An IPD of 60 mm or 65 mm can be used for this spacing. The IPD range of adults is 55–70 mm, and the average IPD values of adult males and females with healthy eyes are 65 mm and 60 mm, respectively. The distance between LMD _L and LMD _R is denoted by a . If only one imaging LMD is used instead of three, measurements can be performed by moving one of the imaging LMDs. The measurement points are the centers of each of the nine circles of whiteboard lines in the test pattern. These points are denoted P _ij (i and j = 1, 2, 3). P _ij can be expressed as (x _ij , y _ij , z _ij ) in the xyz coordinate system.

The step of generating a ghost level (S2203) may be referred to as a method of measuring a ghost image.

As shown in Fig. 18, a certain amount of light reaching the mirror is reflected by a half mirror such as a windshield, and the rest is transmitted out of the mirror. Ghosting images appear as unintended low-intensity copies due to the two physical surfaces of the windshield. The level of ghosting can be measured by the gap between the original and the copy pattern.

In most cases, the glass window is double-layered, so a single-layer ghost image is observed. In reality, it is extremely rare for multiple layers of ghost images to be observed. The measurement method according to the embodiments corresponds to a case where a single-layer ghost image is observed. In the case where multiple layers of ghost images are observed, the user can select the ghost image whose boundary is most clearly observed among the multiple layers.

The measurement method is as follows (see Fig. 19, which illustrates the measurement conditions for evaluating ghost images):

The following specific terms and conditions apply:

a) Test pattern: A test image consisting of ‘nine circles of white board lines on a uniform black background’, as in Fig. 12a;

For example, a test image consisting of nine circles is displayed on a virtual image plane with no parallax, so there is no need to consider ghosting due to crosstalk.

b) Ambient surround condition: dark surround condition;

c) Acquire test pattern images: Using three imaging LMDs positioned in the eye box, three sets of 2D images are captured, i.e., the left, center and right images in Fig. 12b.

The measurement procedure is as follows:

a) Apply a test signal;

b) Three test pattern images are acquired by three imaging LMDs, LMD _L , LMD _C , and LMD _R of Fig. 12b.

If necessary, repeat the measurement with a different spacing between LMD _L and LMD _R.

c) For all positions (x _ij , y _ij ) i, j = 1,2,3 for P ₁₁ ~P ₃₃ of the original test pattern and for the corresponding positions (x _Gij , y _Gij ) i, j = 1,2,3 for P _G11 ~P _G33 , the ghost pattern is determined according to the following first equation;

Meal 1:

M, N are the horizontal and vertical number of pixels in the captured image.

i, j are 1, 2, 3 for P ₁₁ to P ₃₃ of the original test pattern or P _G11 to P _G33 of the ghost pattern;

is the pixel index of the image captured by LMDC.

is the horizontal field of view of the imaging LMD.

is the vertical field of view of the imaging LMD.

D _VI is the virtual image distance in millimeters (mm) between the (0, 0, 0) point of the eye box and the center of the virtual image plane. The measurement method of D _VI is described in IEC 62629-62-11.

D _VI is determined as the manufacturer's design value or as measured value according to IEC 62629-62-11.

d) The level of the ghost image is calculated as the average of the distance or angle between the original and the corresponding position for P ₁₁ to P ₃₃ according to the following second equation.

Second meal:

L _d is the ghost level expressed in distance or angle (millimeters or degrees).

(x _ij , y _ij ) i, j = 1, 2, 3 are positions P ₁₁ to P ₃₃ of the original pattern.

(x _Gij , y _Gij ) i, j = 1, 2, 3 are the positions of P _G11 to P _G33 in the ghost pattern of Fig. 19.

To harmonize the measurement locations specified in SAE J 1757-2, the measurement method described in this section is applicable only to the three center points of P ₁₂ , P ₂₂ , and P ₃₂ of Fig. 19.

Fig. 24 shows an optical measuring device according to embodiments.

The method according to the embodiments may be performed by the device of Fig. 24. Each component of Fig. 24 may correspond to hardware, software, a processor, and/or a combination thereof. Fig. 24 may correspond to the device of Fig. 8.

The light measuring unit may be a camera. The processor may be a processor that performs operations according to the embodiments. The memory may store data and information related to the operation of the processor. The memory may provide necessary data to the processor. The memory may be connected to the light measuring unit, the processor, etc.

With respect to ghost level measurement, referring to the drawings described above, the method according to the embodiments further includes the steps of generating images including points in each pattern for a virtual image plane using one or more optical measurement devices, each image being captured based on the one or more optical measurement devices, and each image corresponding to at least one of a left image, a center image, and a right image; and the steps of generating positions of points based on the one or more optical measurement devices and each pattern; and the steps of generating a level for a ghost image for the virtual image plane based on the generated positions, wherein the positions can be obtained based on a field of view of the one or more optical measurement devices, a gap between the left optical measurement device and the right optical measurement device.

Additionally, the step of generating a level for the ghost image may include the step of generating a test signal, the step of generating positions of a test pattern of a virtual image for the test signal and positions of the ghost pattern for the positions, and the step of generating a ghost level based on the positions of the test pattern and the positions of the ghost pattern.

Additionally, the ghost level can be obtained based on an average of distances between the positions of the test pattern and the positions of the ghost pattern and an average of angles between the positions of the test pattern and the positions of the ghost pattern.

The method according to the embodiments can be performed by the device. The device according to the embodiments includes a photographing unit which generates images including points in each pattern for a virtual image plane using one or more optical measurement devices, each image being captured based on the one or more optical measurement devices, and each image corresponding to at least one of a left image, a center image, and a right image; and a calculating unit which generates positions of points based on the one or more optical measurement devices and each pattern; wherein the calculating unit generates a level for a ghost image for the virtual image plane based on the generated positions, and the position can be obtained based on a field of view of the one or more optical measurement devices, a gap between the left optical measurement device and the right optical measurement device. Each component of the device can be composed of an interface for transmitting and receiving signals, a memory for storing operation-related information, and processors for controlling operations. The operations of the photographing unit and the calculating unit can be performed by the processor.

Additionally, the processor can perform the steps of generating a test signal, generating positions of a test pattern of a virtual image for the test signal and positions of a ghost pattern for the positions, and generating a ghost level based on the positions of the test pattern and the positions of the ghost pattern.

Additionally, the ghost level can be obtained based on an average of the distances between the positions of the test pattern and the positions of the ghost pattern and an average of the angles between the positions of the test pattern and the positions of the ghost pattern.

The output unit can be configured to set conditions including a test image including a test pattern comprising circles consisting of white boundary lines on a single black background, an ambient surround condition, a dark surround condition, and acquisition of the test pattern image by one or more optical measurement devices, for measuring a ghost image.

The output unit is configured to: apply a test signal; acquire test pattern images by one or more optical measurement devices; and calculate positions of patterns in the test pattern image and positions of patterns in a ghost pattern with respect to the test pattern image; wherein X-coordinates of the patterns in the test pattern image are calculated by a virtual image distance between a center of the virtual image plane and an eye box, and a horizontal FOV of the one or more optical measurement devices, and Y-coordinates of the patterns in the ghost pattern can be calculated by a virtual image distance between a center of the virtual image plane and an eye box, and a vertical FOV of the one or more optical measurement devices (see Equation 1).

The output unit is further configured to generate a level for the ghost image based on the positions of the patterns in the test pattern image and the positions of the patterns in the ghost pattern (see Equation 2).

This can improve the 3D HUD vehicle service from the perspective of driver visibility and convenience. In addition, by mounting a light measuring device on the vehicle, measurement parameters for measuring optical characteristics such as points and depth for a HUD virtual image can be efficiently obtained. Based on the obtained parameter information, a calibration process that can reduce errors from the driver's perspective can be efficiently performed. 3D HUD can enable safe and accurate autonomous driving when combined with autonomous driving technology. In addition, accurate and safe vehicle services can be provided to drivers by processing ghost images.

The embodiments have been described in terms of methods and/or devices, and the descriptions of methods and devices may be applied complementarily.

For the convenience of explanation, each drawing has been described separately, but it is also possible to design a new embodiment by combining the embodiments described in each drawing. In addition, designing a computer-readable recording medium in which a program for executing the previously described embodiments is recorded according to the needs of a person skilled in the art also falls within the scope of the embodiments. The devices and methods according to the embodiments are not limited to the configurations and methods of the embodiments described above, but the embodiments may be configured by selectively combining all or part of the embodiments so that various modifications can be made. Although the preferred embodiments of the embodiments have been illustrated and described, the embodiments are not limited to the specific embodiments described above, and various modifications can be made by a person skilled in the art without departing from the gist of the embodiments claimed in the claims, and such modifications should not be individually understood from the technical idea or prospect of the embodiments.

The various components of the device of the embodiments may be performed by hardware, software, firmware, or a combination thereof. The various components of the embodiments may be implemented as one chip, for example, one hardware circuit. According to embodiments, the components according to the embodiments may be implemented as separate chips, respectively. According to embodiments, at least one of the components of the device of the embodiments may be configured with one or more processors capable of executing one or more programs, and the one or more programs may perform, or include instructions for performing, one or more of the operations/methods according to the embodiments. The executable instructions for performing the methods/operations of the device of the embodiments may be stored in non-transitory CRMs or other computer program products configured to be executed by one or more processors, or may be stored in temporary CRMs or other computer program products configured to be executed by one or more processors. In addition, the memory according to the embodiments may be used as a concept including not only volatile memory (e.g., RAM, etc.), but also non-volatile memory, flash memory, PROM, etc. Additionally, it may include implementations in the form of carrier waves, such as transmission over the Internet. Additionally, the processor-readable recording medium may be distributed over network-connected computer systems, so that the processor-readable code may be stored and executed in a distributed manner.

In this document, “/” and “,” are interpreted as “and/or”. For example, “A/B” is interpreted as “A and/or B”, and “A, B” is interpreted as “A and/or B”. Additionally, “A/B/C” means “at least one of A, B, and/or C”. Also, “A, B, C” means “at least one of A, B, and/or C”. Additionally, “or” in this document is interpreted as “and/or”. For example, “A or B” can mean 1) “A” only, 2) “B” only, or 3) “A and B”. In other words, “or” in this document can mean “additionally or alternatively”.

Terms such as first, second, etc. may be used to describe various components of the embodiments. However, the various components according to the embodiments should not be limited in their interpretation by the above terms. These terms are merely used to distinguish one component from another. For example, a first user input signal may be referred to as a second user input signal. Similarly, a second user input signal may be referred to as a first user input signal. The use of these terms should be construed as not departing from the scope of the various embodiments. Although the first user input signal and the second user input signal are both user input signals, they do not mean the same user input signals unless the context clearly indicates otherwise.

The terminology used to describe the embodiments is used for the purpose of describing particular embodiments and is not intended to be limiting of the embodiments. As used in the description of the embodiments and in the claims, the singular is intended to include the plural unless the context clearly dictates otherwise. The expressions “and” and “or” are used to mean including all possible combinations of the terms. The expression “includes” describes the presence of features, numbers, steps, elements, and/or components, and does not mean that additional features, numbers, steps, elements, and/or components are not included. Conditional expressions such as “if,” “when,” etc., used to describe the embodiments are not intended to be limited to only optional cases. When a particular condition is satisfied, a related action is performed in response to a particular condition, or a related definition is intended to be interpreted.

In addition, the operations according to the embodiments described in this document may be performed by a transceiver device including a memory and/or a processor according to the embodiments. The memory may store programs for processing/controlling the operations according to the embodiments, and the processor may control various operations described in this document. The processor may be referred to as a controller, etc. The operations according to the embodiments may be performed by firmware, software, and/or a combination thereof, and the firmware, software, and/or a combination thereof may be stored in the processor or in the memory.

Meanwhile, the operations according to the embodiments described above may be performed by a transmitting device and/or a receiving device according to the embodiments. The transmitting/receiving device may include a transmitting/receiving unit for transmitting and receiving media data, a memory for storing instructions (program codes, algorithms, flowcharts, and/or data) for a process according to the embodiments, and a processor for controlling operations of the transmitting/receiving device.

The processor may be referred to as a controller, etc., and may correspond to, for example, hardware, software, and/or a combination thereof. The operations according to the embodiments described above may be performed by the processor. In addition, the processor may be implemented as an encoder/decoder, etc. for the operations of the embodiments described above.

Claims (20)

A step of generating images including points within each pattern for a virtual image plane using one or more optical measuring devices,
Each of the above images is captured based on one or more of the above optical measuring devices,
Each image corresponds to at least one of the left image, the center image, and the right image; and
A step of generating the positions of the above points based on the one or more optical measuring devices and each pattern; and
Further comprising a step of generating a level for a ghost image for the virtual image plane based on the generated positions;
The above position is obtained based on the field of view of one or more of the optical measurement devices, the gap between the left optical measurement device and the right optical measurement device.
Method for measuring optical properties.
In the first paragraph,
The left image including points within the pattern is captured based on the left optical measurement device of the one or more optical measurement devices, the central image including points within the pattern is captured based on the central optical measurement device of the one or more optical measurement devices, and the right image including points within the pattern is captured based on the right optical measurement device of the one or more optical measurement devices.
Method for measuring optical properties.
In the first paragraph,
The coordinate values of the above position are calculated based on the horizontal pixel index of the left light measuring device, the horizontal pixel index of the right light measuring device, the horizontal pixel index of the central light measuring device, and the field of view of the left light measuring device.
Method for measuring optical properties.
In the first paragraph, the method
Further comprising a step of measuring a virtual image distance for the virtual image plane based on a position within the pattern at the center and the optical measuring device.
Method for measuring optical properties.
In the fourth paragraph, the method
Further comprising a step of measuring a look down angle and a look up angle for the virtual image plane based on the position within the pattern and the virtual image distance.
Method for measuring optical properties.
In the first paragraph, the method
A step of measuring a horizontal field of view for the virtual image plane based on a distance to a left point of the center and a distance to a right point of the center; and
further comprising: a step of measuring a vertical field of view for the virtual image plane based on a distance to the central top point and a distance to the central bottom point;
Method for measuring optical properties.
In the first paragraph, the method
A step of measuring horizontal distortion for the virtual image plane based on a line between the center and the top point of the center and the bottom point of the center; and
A step of measuring vertical distortion for the virtual image plane based on a line between the center and the left point of the center and the right point of the center; comprising;
Method for measuring optical properties.
In the first paragraph,
The steps for generating levels for the above ghost images are:
To measure the above ghost image, a test image including a test pattern including circles consisting of white boundary lines on a single black background, an ambient surround condition, a dark surround condition, and conditions including acquisition of the test pattern image by one or more of the above optical measurement devices are set.
Method for measuring optical properties.
In the first paragraph,
The steps for generating levels for the above ghost images are:
Apply a test signal;
Obtaining test pattern images by one or more of the above optical measuring devices;
Comprising: calculating positions of patterns in the test pattern image and positions of patterns in a ghost pattern for the test pattern image;
The X-coordinates of the patterns in the test pattern image are derived from the virtual image distance between the center of the virtual image plane and the eye box, and the horizontal FOV of the one or more optical measurement devices,
The Y coordinates of the patterns in the above ghost pattern are derived from the virtual image distance between the center of the virtual image plane and the eye box, and the vertical FOV of the one or more optical measurement devices.
Method for measuring optical properties.
In Article 9,
The steps for generating levels for the above ghost images are:
generating a level for the ghost image based on the positions of the patterns within the test pattern image and the positions of the patterns within the ghost pattern;
Method for measuring optical properties.
A photographing unit that generates images containing points within each pattern for a virtual image plane using one or more optical measuring devices;
Each of the above images is captured based on one or more of the above optical measuring devices,
Each image corresponds to at least one of the left image, the center image, and the right image; and
A calculation unit for generating the positions of the above points based on one or more of the optical measurement devices and each pattern;
The above-mentioned generating unit generates a level for a ghost image for the virtual image plane based on the above-mentioned generated positions,
The above position is obtained based on the field of view of one or more of the optical measurement devices, the gap between the left optical measurement device and the right optical measurement device.
Optical property measuring device.
In Article 11,
The left image including points within the pattern is captured based on the left optical measurement device of the one or more optical measurement devices, the central image including points within the pattern is captured based on the central optical measurement device of the one or more optical measurement devices, and the right image including points within the pattern is captured based on the right optical measurement device of the one or more optical measurement devices.
Optical property measuring device.
In Article 11,
The coordinate values of the above position are calculated based on the horizontal pixel index of the left light measuring device, the horizontal pixel index of the right light measuring device, the horizontal pixel index of the central light measuring device, and the field of view of the left light measuring device.
Optical property measuring device.
In Article 11,
The above output section
Measuring a virtual image distance for the virtual image plane based on the position within the pattern at the center and the optical measuring device;
Optical property measuring device.
In Article 14,
The above output section
Based on the position within the above pattern and the virtual image distance, a look down angle and a look up angle for the virtual image plane are measured.
Optical property measuring device.
In Article 11,
The above output section
Measuring a horizontal field of view for the virtual image plane based on a distance to a left point of the center and a distance to a right point of the center, and
Measuring a vertical field of view for the virtual image plane based on the distance to the top point in the center and the distance to the bottom point in the center.
Optical property measuring device.
In Article 11,
The above output section
Measuring horizontal distortion for the virtual image plane based on a line between the center and the center top point and the center bottom point; and
Measuring vertical distortion for the virtual image plane based on a line between the center and the left point of the center and the right point of the center;
Optical property measuring device.
In Article 11,
The above output section,
To measure the above ghost image, a test image including a test pattern including circles consisting of white boundary lines on a single black background, an ambient surround condition, a dark surround condition, and conditions including acquisition of a test pattern image by one or more of the above optical measurement devices are set.
Optical property measuring device.
In Article 11,
The above output section,
Apply a test signal;
Obtaining test pattern images by one or more of the above optical measuring devices;
configured to calculate the positions of patterns in the test pattern image and the positions of patterns in the ghost pattern for the test pattern image;
The X-coordinates of the patterns in the test pattern image are derived from the virtual image distance between the center of the virtual image plane and the eye box, and the horizontal FOV of the one or more optical measurement devices,
The Y coordinates of the patterns in the above ghost pattern are derived from the virtual image distance between the center of the virtual image plane and the eye box, and the vertical FOV of the one or more optical measurement devices.
Optical property measuring device.
In Article 19,
The above output section,
Further configured to generate a level for the ghost image based on the positions of the patterns in the test pattern image and the positions of the patterns in the ghost pattern;
Optical property measuring device.