KR102911790B1 - Method and apparatus of correcting crosstalk - Google Patents

Abstract

A method and device for correcting crosstalk according to one embodiment estimate an area where crosstalk will occur based on a three-dimensional positional relationship between the positions of a user's two eyes and the positions of virtual images of a virtual content object, generate a concealer image for correcting the area where crosstalk will occur based on the estimated area and the virtual content object, and correct the crosstalk by synthesizing the virtual content object and the concealer image.

Description

METHOD AND APPARATUS OF CORRECTING CROSSTALK

The following embodiments relate to a method and device for compensating crosstalk.

Stereo, which allows users to experience a three-dimensional effect by projecting separate images from each eye's perspective, has no problem if accurate graphic information is provided to both eyes. However, if accurate graphics are not projected to both eyes due to physical phenomena such as light bleed, 3D crosstalk may occur. For example, in a glasses-free 3D display, if the stereo images are incompletely separated due to optical phenomena such as light bleed, preventing accurate augmented reality (AR) graphics from being projected to both eyes, the user will not be able to perceive the three-dimensional effect. Methods to reduce this 3D crosstalk include blurring the image or lowering the brightness of the image. However, all of the above-mentioned methods have the problem of degrading the image quality.

The background technology described above is technology that the inventor possessed or acquired in the process of deriving the disclosure of the present application, and cannot necessarily be said to be publicly known technology disclosed to the general public prior to the present application.

According to one embodiment, a method for correcting crosstalk includes: estimating an area where crosstalk occurs based on a three-dimensional positional relationship between positions of a user's two eyes and positions of a virtual image of a virtual content object; generating a concealer image for correcting the area where crosstalk occurs based on the estimated area and the virtual content object; and correcting the crosstalk by synthesizing the virtual content object and the concealer image.

The step of generating the concealer image may include a step of adjusting at least one effect element among the afterimage range, contrast, and brightness of the concealer image.

The step of adjusting at least one effect element may include at least one or a combination of the steps of adjusting an afterimage range of the concealer image in comparison with the size of an image of the virtual content object based on the three-dimensional positional relationship; adjusting contrast and brightness of the concealer image based on the three-dimensional positional relationship; and adjusting brightness of the concealer image based on illumination of the surrounding environment.

The step of adjusting the afterimage range of the concealer image may include a step of gradually expanding the afterimage range of the concealer image in comparison with the size of the virtual image of the virtual content object as the distance between the positions of the user's eyes and the positions of the virtual image becomes further than a reference distance; and a step of gradually reducing the afterimage range of the concealer image in comparison with the size of the virtual image of the virtual content object as the distance between the positions of the user's eyes and the positions of the virtual image becomes closer than the reference distance.

The step of adjusting the contrast and brightness of the concealer image may include a step of gradually darkening the contrast and brightness of the concealer image as the distance between the positions of the user's eyes and the positions of the virtual image becomes further than a reference distance; and a step of gradually brightening the contrast and brightness of the concealer image as the distance between the positions of the user's eyes and the positions of the virtual image becomes closer than the reference distance.

The step of adjusting the brightness of the concealer image may include a step of gradually darkening the brightness of the concealer image as the illuminance increases above the reference illuminance; and a step of gradually brightening the brightness of the concealer image as the illuminance decreases below the reference illuminance.

The step of generating the concealer image may include a step of generating the concealer image by blurring the virtual content object so as to correspond to the shape of the virtual content object.

The step of generating the concealer image may include the step of generating a blurred image by blurring the virtual content object; and the step of adjusting at least one effect element among the residual image range, contrast, and brightness corresponding to the blurred image.

The step of generating the above-mentioned blur image may include a step of generating the blur image so as to correspond to an expanded shape of the virtual content object based on the three-dimensional positional relationship.

The step of generating the above-described blur image may include a step of generating a first blur image corresponding to the left eye and a second blur image corresponding to the right eye among the user's two eyes based on the shape of the virtual content object; and a step of generating the blur image by combining the first blur image and the second blur image.

The step of generating the blurred image by combining the first blurred image and the second blurred image may include the step of separating the first blurred image and the second blurred image by a distance based on the disparity of the user's two eyes; and the step of combining the first blurred image and the second blurred image so that the distance is maintained.

The step of combining the first blur image and the second blur image so that the spaced interval is maintained may include a step of generating the blur image by interpolating the spaced first blur image and the second blur image.

The method may include: arranging the concealer image on a three-dimensional space including the virtual content object; generating a left-eye image and a right-eye image by rendering the virtual content object and the concealer image together; and projecting the left-eye image and the right-eye image.

The step of correcting the crosstalk may include a step of generating a left-eye image and a right-eye image by rendering the virtual content object; and a step of projecting the left-eye image, the right-eye image, and the concealer image.

The step of estimating the area where the crosstalk will occur may include a step of estimating the area where the crosstalk will occur based on the parallax of the user's two eyes and the three-dimensional positional relationship.

The step of estimating the area where the crosstalk will occur may include a step of estimating the area where the crosstalk will occur by adjusting the range of the residual image of the area where the crosstalk will occur based on the parallax of the user's two eyes and the three-dimensional positional relationship.

The method for correcting the crosstalk may further include a step of detecting a positional shift of the virtual content object, and the step of estimating an area where the crosstalk will occur may include a step of estimating an area where the crosstalk will occur based on at least one of the three-dimensional positional relationship changed according to the positional shift and the parallax between the user's two eyes.

The area where the above crosstalk occurs may correspond to the location and range where the above concealer image is placed.

According to one embodiment, a device for compensating crosstalk includes a sensor for detecting positions of both eyes of a user; and a processor for estimating an area where crosstalk will occur based on a three-dimensional positional relationship between the positions of both eyes of the user and positions of virtual images of a virtual content object, generating a concealer image for compensating for the area where crosstalk will occur based on the estimated area and the virtual content object, and synthesizing the virtual content object and the concealer image to compensate for the crosstalk.

The processor can adjust at least one effect element among the afterimage range, contrast, and brightness of the concealer image.

The processor may perform at least one or a combination of the following operations: adjusting an afterimage range of the concealer image in comparison with the size of an image of the virtual content object based on the three-dimensional positional relationship; adjusting a contrast of the concealer image based on the three-dimensional positional relationship; and adjusting the brightness of the concealer image based on the illuminance of the surrounding environment.

The processor may gradually expand the afterimage range of the concealer image in comparison with the size of the virtual image of the virtual content object as the distance between the positions of the user's eyes and the positions of the virtual image becomes further than the reference distance, and may gradually reduce the afterimage range of the concealer image in comparison with the size of the virtual image of the virtual content object as the distance between the positions of the user's eyes and the positions of the virtual image becomes closer than the reference distance.

The processor may gradually adjust the contrast and brightness of the concealer image to be darker as the distance between the positions of the user's eyes and the positions of the virtual image becomes farther than the reference distance, and may gradually adjust the contrast and brightness of the concealer image to be brighter as the distance between the positions of the user's eyes and the positions of the virtual image becomes closer than the reference distance.

The above processor can gradually darken the brightness of the concealer image as the illuminance increases above the reference illuminance, and can gradually brighten the brightness of the concealer image as the illuminance decreases below the reference illuminance.

The processor can generate the concealer image by blurring the virtual content object to correspond to the shape of the virtual content object.

The processor can generate a blurred image by blurring the virtual content object, and adjust at least one effect element among an afterimage range, contrast, and brightness corresponding to the blurred image.

The above processor can generate the blurred image to correspond to an expanded form of the virtual content object based on the three-dimensional positional relationship.

The processor can generate a first blur image corresponding to the left eye and a second blur image corresponding to the right eye among the user's two eyes based on the shape of the virtual content object, and can generate the blur image by combining the first blur image and the second blur image.

The processor may separate the first blur image and the second blur image by an interval based on the parallax between the user's two eyes, and combine the first blur image and the second blur image so that the separated interval is maintained.

The processor can generate the blurred image by interpolating the spaced first blurred image and the second blurred image.

The processor can place the concealer image on a three-dimensional space including the virtual content object, render the virtual content object and the concealer image together, thereby generating a left-eye image and a right-eye image, and project the left-eye image and the right-eye image.

The processor can generate a left-eye image and a right-eye image by rendering the virtual content object, and project the left-eye image, the right-eye image, and the concealer image.

The processor can estimate an area where crosstalk will occur based on the parallax between the user's two eyes and the three-dimensional positional relationship.

The processor can estimate the area where the crosstalk will occur by adjusting the range of the residual image of the area where the crosstalk will occur based on the parallax of the user's two eyes and the three-dimensional positional relationship.

The processor can detect a movement in the position of the virtual content object, and estimate an area where the crosstalk will occur based on at least one of the three-dimensional positional relationship changed according to the movement in the position and the parallax between the user's eyes.

The area where the above crosstalk occurs may correspond to the location and range where the above concealer image is placed.

According to one embodiment, a content visualization device includes a display that projects a virtual content object onto a projection plane to visualize it; and a processor that estimates an area where crosstalk will occur based on a three-dimensional positional relationship between positions of a user's two eyes and positions of virtual images of the virtual content object, generates a concealer image for correcting the area where crosstalk will occur based on the estimated area and the virtual content object, and synthesizes the virtual content object and the concealer image to correct the crosstalk.

According to one embodiment, an augmented reality glass (AR glass) device includes a transparent display for visualizing a virtual content object; and a processor for estimating an area where crosstalk will occur based on a three-dimensional positional relationship between positions of both eyes of a user and positions of virtual images of the virtual content object, generating a concealer image for correcting the area where the crosstalk will occur based on the estimated area and the virtual content object, and synthesizing the virtual content object and the concealer image to correct the crosstalk.

According to one embodiment, a vehicle includes a sensor for sensing a position of the vehicle; a head-up display (HUD) for visualizing a virtual content object to be provided to a user on a projection plane; and a processor for determining the virtual content object based on the position of the vehicle, estimating an area where crosstalk will occur based on a three-dimensional positional relationship between positions of the user's two eyes and positions of virtual images of the virtual content object, generating a concealer image for correcting the area where the crosstalk will occur based on the estimated area and the virtual content object, and synthesizing the virtual content object and the concealer image to correct the crosstalk.

FIG. 1 is a diagram conceptually illustrating a method for compensating crosstalk according to one embodiment.
FIG. 2 is a drawing illustrating an example of a virtual content object according to one embodiment.
FIG. 3 is a block diagram illustrating a configuration of a device for compensating crosstalk according to one embodiment.
FIG. 4 is a block diagram illustrating a configuration of a head-up display (HUD) that displays a crosstalk-compensated virtual content object according to one embodiment.
Figure 5 is a flowchart illustrating a method for compensating crosstalk according to one embodiment.
FIG. 6 is a diagram illustrating a method for compensating crosstalk according to one embodiment.
FIG. 7 is a flowchart illustrating a method for generating a concealer image according to one embodiment.
FIG. 8 is a diagram illustrating a method for generating a blurred image according to one embodiment.
FIG. 9 is a diagram illustrating a principle in which parallax increases as the distance between a user's eyes and a virtual content object increases according to one embodiment.
FIGS. 10 to 12 are drawings for explaining a method of controlling at least one effect element according to embodiments.
Fig. 13 is a flowchart illustrating a method for compensating crosstalk according to another embodiment.

Specific structural or functional descriptions of the embodiments are disclosed for illustrative purposes only and may be modified and implemented in various forms. Therefore, the actual implementation is not limited to the specific embodiments disclosed, and the scope of this specification includes modifications, equivalents, or alternatives within the technical concepts described in the embodiments.

Although terms such as "first" or "second" may be used to describe various components, these terms should be interpreted solely to distinguish one component from another. For example, a first component may be referred to as a second component, and similarly, a second component may also be referred to as a first component.

When it is said that a component is "connected" to another component, it should be understood that it may be directly connected or connected to that other component, but there may also be other components in between.

Singular expressions include plural expressions unless the context clearly dictates otherwise. In this specification, the terms "comprises" or "has" should be understood to indicate the presence of a described feature, number, step, operation, component, part, or combination thereof, but not to exclude the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by a person of ordinary skill in the art. Terms defined in commonly used dictionaries should be interpreted to have a meaning consistent with their meaning in the context of the relevant technology, and will not be interpreted in an idealized or overly formal sense unless explicitly defined herein.

Hereinafter, embodiments will be described in detail with reference to the attached drawings. In the description with reference to the attached drawings, identical components are assigned the same reference numerals regardless of the drawing numbers, and redundant descriptions thereof will be omitted.

FIG. 1 is a diagram conceptually illustrating a method for compensating for crosstalk according to one embodiment. Referring to FIG. 1, crosstalk (140) occurring around a virtual content object (130) visualized on a virtual screen (110) is illustrated due to the user's binoculars not receiving accurate graphic information regarding an object (101) to be visualized on a glasses-free 3D display.

For example, a virtual image (120) of an object (101) to be visualized can be visualized through a left-eye image (112) provided to the user's left eye and a right-eye image (114) provided to the user's right eye on a virtual screen (110). At this time, if the stereo images are incompletely separated due to an optical phenomenon such as light scattering, such that the left-eye image (112) is provided to the user's right eye and the right-eye image (114) is provided to the user's left eye, crosstalk (140) may occur due to the overlapping of the left-eye image (112) and the right-eye image (114) around the virtual content object (130) displayed on the virtual screen (110). At this time, the virtual screen (110) may be, for example, a virtual screen of a glasses-free 3D display or a head-up display (HUD), or a virtual screen of a transparent display of an augmented reality device such as an augmented reality glasses device, but is not necessarily limited thereto. The virtual content object (130) and the space in which the virtual content object (130) is visualized are described in more detail with reference to FIG. 2 below.

In one embodiment, an image (150) rendered to the extent that the crosstalk (140) occurs may be attached to the periphery of a virtual content object (130) to cover the area where the crosstalk (140) occurs. The image (150) rendered to the extent that the crosstalk occurs may be called a "concealer image" in the sense that it conceals or hides the area where the crosstalk occurs. Hereinafter, the image (150) rendered to the extent that the crosstalk occurs and the concealer image may be understood to have the same meaning.

In one embodiment, for example, a concealer image (150) may be generated by obtaining a hint from the color of a virtual content object (140), thereby naturally covering an area where crosstalk occurs by the concealer image (150) without causing a sense of incongruity with the surroundings. A method for generating a concealer image will be described in more detail with reference to FIGS. 7 and 8 below.

In addition, in one embodiment, by adjusting various effect elements such as brightness by distance, width, outdoor brightness, HUD brightness, color, etc. for the concealer image (150), the afterimage of the crosstalk (140) can be offset by covering the part where the crosstalk (140) occurs. At this time, the effect element may be for covering the part where the crosstalk (140) occurs and providing a glow effect to the concealer image (150) to reduce the sense of discomfort with the surroundings.

The range of the concealer image (150) can be determined, for example, based on the amount of change in disparity according to the distance between the position of the virtual image (120) of the virtual content object visualized on the virtual screen (110) and the position of the user's eyes. The relationship between the distance and disparity between the position of the virtual image (120) of the virtual content object and the position of the user's eyes is described in more detail with reference to FIG. 9 below.

Additionally, the brightness level of the concealer image (150) can be adjusted, for example, depending on the illuminance of the external environment and the brightness of the virtual screen (110). In one embodiment, by variously adjusting the effect elements of the concealer image (150), crosstalk (140) occurring in the virtual screen (110) can be eliminated, while the brightness and clarity of the virtual screen (110) can be optimized. A method of adjusting the effect elements according to one embodiment will be described in more detail with reference to FIGS. 10 to 12 below.

FIG. 2 is a drawing illustrating an example of a virtual content object according to one embodiment. Referring to FIG. 2, a result of visualizing a virtual content object (221) at an arbitrary location within an object placement space (210) is illustrated. Hereinafter, for convenience of explanation, a case in which a projection panel on which a virtual image of a virtual content object (221) is displayed is described as an example of a head-up display, but is not necessarily limited thereto, and the virtual content object (221) may be visualized on various projection panels, such as a transparent display of an augmented reality glasses device.

A device for compensating crosstalk according to one embodiment (hereinafter, "compensation device") can compensate for crosstalk by synthesizing a virtual content object and a concealer image, and visualize the crosstalk-compensated virtual content object. The virtual content object can represent a graphic object corresponding to the content. For example, the virtual content object can include a graphic object corresponding to path guidance content (221) in the form of an arrow.

For example, the correction device can place and visualize a graphic object at any location within the object placement space (210). The correction device can provide a left-eye image to the user's left eye and a right-eye image to the user's right eye, thereby providing a three-dimensional graphic object. The left-eye image and the right-eye image can include graphic objects spaced apart from each other along a horizontal axis by the parallax between the user's two eyes according to depth. Therefore, the user can perceive the depth of the three-dimensionally rendered graphic object.

However, as described above, if the left-eye image is provided to the user's right eye and the right-eye image is incorrectly provided to the user's left eye, crosstalk may occur as the left-eye image and the right-eye image overlap.

In this case, the compensation device can compensate for crosstalk by synthesizing a concealer image and a virtual content object to compensate for the area where crosstalk occurs, as described above with reference to FIG. 1. An example of the configuration of the compensation device for this purpose is described in detail with reference to FIGS. 3 and 4 below. In addition, the method by which the compensation device compensates for crosstalk is described in detail with reference to FIG. 5 below.

Below, before looking at how to compensate for crosstalk, we will first look at virtual content objects and the space in which they are visualized.

The object placement space (210) may correspond to a three-dimensional space in which graphic objects can be placed. The object placement space (210) may represent a space in which graphic objects having depth can be visualized in three dimensions. The coordinates of each point belonging to the object placement space (210) may be mapped to a scale that is the same as or similar to the position coordinates of the physical world. The boundary of the object placement space (210) may be determined according to the structure of the head-up display. For example, the correction device may visualize a graphic object within a space from a minimum depth (e.g., a minimum distance) to a maximum depth (e.g., a maximum distance) determined according to the structure of the head-up display.

The object placement space (210) can be determined based on a space corresponding to the user's field of view. The shape and size of the object placement space (210) can be determined based on the size of the eye box (290) and the field of view (FOV) that can be provided by the head-up display. The object placement space (210) can be, for example, a rectangular cone-shaped space that extends from the eye box (290) in correspondence to the field of view. The eye box (290) can represent an area set to be where both eyes of the user are located. The position of the eye box (290) can be fixed, but is not necessarily limited thereto, and can also change depending on the position of the eyes detected for the user. The shape of the object placement space (210) is also not limited to a rectangular cone-shaped space, and can change depending on the design.

A correction device according to one embodiment can be implemented to visualize various types of graphic representations at display locations within an object placement space (210) beyond a wind shield glass (280), and different types of graphic representations can be visualized in different object placement spaces (210).

The types of graphic representations that can be displayed within the object placement space (210) may include, for example, path guidance content (221), path indication lines (222), and warning content (223). In addition, the types of graphic representations that can be displayed within the object placement space (210) may be changed and implemented in various forms.

A user may perceive that the corresponding graphic representations exist at physical locations corresponding to the display positions within the object placement space (210). As described above, each coordinate within the object placement space (210) may be mapped one-to-one with a physical coordinate. Furthermore, a virtual content object may occupy a space corresponding to its shape. For example, a virtual content object may occupy a portion of the space within the object placement space (210).

Route guidance content (221) is content in which route guidance information that must be provided to proceed along a route to a destination is visualized, and may include, for example, numbers and letters indicating the distance the user must proceed straight, arrows indicating a change of direction (e.g., a left turn or a right turn) at a fork in the road, a speed limit on the road (270) currently being driven, an area name of the current driving location, a road name, etc. Route guidance information may indicate information guiding the user's movement along a route set by the user and information associated with the route. For example, route guidance information may include the distance the user must proceed straight, a change of direction (a turn) at a fork in the road, etc. A route may indicate a path that the user must pass through from a starting point to a destination. In addition, route guidance information may include, as information associated with a route to a destination, the location, area, name, attribute, and safety information (e.g., a speed limit, construction information, accident information, etc.) of a road (270) included in the route. The route indication line (222) is a line indicating the route to the destination, and can be visualized as route guidance information in a form distinct from the route guidance content (221). The warning content (223) can include a warning message that should be given to the user when necessary in the current driving environment.

In addition, a correction device according to one embodiment can estimate the position of a vehicle using sensors (e.g., a camera sensor, a GNSS module, a radar sensor, a LIDAR sensor, etc.). The correction device can visualize a graphic object corresponding to the route guidance content (221) by aligning it with an actual road (270) by considering an error interval equivalent to the driver's eye position from the vehicle's position. When HD (High Definition) map data is used, the correction device can align the route guidance content (221) with a more accurate position. Through such alignment, the psychological stability of the driver can be improved.

Hereinafter, for convenience of explanation, the present specification describes the correction of crosstalk that occurs for route guidance content (221), which is an example of a virtual content object, but is not necessarily limited thereto. Crosstalk correction and content visualization according to one embodiment can be applied to all graphic objects visualized within an object placement space.

FIG. 3 is a block diagram illustrating a configuration of a crosstalk compensation device according to one embodiment. Referring to FIG. 3, a crosstalk compensation device (hereinafter, “compensation device”) (300) according to one embodiment includes a sensor (310) and a processor (330). The compensation device (300) includes a memory (350), a display (370), and a communication interface (290).

The sensor (310) detects the position of the user's eyes. The sensor (310) may be, for example, an eye tracker or an iris recognition sensor. In addition, the sensor (310) may detect information necessary for visualizing content. According to one embodiment, the sensor (310) may measure the distance to objects located around the user, and may include, for example, LIDAR (Light Detection and Ranging) and RADAR (Radio Detection and Ranging). In addition, the sensor (310) may sense information related to the status of a device on which the correction device (300) is mounted. For example, when mounted on a vehicle, the sensor (310) may sense information related to the vehicle, and the information related to the vehicle may include vehicle location information, road information corresponding to the vehicle location, and driving-related information of the vehicle. Here, the driving-related information may be information related to driving of the vehicle, and may include, for example, information related to the speed, acceleration, location, fuel, and maintenance of the vehicle. Additionally, the sensor (310) may include an internal sensor that photographs the interior of the vehicle. The internal sensor may be, but is not limited to, a camera sensor (e.g., a color camera), an infrared sensor, a depth sensor, a thermal imaging sensor, etc. The internal sensor may obtain information related to the eyes of a user within the vehicle, and the processor (330) may determine the position of the user's eyes to use for setting an object placement space and visualizing a stereoscopic image (e.g., a pair of left and right images).

The vehicle's location information may be the current coordinates of the vehicle's location, and may also indicate the lane information on which the vehicle is currently driving. For example, the sensor (310) may obtain the two-dimensional coordinates of the vehicle through a Global Navigation Satellite System (GNSS). In addition, the sensor (310) may obtain a forward image of the front of the vehicle, and the processor (330) may determine the lane (ego lane) on which the vehicle is currently driving among a plurality of lanes constituting the road from the forward image. However, the present invention is not limited thereto, and the processor (330) may also estimate the current location of the vehicle based on the image collected from the sensor (310).

Road information may include one or more of the following: road width, number of lanes constituting the road, lane width, center line, turn point, traffic signal, and other traffic-related information.

The sensor (310) may be, for example, a sensor module including various types of sensors.

The processor (330) estimates an area where crosstalk will occur based on the three-dimensional positional relationship between the positions of the user's eyes detected by the sensor (310) and the positions of the virtual images of the virtual content objects. The processor (330) generates a concealer image for correcting the area where crosstalk will occur based on the estimated area and the virtual content object. The processor (330) corrects the crosstalk by synthesizing the virtual content object and the concealer image. The processor (330) may directly calculate the three-dimensional positional relationship between the positions of the user's eyes and the positions of the virtual images of the virtual content objects, or may receive it from the outside through the communication interface (390). In this case, the three-dimensional positional relationship may be understood to include the distance between the positions of the user's eyes and the positions of the virtual images of the virtual content objects.

In addition, the processor (330) can generate content visualized through the display (370). According to one embodiment, the processor (330) can generate an object placement space based on the estimated front road area. The processor (330) can render virtual content objects placed in the object placement space in a form projected onto a projection plane and provide the virtual content objects to the user through the display (370). The processor (330) can also adjust the virtual content objects based on the form in which the virtual content objects are projected onto the projection plane. For example, the processor (330) can deform or rotate the virtual content objects based on an arbitrary reference point and reference axis. However, the operation of the processor (330) is not limited to the operation described above, and the processor (330) can perform the above-described operation together with at least one of the operations described in FIGS. 4 to 13 below. More specific operations of the processor (330) are described with reference to FIGS. 4 to 13 below.

The processor (330) may be an image processing device implemented as hardware having a circuit having a physical structure for executing desired operations. For example, the desired operations may include code or instructions included in a program. For example, the correction device implemented as hardware may include a microprocessor, a central processing unit (CPU), a graphic processing unit (GPU), a processor core, a multi-core processor, a multiprocessor, an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), a neural processing unit (NPU), etc.

The processor (330) can execute a program and control the correction device (300). The program code executed by the processor (330) can be stored in the memory (350).

The memory (350) may store information on a three-dimensional positional relationship including the distance between the positions of the user's two eyes and the positions of virtual images of virtual content objects and/or information on an area where crosstalk will occur as estimated by the processor (330). In addition, the memory (350) may store a concealer image and/or an image resulting from synthesizing the concealer image with a virtual content object.

In addition, the memory (350) can temporarily or permanently store information required for content visualization generated by the processor (330). For example, the memory (350) can store commands executed by the processor (330) to perform operations according to FIGS. 4 to 13 below. In addition, the memory (350) can also store virtual content objects, route guidance information, and a map database. The map database can refer to a database that stores map data. The map database can store, for example, HD (high definition) map data. The HD map data can include information regarding fine data such as the number of lanes, the width of the lanes, and the location of the center line.

The display (370) can visualize and provide a crosstalk-compensated virtual content object to a user. The display (370) can visualize the virtual content object by projecting it onto a projection plane. According to one embodiment, the display (370) can be a head-up display, and can form a projection plane in front of the user and provide content to the user through the projection plane. According to an embodiment, the display (370) can be a transparent display for visualizing a virtual content object in an augmented reality (AR) glass device. According to an embodiment, the transparent display can provide a crosstalk-compensated virtual content object in a real-world environment.

According to one embodiment, the display (370) may provide a left image to the user's left eye and a right image to the user's right eye. For example, if the processor (210) generates a left image including a first graphical representation corresponding to the driver's left eye and a right image including a second graphical representation corresponding to the driver's right eye, the display (370) may provide the left and right images with a parallax relative to each other. The display (370) may provide the user with content having depth by visualizing a three-dimensional graphical object by separating a graphical object visualizing content in the left image and a graphical object visualizing content in the right image based on binocular disparity. A parallax may be determined for each pixel of the graphical object, and a sense of depth may be expressed for each pixel. For example, pixels corresponding to a proximal part of a graphical object closer to the user may be separated by a large parallax in the left-eye image and the right-eye image. Conversely, pixels corresponding to distal parts of a graphical object that are farther from the user may be spaced apart by a small amount of parallax in the left and right eye images.

For example, the display (370) can visualize a graphic object corresponding to route guidance content in a display area determined by the processor (330). The positions of the user's eyes can be measured by the aforementioned sensor (310) (e.g., an internal sensor) and provided to the processor (330). The positions of the user's eyes can be constantly tracked while the vehicle is moving, so that content can be delivered in three dimensions even when the driver moves his or her head up, down, left, and right or adjusts the height of the seat.

The communication interface (390) may receive a three-dimensional positional relationship including the distance between the positions of the user's two eyes and the positions of virtual images of the virtual content object from the outside of the correction device (300). Alternatively, the communication interface (390) may output an image resulting from the processor (330) synthesizing the virtual content object and the concealer image and/or an image resulting from visualizing the crosstalk-corrected virtual content object on a projection plane to the outside of the correction device (300).

The correction device (300) according to one embodiment may correspond to devices in various fields, such as, for example, an Advanced Drivers Assistance System (ADAS), a Head Up Display (HUD) device, a 3D Digital Information Display (DID), a navigation device, a neuromorphic device, a 3D mobile device, a smart phone, a smart TV, a smart vehicle, an Internet of Things (IoT) device, a medical device, and a measuring device. Here, the 3D mobile device may be understood to include, for example, a display device for displaying Augmented Reality (AR), Virtual Reality (VR), and/or Mixed Reality (MR), a Head Mounted Display (HMD), and a Face Mounted Display (FMD).

Alternatively, the correction device (300) according to one embodiment may be implemented as a 3D head-up display device for a vehicle and a navigation device that indicates the driving path of the vehicle. Furthermore, the correction device (300) may be implemented to provide augmented reality to a user. For example, the correction device (300) may display content at a depth of a certain range beyond the vehicle's hood (e.g., 3.7 m to 70 m from the vehicle). However, the applications of the correction device (300) are not necessarily limited thereto.

FIG. 4 is a block diagram illustrating a configuration of a head-up display (HUD) that displays a crosstalk-compensated virtual content object according to one embodiment. Referring to FIG. 4, a block diagram illustrating a configuration of a head-up display (HUD) of a compensation device according to one embodiment is provided.

The content visualization system (400) is a system that provides a virtual content object (461) to a user (490), and may be, for example, a device equipped with a correction device (410).

The compensation device (410) may include a sensor (411), a processor (412), and a head-up display (413).

The sensor (411) can detect an object present in front. For example, the sensor (411) can measure the distance to an object present in front. However, the present invention is not necessarily limited thereto, and the sensor (411) can measure the distance to an object present around the vehicle and generate a surrounding distance map indicating the distance to the object present in the surrounding. In addition, the sensor (411) can capture the environment in front, behind, left, and right of the vehicle to generate an image. The sensor (411) may also include a module that measures and estimates the position of the correction device (410), such as a GNSS.

The processor (412) can obtain a virtual content object (461) to be provided to the user (490). The virtual content object (461) can represent an object for providing information to the user. In addition, the processor (412) can analyze surrounding information sensed by the sensor (411) (e.g., distance to surrounding objects and images including objects, etc.) to model objects, detect the location of objects, recognize objects, etc. As another example, the processor (412) can determine the current location of the correction device (410). The processor (412) can also select and load a virtual content object (461) to be provided to the user based on the object placement space according to the current location and the viewing angle of the head-up display (413).

The head-up display (413) can visualize a virtual content object (461) in a visible region of the user (490) positioned in front of the user (490). For example, the head-up display (413) can visualize the virtual content object (461) on a window (e.g., a windshield glass of a vehicle) positioned in front of the user (490). The head-up display (413) can form a virtual projection plane (450). The projection plane (450) can represent a plane on which a virtual image including the virtual content object (461) generated by the head-up display (413) is displayed. The user (490) can recognize that the virtual image is positioned on the projection plane (450). The projection plane (450) can be formed in an area observable by the eyes of the user (490).

In addition, the head-up display (413) can visualize a virtual content object (461) to be provided to a user having depth on the projection plane (450). For example, the processor (412) can determine the depth at which the virtual content object (461) can be visualized based on the projection plane (450) for each pixel constituting the object, and the head-up display (413) can visualize the virtual content object (461) to have a depth farther or closer from the projection plane (450) based on the determined depth for each pixel constituting the virtual content object (461). In other words, binocular parallax can be determined for each pixel constituting the virtual content object (461). The head-up display (413) can visualize a virtual content object (461) having a depth corresponding to a virtual area (360) on the projection plane (450). Here, the processor (412) can render the virtual content object (461) into a three-dimensional graphic representation based on the optical system of the head-up display (413). The three-dimensional graphic representation can represent a three-dimensional graphic representation having depth, and can be referred to as a graphic object below. The head-up display (413) can form a projection plane (450) on which a left image and a right image are output based on the depth of the virtual content object (461), and can provide the left image to the left eye of the user (490) and the right image to the right eye of the user (490) through the projection plane (450). Therefore, the user (490) can recognize the depth of the three-dimensionally rendered virtual content object (461).

A head-up display (413) according to one embodiment may include, for example, a picture generation unit (414), a fold mirror (415), and a concave mirror (416). However, the configuration of the head-up display (413) is not necessarily limited thereto, and may include various components that form a projection plane (450) on which a virtual image is formed through projection onto a window positioned in front of a user (490) depending on the design.

In one embodiment, the correction device (410) is described with an example of being mounted on a vehicle, but is not necessarily limited thereto. For example, the correction device (410) may be applied to technologies that combine real and virtual information, such as augmented reality glasses (AR glasses) and mixed reality (MR) devices.

According to one embodiment, the correction device (410) can express continuous depth without changing the position of the projection plane (450) formed by the head-up display (413) by adjusting the depth of the virtual content object (461). In addition, since the correction device (410) does not require changing the position of the projection plane (450), it does not require physical manipulation of the components included in the head-up display (413). When the correction device (410) is mounted on a vehicle, the correction device (410) can dynamically visualize a three-dimensional virtual content object (461) in front of the driver. As illustrated in FIG. 4, a scene (491) observed by a user may include a virtual content object (461) visualized by being superimposed on an actual physical object and background.

A correction device (410) according to one embodiment can adjust at least one effect element of a concealer image for a virtual content object (461) to enable more natural visualization. The specific operation of the correction device (410) is described in more detail with reference to the drawings below.

FIG. 5 is a flowchart illustrating a method for compensating crosstalk according to one embodiment, and FIG. 6 is a diagram for explaining a method for compensating crosstalk according to one embodiment. Referring to FIGS. 5 and 6, a process in which a compensation device according to one embodiment compensates for crosstalk of a virtual content object through steps (510) to (530) is illustrated.

In step (510), the correction device estimates an area where crosstalk will occur based on the three-dimensional positional relationship between the positions of the user's two eyes and the positions of the virtual images of the virtual content object. Here, the "three-dimensional positional relationship" can be understood to mean the distance between the positions of the user's two eyes and the positions of the virtual images of the virtual content object.

In step (510), the correction device can estimate an area (625) where crosstalk will occur, as shown in the drawing (620), on the virtual screen (610), based on the parallax and three-dimensional positional relationship of the user's eyes. At this time, the area (625) where crosstalk will occur may correspond to a location where a concealer image will be rendered for future crosstalk correction, that is, a location and range where the concealer image will be placed. The correction device can estimate an area where crosstalk will occur by adjusting the afterimage range of the area (625) where crosstalk will occur, based on the parallax and three-dimensional positional relationship of the user's eyes. For example, the correction device can estimate an area (625) where crosstalk will occur by gradually expanding the afterimage range as the distance between the positions of the user's eyes and the positions of the virtual images becomes further than a preset reference distance. Alternatively, the correction device can estimate an area (625) where crosstalk will occur by gradually reducing the afterimage range as the distance between the positions of the user's eyes and the positions of the virtual images becomes further than a reference distance.

Additionally, the correction device can detect the displacement of a virtual content object. In this case, the correction device can estimate the area (625) where crosstalk will occur based on at least one of the three-dimensional positional relationship changed by the displacement and the parallax between the user's eyes.

In step (520), the correction device generates a concealer image (630) for correcting the area (625) where crosstalk will occur, based on the area (625) and the virtual content object (640) estimated in step (510). The correction device may generate the concealer image (630), for example, based on the color of the virtual content object (640). The correction device may generate the concealer image (630) by blurring the virtual content object so as to correspond to the shape of the virtual content object (640). At this time, the degree to which the virtual content object is blurred may be variously adjusted by, for example, the illuminance of the surrounding environment, the brightness of the three-dimensional positional relationship including the distance between the positions of the user's two eyes and the positions of the virtual images of the virtual content object, the brightness of the virtual content object (640), etc.

Additionally, the correction device can adjust at least one effect element of, for example, the afterimage range, contrast, and brightness of the concealer image (630).

In step (520), the correction device may adjust the afterimage range of the concealer image (630) in comparison with the size of the virtual content object, or adjust the contrast of the concealer image (630), for example, based on a three-dimensional positional relationship. The correction device may adjust the brightness of the concealer image (630) based on the illuminance of the surrounding environment, or adjust a combination of the afterimage range, contrast, and brightness of the concealer image (630) based on the three-dimensional positional relationship and the illuminance of the surrounding environment. The method by which the correction device generates the concealer image (630) will be described in more detail with reference to FIGS. 7 to 12 below.

In step (530), the correction device compensates for crosstalk by synthesizing the virtual content object (640) and the concealer image (630). The correction device can compensate for crosstalk by synthesizing the virtual content object (640) and the concealer image (630) to cover the area (625) where crosstalk occurs, as shown in the drawing (650). Here, "synthesis" can be understood to mean including superposition.

According to one embodiment, the correction device can place a concealer image (630) on a three-dimensional space including a virtual content object (640). The correction device can render an image (650) in which the virtual content object (640) and the concealer image (630) are superimposed together. The correction device can provide a crosstalk-corrected virtual content object to a user by visualizing (e.g., projecting) the left-eye image and the right-eye image generated as a result of the rendering.

In another embodiment, although not illustrated in the drawing, the correction device can render a virtual content object. The correction device can provide a crosstalk-corrected virtual content object to a user by visualizing (e.g., projecting) the left-eye and right-eye images generated as a result of the rendering, overlaid with a concealer image.

FIG. 7 is a flowchart illustrating a method for generating a concealer image according to one embodiment. Referring to FIG. 7, a process in which a correction device according to one embodiment adjusts at least one effect element of a concealer image through steps (710) to (720) is illustrated.

In step (710), the correction device can generate a blurred image by blurring the virtual content object. The method by which the correction device generates the blurred image is described in more detail with reference to FIG. 8 below.

In step (720), the correction device can adjust at least one effect element among the residual image range, contrast, and brightness corresponding to the blurred image. The method by which the correction device adjusts at least one effect element is described in more detail with reference to FIGS. 10 to 12 below.

FIG. 8 is a diagram illustrating a method for generating a blurred image according to one embodiment. Referring to FIG. 8, blurred images (813, 815, 823, 835) blurred based on the unique shape of a virtual content object and resulting images (810, 820, 830) synthesized with the virtual content object are illustrated according to one embodiment.

The correction device can generate blur images (813, 815) and synthesize them with the virtual content object (811) so that the unique shape (e.g., triangular shape) of the virtual content object (811) is maintained, for example, as in the result image (810). The correction device can generate a first blur image (813) corresponding to the left eye of the user and a second blur image (815) corresponding to the right eye based on the shape of the virtual content object (811). At this time, each of the first blur image (813) and the second blur image (815) can correspond to the unique shape of the virtual content object (811).

The correction device can generate a blur image by combining a first blur image (813) and a second blur image (815). The correction device can space the first blur image (813) and the second blur image (815) apart by an interval based on, for example, the parallax of the user's eyes. The correction device can combine the first blur image (813) and the second blur image (815) so that the spaced interval is maintained. At this time, depending on the parallax of the user's eyes, the first blur image (813) and the second blur image (815) may partially overlap, or the first blur image (813) and the second blur image (815) may be spaced apart.

The correction device determines a blur image that combines the first blur image (813) and the second blur image (815), as a result image (810), according to the parallax of the user's eyes, as a concealer image, and can correct crosstalk by synthesizing it with a virtual content object (811). The relationship between the user's eyes and parallax is described in more detail with reference to FIG. 9 below.

Alternatively, the correction device may synthesize the virtual content object (821) and the blurred image (823) generated in a form that expands the unique shape of the virtual content object (821), such as the result image (820). The correction device may expand the virtual content object (821) up, down, left, and right based on a three-dimensional positional relationship including the distance between the positions of the user's two eyes and the positions of virtual images of the virtual content object (821). Based on the three-dimensional positional relationship, the correction device may expand only the upper part of the virtual content object (821), and not expand the lower part, as illustrated in the result image (820), for example, or may expand both the upper and lower parts of the virtual content object (821).

The correction device can generate a blur image (823) corresponding to the expanded form of the virtual content object (821). The correction device can determine the blur image (823) as a concealer image and synthesize it with the virtual content object (821) to correct crosstalk.

In addition, the correction device can generate a blur image (835) by interpolating the left and right of the generated blur image (813, 815) so that the unique shape of the virtual content object is maintained, such as, for example, the result image (830).

The correction device can generate a blur image (835) through interpolation connecting a point (e.g., a vertex) (814) on a spaced blur image (813) and a point (e.g., a vertex) (816) on a blur image (815). The correction device can determine the blur image (835) as a concealer image and correct crosstalk by synthesizing it with a virtual content object (831).

FIG. 9 is a diagram illustrating a principle in which parallax increases as the distance between a user's eyes and a virtual content object increases, according to one embodiment. Referring to FIG. 9, left and right eye images (920) provided to each of the user's eyes (910), a display (930) displaying a virtual content object visualized by the left and right eye images (920), and virtual images (950, 970, 990) of the virtual content object visualized by the left and right eye images (920) are illustrated.

The correction device can provide a three-dimensional graphic object by providing a left-eye image to the user's left eye and a right-eye image to the user's right eye. The left-eye image and the right-eye image may include graphic objects separated from each other by a parallax difference according to depth along the horizontal axis of the display (930). Therefore, the user can perceive the depth of the three-dimensionally rendered graphic object.

The display (930) can provide content with depth to the user by visualizing the graphic objects in which the content is visualized in the left and right eye images (920) as three-dimensional graphic objects by spacing them apart according to the parallax of the user's eyes (910). The parallax can be determined for each pixel of the graphic object, and the sense of depth can be expressed for each pixel. At this time, the parallax of the user's eyes can vary depending on the distance between the user's eyes (910) and the virtual images (950, 970, 990) of the virtual content object.

For example, let's say that a second distance between the user's binoculars (910) and the virtual image (990) of the virtual content object is greater than a first distance between the user's binoculars (910) and the virtual image (950) of the virtual content object. In this case, the parallax of the user's binoculars (910) corresponding to the second distance may increase compared to the parallax of the user's binoculars (910) corresponding to the first distance. In contrast, let's say that a third distance between the user's binoculars (910) and the virtual image (970) of the virtual content object is shorter than the first distance. In this case, the parallax of the user's binoculars (910) corresponding to the third distance may decrease compared to the parallax of the user's binoculars (910) corresponding to the first distance. In this way, as the distance between the user's binoculars (910) and the virtual image of the virtual content object increases, the parallax increases, and as the distance between the user's binoculars (910) and the virtual image of the virtual content object decreases, the parallax may decrease.

FIGS. 10 to 12 are diagrams for explaining a method of adjusting at least one effect element according to embodiments. Referring to FIG. 10, graphs (1010, 1020) illustrating the relationship between parallax or the distance between the position of the user's eyes and the virtual image of the virtual content object and the afterimage range of the concealer image, and drawings (1030, 1040) for explaining the aforementioned relationship are illustrated. Hereinafter, the 'distance' between the position of the user's eyes and the virtual image of the virtual content object may also be referred to as the 'depth' between the position of the user's eyes and the virtual image of the virtual content object.

In a glasses-free 3D display, the distance between the user's binocular positions and the virtual images of virtual content objects may be closely related to the residual image range of crosstalk. The residual image range of crosstalk does not increase linearly with parallax. As shown in graph (1010), it initially increases rapidly and then gradually converges to a constant value as parallax or the distance between the user's binocular positions and the virtual images of virtual content objects increases.

The correction device can also determine the distance between the position of the user's eyes and the virtual image of the virtual content object in the same manner as the light field display shown in the graph (1010), and then render the concealer image by adjusting at least one effect element of the concealer image to correspond to the distance, as shown in the graph (1020).

For example, referring to drawing (1030), a case is shown where the distance between the position of the user's eyes and the virtual image (1031) of the virtual content object is close and a case is shown where the distance between the position of the user's eyes and the virtual image (1033) of the virtual content object is far.

As illustrated in the graph (1020), as the distance between the positions of the user's eyes and the positions of the virtual image of the virtual content object increases, the range of crosstalk may widen, and as the distance decreases, the range of crosstalk may narrow. To eliminate this side effect, the correction device may narrow the range of the concealer image as the distance to the virtual image of the virtual content object decreases. In other words, as the distance between the positions of the user's eyes and the positions of the virtual image (1041) of the virtual content object decreases below the reference distance, as shown on the left side of the drawing (1040), the correction device may gradually reduce the afterimage range (1043) of the concealer image in contrast to the size of the virtual image (1041) of the virtual content object.

In addition, the correction device can expand the range of the concealer image as the distance from the virtual image of the virtual content object increases. In other words, as shown on the right side of the drawing (1040), the correction device can gradually expand the afterimage range (1047) of the concealer image in comparison to the size of the virtual image (1045) of the virtual content object as the distance between the positions of the user's eyes and the positions of the virtual image of the virtual content object increases beyond the reference distance.

Referring to FIG. 11, a drawing (1110) for explaining the relationship between the distance between the position of the user's eyes and the position of a virtual image of a virtual content object and the size of the virtual content object, a drawing (1130) showing the size of the virtual image of the virtual content object according to the distance, and a graph (1130) showing the relationship between the distance and the brightness of the virtual content object are shown.

As illustrated in the drawing (1110), as the distance between the position of the virtual image (1111) of the virtual content object and the position of the user's eyes becomes closer, the size of the virtual image (1111) of the virtual content object may become larger, and as the distance between the position of the virtual image (1115) of the virtual content object and the position of the user's eyes becomes larger, the size of the virtual image (1115) of the virtual content object may become smaller.

The size of the virtual image of the virtual content object according to the distance can be represented, for example, as shown in the drawing (1130). At this time, as the size of the virtual image of the virtual content object increases in the drawing (1130), the area of the virtual image of the virtual content object also increases and the brightness of the virtual image becomes brighter. In this case, the correction device can reduce the sense of incongruity with the surrounding environment by brightly expressing the contrast and brightness of the concealer image corresponding to the virtual image of the virtual content object. For example, the correction device can gradually adjust the contrast and brightness of the concealer image to become brighter as the distance between the positions of the user's eyes and the positions of the virtual image of the virtual content object becomes closer than the reference distance, as shown in the graph (1150).

In addition, as the size of the virtual image of the virtual content object in the drawing (1130) decreases, the area of the virtual image of the virtual content object also decreases and the brightness of the virtual image of the virtual content object darkens. In this case, the correction device can reduce the sense of incongruity with the surrounding environment by darkening the contrast and brightness of the concealer image corresponding to the virtual image of the virtual content object. For example, the correction device can gradually darken the contrast and brightness of the concealer image as the distance between the positions of the user's eyes and the positions of the virtual image of the virtual content object becomes farther than the reference distance, as shown in the graph (1150).

Referring to Figure 12, a graph is shown showing the relationship between the illumination of the surrounding environment and the brightness of the concealer image.

Light sources in the actual surrounding environment can have a significant impact on crosstalk. For example, the brighter the virtual content objects visualized by the head-up display at night, the stronger the crosstalk. The darker the virtual content objects, the weaker the crosstalk.

Accordingly, in one embodiment, in order to prepare for the crosstalk becoming more severe as the brightness of the virtual content object becomes relatively brighter at night when the ambient light level is low, as shown in the graph in FIG. 12, the brightness of the concealer image can be brightened at night, and the brightness of the concealer image can be darkened during the day when the ambient light level is high.

The compensation device can, for example, reduce crosstalk by gradually darkening the brightness of the concealer image as the ambient illuminance increases above the reference illuminance, and by gradually brightening the brightness of the concealer image as the ambient illuminance decreases below the reference illuminance.

FIG. 13 is a flowchart illustrating a method for compensating for crosstalk according to another embodiment. Referring to FIG. 13, a process of a compensation device according to one embodiment compensating for crosstalk through steps (1310) to (1360) is illustrated.

In step (1310), the correction device can detect movement of the virtual content object.

In step (1320), the correction device can determine the position and range in which to place the concealer image by determining the distance moved in step (1310). Here, the position and range in which to place the concealer image may correspond to an area in which crosstalk will occur.

At step (1330), the correction device can generate a concealer image.

In step (1340), the correction device can superposition the concealer image generated in step (1330) on the area where crosstalk occurs, i.e., the location and range where the concealer image determined in step (1320) is to be placed.

In step (1350), the correction device can determine the distance between the position of the virtual image of the virtual content object and the position of the user's two eyes and adjust the contrast of the concealer image superimposed on the area where crosstalk occurs.

At step (1360), the correction device can determine an external light source and adjust the brightness of the concealer image whose contrast was adjusted at step (1350).

Depending on the embodiment, step (1350) and/or step (1360) may be included and performed together in the process of generating a concealer image in step (1330), or may be performed in the form of modifying a concealer image superimposed on an area where crosstalk occurs after generating the concealer image, as in FIG. 13.

The embodiments described above may be implemented using hardware components, software components, and/or a combination of hardware components and software components. For example, the devices, methods, and components described in the embodiments may be implemented using a general-purpose computer or a special-purpose computer, such as, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a programmable logic unit (PLU), a microprocessor, or any other device capable of executing instructions and responding to them. The processing device may execute an operating system (OS) and software applications running on the operating system. Furthermore, the processing device may access, store, manipulate, process, and generate data in response to the execution of the software. For ease of understanding, the processing device is sometimes described as being used alone; however, one of ordinary skill in the art will recognize that the processing device may include multiple processing elements and/or multiple types of processing elements. For example, a processing unit may include multiple processors, or a processor and a controller. Other processing configurations, such as parallel processors, are also possible.

Software may include a computer program, code, instructions, or a combination of one or more of these, which may configure a processing device to perform a desired operation or may, independently or collectively, command the processing device. The software and/or data may be permanently or temporarily embodied in any type of machine, component, physical device, virtual equipment, computer storage medium or device, or transmitted signal wave, for interpretation by the processing device or for providing instructions or data to the processing device. The software may also be distributed over networked computer systems and stored or executed in a distributed manner. The software and data may be stored on a computer-readable recording medium.

The method according to the embodiment may be implemented in the form of program commands that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may include program commands, data files, data structures, etc., alone or in combination, and the program commands recorded on the medium may be those specially designed and configured for the embodiment or may be known and available to those skilled in the art of computer software. Examples of the computer-readable recording medium include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and hardware devices specially configured to store and execute program commands such as ROMs, RAMs, and flash memories. Examples of program commands include not only machine language codes such as those generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter, etc.

The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

Although the embodiments described above have been described with limited drawings, those skilled in the art will appreciate that various technical modifications and variations can be applied based on the described embodiments. For example, appropriate results can still be achieved even if the described techniques are performed in a different order than described, and/or components of the described systems, structures, devices, circuits, etc. are combined or combined in a different manner than described, or are replaced or substituted with other components or equivalents. Therefore, other implementations, other embodiments, and equivalents of the claims also fall within the scope of the claims described below.

Claims (40)

In a method for compensating crosstalk, a device for compensating crosstalk,
A step of estimating an area where crosstalk will occur based on a three-dimensional positional relationship between the positions of the user's two eyes and the positions of virtual images of virtual content objects;
A step of generating a concealer image for correcting the area where the crosstalk occurs based on the estimated area and the virtual content object; and
A step of compensating the crosstalk by synthesizing the virtual content object and the concealer image.
Including,
The above concealer image corresponds to an image rendered only in the area where the crosstalk occurs around the virtual content object,
The range of the above concealer image is determined by the amount of change in parallax according to the distance between the position of the virtual image of the virtual content object and the position of the user's two eyes.
How to compensate for crosstalk. In the first paragraph,
The steps to create the above concealer image are
A step of adjusting at least one effect element among the afterimage range, contrast, and brightness of the above concealer image.
A method for compensating crosstalk, comprising: In the second paragraph,
The step of adjusting at least one of the above effect elements
A step of adjusting the afterimage range of the concealer image in comparison with the size of the virtual image of the virtual content object based on the three-dimensional positional relationship;
A step of adjusting the contrast and brightness of the concealer image based on the three-dimensional positional relationship; and
A step of adjusting the brightness of the concealer image based on the illumination of the surrounding environment.
A method for compensating crosstalk, comprising at least one or a combination thereof. In the third paragraph,
The step of adjusting the afterimage range of the above concealer image is
A step of gradually expanding the afterimage range of the concealer image in contrast to the size of the virtual image of the virtual content object as the distance between the position of the user's eyes and the position of the virtual image becomes further than the reference distance; and
A step of gradually reducing the afterimage range of the concealer image in comparison with the size of the virtual image of the virtual content object as the distance between the position of the user's two eyes and the position of the virtual image becomes closer than the reference distance.
A method for compensating crosstalk, comprising: In the third paragraph,
The steps for adjusting the contrast and brightness of the above concealer image are
A step of gradually darkening the contrast and brightness of the concealer image as the distance between the position of the user's eyes and the position of the virtual image becomes farther than the reference distance; and
A step of gradually brightening the contrast and brightness of the concealer image as the distance between the position of the user's two eyes and the position of the virtual image becomes closer than the reference distance.
A method for compensating crosstalk, comprising: In the third paragraph,
The steps to adjust the brightness of the above concealer image are
A step of gradually darkening the brightness of the concealer image as the illuminance increases above the reference illuminance; and
A step of gradually adjusting the brightness of the concealer image as the above illuminance becomes lower than the above reference illuminance.
A method for compensating crosstalk, comprising: In the first paragraph,
The steps to create the above concealer image are
A step of generating the concealer image by blurring the virtual content object so that it corresponds to the shape of the virtual content object.
A method for compensating crosstalk, comprising: In the first paragraph,
The steps to create the above concealer image are
A step of generating a blurred image by blurring the above virtual content object; and
A step of adjusting at least one effect element among the afterimage range, contrast, and brightness corresponding to the above blurred image.
A method for compensating crosstalk, comprising: In paragraph 8,
The steps for generating the above blurred image are
A step of generating the blur image to correspond to the expanded form of the virtual content object based on the three-dimensional positional relationship.
A method for compensating crosstalk, comprising: In paragraph 8,
The steps for generating the above blurred image are
A step of generating a first blur image corresponding to the left eye and a second blur image corresponding to the right eye among the user's two eyes based on the shape of the virtual content object; and
A step of generating the blurred image by combining the first blurred image and the second blurred image.
A method for compensating crosstalk, comprising: In Article 10,
The step of generating the blurred image by combining the first blurred image and the second blurred image is
A step of separating the first blurred image and the second blurred image by an interval based on the disparity of the user's two eyes; and
A step of combining the first blur image and the second blur image so that the above-mentioned gap is maintained.
A method for compensating crosstalk, comprising: In Article 11,
The step of combining the first blurred image and the second blurred image so that the above-mentioned gap is maintained
A step of generating the blur image by interpolating the first blur image and the second blur image that are spaced apart from each other.
A method for compensating crosstalk, comprising: In the first paragraph,
The step of compensating the above crosstalk is
A step of placing the concealer image on a three-dimensional space including the virtual content object;
A step of generating a left-eye image and a right-eye image by rendering the virtual content object and the concealer image together; and
A step of projecting the above left-eye image and the above right-eye image
A method for compensating crosstalk, comprising: In the first paragraph,
The step of compensating the above crosstalk is
A step of generating a left-eye image and a right-eye image by rendering the above virtual content object; and
A step of projecting the left eye image, the right eye image, and the concealer image.
A method for compensating crosstalk, comprising: In the first paragraph,
The step of estimating the area where the above crosstalk will occur is
A step of estimating an area where the crosstalk will occur based on the parallax between the user's two eyes and the three-dimensional positional relationship.
A method for compensating crosstalk, comprising: In the first paragraph,
The step of estimating the area where the above crosstalk will occur is
A step of estimating an area where the crosstalk will occur by adjusting the range of residual images in the area where the crosstalk will occur based on the parallax of the user's two eyes and the three-dimensional positional relationship.
A method for compensating crosstalk, comprising: In the first paragraph,
A step of detecting the movement of the position of the above virtual content object
Including more,
The step of estimating the area where the above crosstalk will occur is
A step of estimating an area where the crosstalk will occur based on at least one of the three-dimensional positional relationship changed according to the positional movement and the parallax between the user's two eyes.
A method for compensating crosstalk, comprising: In the first paragraph,
The area where the above crosstalk occurs is
A method for correcting crosstalk corresponding to the position and range in which the above concealer image is to be placed. A computer program stored on a computer-readable recording medium for executing any one of the methods of claims 1 to 18 in combination with hardware. A sensor that detects the position of the user's eyes; and
A processor that estimates an area where crosstalk will occur based on a three-dimensional positional relationship between the positions of the user's two eyes and the positions of virtual images of virtual content objects, generates a concealer image for correcting the area where crosstalk will occur based on the estimated area and the virtual content object, and corrects the crosstalk by synthesizing the virtual content object and the concealer image.
Including,
The above concealer image corresponds to an image rendered only in the area where the crosstalk occurs around the virtual content object,
A device for compensating crosstalk, wherein the range of the above concealer image is determined according to the amount of change in parallax according to the distance between the position of the virtual image of the virtual content object and the position of the user's two eyes. In Article 20,
The above processor
A device for compensating crosstalk, which adjusts at least one effect element among the afterimage range, contrast, and brightness of the above concealer image. In Article 21,
The above processor
A device for correcting crosstalk, which performs at least one or a combination of the following operations: an operation of adjusting an afterimage range of the concealer image in comparison with the size of an image of the virtual content object based on the three-dimensional positional relationship; an operation of adjusting the contrast and brightness of the concealer image based on the three-dimensional positional relationship; and an operation of adjusting the brightness of the concealer image based on the illuminance of the surrounding environment. In paragraph 22,
The above processor
A device for correcting crosstalk, which gradually expands the afterimage range of the concealer image in comparison with the size of the virtual image of the virtual content object as the distance between the positions of the user's eyes and the positions of the virtual image becomes further than the reference distance, and gradually reduces the afterimage range of the concealer image in comparison with the size of the virtual image of the virtual content object as the distance between the positions of the user's eyes and the positions of the virtual image becomes closer than the reference distance. In Article 23,
The above processor
A crosstalk correction device that gradually darkens the contrast and brightness of the concealer image as the distance between the positions of the user's eyes and the positions of the virtual image becomes farther than the reference distance, and gradually brightens the contrast and brightness of the concealer image as the distance between the positions of the user's eyes and the positions of the virtual image becomes closer than the reference distance. In paragraph 22,
The above processor
A device for compensating crosstalk, which gradually darkens the brightness of the concealer image as the illuminance increases above the reference illuminance, and gradually brightens the brightness of the concealer image as the illuminance decreases below the reference illuminance. In Article 20,
The above processor
A device for correcting crosstalk, which creates the concealer image by blurring the virtual content object so that it corresponds to the shape of the virtual content object. In Article 20,
The above processor
A device for compensating crosstalk, which generates a blurred image by blurring the above virtual content object and adjusts at least one effect element among the afterimage range, contrast, and brightness corresponding to the blurred image. In Article 27,
The above processor
A device for correcting crosstalk that generates the blurred image so as to correspond to the expanded form of the virtual content object based on the three-dimensional positional relationship. In Article 27,
The above processor
A device for correcting crosstalk, which generates a first blur image corresponding to the left eye and a second blur image corresponding to the right eye among the user's two eyes based on the shape of the virtual content object, and generates the blur image by combining the first blur image and the second blur image. In Article 29,
The above processor
A crosstalk correction device that separates the first blur image and the second blur image by an interval based on the parallax between the user's two eyes, and combines the first blur image and the second blur image so that the separated interval is maintained. In Article 30,
The above processor
A device for correcting crosstalk that generates the blur image by interpolating the first blur image and the second blur image that are spaced apart from each other. In Article 20,
The above processor
A device for correcting crosstalk, which places the concealer image on a three-dimensional space including the virtual content object, renders the virtual content object and the concealer image together to generate a left-eye image and a right-eye image, and projects the left-eye image and the right-eye image. In Article 20,
The above processor
A device for correcting crosstalk by rendering the above virtual content object, generating a left-eye image and a right-eye image, and projecting the left-eye image, the right-eye image, and the concealer image. In Article 20,
The above processor
A device for compensating crosstalk, which estimates an area where crosstalk will occur based on the parallax of the user's two eyes and the three-dimensional positional relationship. In Article 20,
The above processor
A device for compensating crosstalk, which estimates an area where crosstalk occurs by adjusting the range of residual images in the area where crosstalk occurs based on the parallax of the user's two eyes and the three-dimensional positional relationship. In Article 20,
The above processor
A device for compensating crosstalk, which detects a positional movement of the virtual content object and estimates an area where the crosstalk will occur based on at least one of the three-dimensional positional relationship changed according to the positional movement and the parallax between the user's eyes. In Article 20,
The area where the above crosstalk occurs is
A device for correcting crosstalk corresponding to the position and range in which the above concealer image is to be placed. In content visualization devices,
A display that visualizes virtual content objects by projecting them onto a projection plane; and
A processor that estimates an area where crosstalk will occur based on a three-dimensional positional relationship between the positions of the user's two eyes and the positions of virtual images of the virtual content object, generates a concealer image for correcting the area where crosstalk will occur based on the estimated area and the virtual content object, and corrects the crosstalk by synthesizing the virtual content object and the concealer image.
Including,
The above concealer image corresponds to an image rendered only in the area where the crosstalk occurs around the virtual content object,
A content visualization device, wherein the range of the above concealer image is determined according to the amount of change in parallax according to the distance between the position of the virtual image of the virtual content object and the position of the user's two eyes. In the case of augmented reality glasses (AR glass) devices,
A transparent display that visualizes virtual content objects; and
A processor that estimates an area where crosstalk will occur based on a three-dimensional positional relationship between the positions of the user's two eyes and the positions of virtual images of the virtual content object, generates a concealer image for correcting the area where crosstalk will occur based on the estimated area and the virtual content object, and corrects the crosstalk by synthesizing the virtual content object and the concealer image.
Including,
The above concealer image corresponds to an image rendered only in the area where the crosstalk occurs around the virtual content object,
An augmented reality glasses device, wherein the range of the concealer image is determined according to the amount of change in parallax according to the distance between the position of the virtual image of the virtual content object and the position of the user's two eyes. In the vehicle,
A sensor for sensing the position of the vehicle;
A head-up display (HUD) that visualizes virtual content objects to be provided to the user on a projection plane; and
A processor that determines the virtual content object based on the location of the vehicle, estimates an area where crosstalk will occur based on a three-dimensional positional relationship between the locations of the user's two eyes and the virtual location of the virtual content object, generates a concealer image for correcting the area where the crosstalk will occur based on the estimated area and the virtual content object, and synthesizes the virtual content object and the concealer image to correct the crosstalk.
Including,
The above concealer image corresponds to an image rendered only in the area where the crosstalk occurs around the virtual content object,
The range of the above concealer image is determined by the amount of change in parallax according to the distance between the position of the virtual image of the virtual content object and the position of the user's two eyes.