CN118732267A

CN118732267A - Display control method, display control device and head-up display device

Info

Publication number: CN118732267A
Application number: CN202410352988.1A
Authority: CN
Inventors: 秦诚
Original assignee: Nippon Seiki Co Ltd
Current assignee: Nippon Seiki Co Ltd
Priority date: 2023-03-28
Filing date: 2024-03-26
Publication date: 2024-10-01
Also published as: JP2024140723A; DE102024107766A1

Abstract

The present invention relates to a display control method, a display control device and a head-up display device, which suppress the reduction of the virtual reality sense of an image (virtual object) caused by the posture change of a vehicle. The head-up display device at least displays a first virtual image displayed in a first area of a virtual image display area overlapping with a road surface in front of the vehicle and a second virtual image displayed in a second area closer to the end of the virtual image display area in the up-down direction than the first area. In order to suppress the relative positional deviation between the virtual image and the road surface caused by the posture change of the vehicle, a first position adjustment amount that changes dynamically is set in combination with posture change amount information. When the posture of the vehicle changes, a first image adjustment process that adjusts the position of the first virtual image according to the first position adjustment amount is performed, and a second image adjustment process that adjusts the position of the second virtual image according to the second position adjustment amount is performed to suppress the adjustment of the position of the virtual image relative to the posture change of the vehicle compared with the first image adjustment process.

Description

Display control method, display control device and head-up display device

Technical Field

The present invention relates to a display control device, a head-up display device, a display control method, and the like for visually observing a moving body such as a vehicle by superimposing an image on the foreground of the moving body (the real scene in the forward direction of the moving body, which is observed from an occupant of the vehicle).

Background

A Head Up Display (HUD) device can Display an image (virtual object) superimposed on a scene in front of a vehicle, and express augmented reality (AR: augmented Reality) obtained by adding/emphasizing information to the real scene or a real object existing in the real scene, thereby suppressing the movement of the line of sight of a user driving the vehicle as much as possible, and accurately providing desired information, thereby contributing to safe and comfortable vehicle running.

The head-up display device described in patent document 1 displays an image (virtual object) at a display reference position, and moves the image to the outside of the display reference position with the posture change of the vehicle so as to suppress a relative positional shift between the image (virtual object) and the real scene due to the posture change of the vehicle. Thus, even if the vehicle posture fluctuates, the relative positional relationship between the image and the live view can be maintained (positional displacement between the image and the live view is suppressed), and therefore the image and the live view can be further coordinated.

Prior art literature

Patent literature

Patent document 1: international publication No. 2018/088362

Disclosure of Invention

Problems to be solved by the invention

The region of the live-action where the images are visually observed in an original overlapping manner before the posture of the vehicle changes is referred to as an overlapping live-action region. In the head-up display device, when the virtual image display region is shifted to such an extent that it cannot overlap with the overlapping live-action region due to a large posture variation of the vehicle, if the position of the image is adjusted so as to match the large posture variation, a part or all of the adjusted image does not fall within the virtual image display region (cannot be displayed). If a part or all of the image is missing due to the posture change, it is considered that the virtual reality of the virtual image is impaired, giving the observer an uncomfortable feeling.

In addition, when the position of the image is not adjusted according to the posture change in order to avoid the shortage of a part or the whole of the image, the relative positional relationship between the image and the live view changes so as to match the posture change of the vehicle, and it is considered that the virtual reality is impaired, giving the observer a sense of incongruity.

A brief description of specific embodiments disclosed in the present specification is shown below. It should be understood that these aspects are presented merely to provide the reader with a brief summary of these particular embodiments and that these aspects are not intended to limit the scope of the invention. In practice, the present invention may include a combination of the embodiments described below and various embodiments not described below.

The summary of the present invention relates to reducing the sense of incongruity of visual observation of virtual images. More specifically, the present invention relates to suppressing a decrease in the virtual reality of an image (virtual object) due to a change in the posture of a vehicle.

Accordingly, the display control method, the display control device, and the head-up display device described in the present specification adopt the following means to solve the above-described problems. The gist of the present embodiment is to perform a first image adjustment process for adjusting the position of a first virtual image displayed in a first region in a virtual image display region according to a first position adjustment amount, and to perform a second image adjustment process for adjusting the position of a second virtual image displayed in a second region closer to the end of the virtual image display region in the up-down direction than the first region according to a second position adjustment amount for suppressing adjustment of the position of the virtual image with respect to the change in the posture of the vehicle according to the first image adjustment process.

Accordingly, the display control method according to the first embodiment described in the present specification controls a head-up display device that is mounted on a vehicle and includes a display unit that displays an image on a display surface, and that visually observes a virtual image by projecting light of the image onto a projection unit so that the virtual image overlaps with a virtual image display region overlapping a road surface in front of the vehicle, the method including:

Displaying at least a first virtual image displayed in a first region within the virtual image display region and a second virtual image displayed in a second region closer to an end of the virtual image display region in the up-down direction than the first region;

acquiring posture change amount information indicating a posture change of the vehicle;

in order to suppress relative positional displacement of the virtual image and the road surface due to the posture variation of the vehicle, a first position adjustment amount that dynamically changes is set in conjunction with the posture variation amount information;

in the case where the posture of the vehicle is changed,

1) Performing a first image adjustment process of adjusting the position of the first virtual image in accordance with the first position adjustment amount;

2) And performing a second image adjustment process of adjusting the position of the second virtual image according to a second position adjustment amount that suppresses adjustment of the position of the virtual image with respect to the change in the posture of the vehicle, compared to the first image adjustment process. In this way, it is considered that the position adjustment is performed on the second virtual image near the end portion in the virtual image display region according to the posture change of the vehicle, thereby suppressing the reduction of the virtual reality, and the amount of the position adjustment is suppressed, so that it is possible to prevent that a part or all of the second virtual image cannot be completely seen (does not fall into the virtual image display region), while the positional displacement between the virtual image and the real scene due to the posture change of the vehicle in the first virtual image near the central portion in the vertical direction in the virtual image display region is significantly reduced.

In the display control method of the second embodiment which can be subordinate to the first embodiment, further comprising:

Further acquiring information indicating a prediction of the attitude change of the vehicle or prediction-related information including a predicted value of the attitude change of the vehicle;

in the case where the prediction of the change in the posture of the vehicle is the first prediction state based on the prediction related information,

2) A second image adjustment process of performing adjustment of the position of the second virtual image that suppresses a change in the posture of the virtual image with respect to the vehicle than the first image adjustment process;

in the case where the predicted posture change of the vehicle is the second predicted state smaller than the first predicted state,

3) A first image adjustment process of adjusting the positions of the first virtual image and the second virtual image in accordance with the first position adjustment amount is performed. In this way, it is also considered that the position adjustment of the second virtual image, which is highly likely to be generated in a complete manner, can be quickly suppressed when the posture change of the vehicle is predicted to be large, and the position shift between the virtual image and the real scene due to the posture change of the vehicle can be significantly reduced without suppressing the position adjustment when the posture change of the vehicle is predicted to be small.

The display control method according to the third embodiment, which can be applied to the first embodiment or the second embodiment, further includes: if it is determined that the vehicle posture is changed greatly or the vehicle posture is changed frequently, the number of contents of the second image adjustment process is increased. This also has the advantage of automatically expanding the range of application of the second image adjustment process for suppressing the incomplete visibility in accordance with the posture change of the vehicle. In the display control method according to the third embodiment, when it is determined that the vehicle posture is changed greatly or the vehicle posture is changed frequently, the second region of the second virtual image, which is regarded as the second image adjustment processing, is enlarged toward the center in the vertical direction of the virtual image display region. That is, in the display control method according to the third embodiment, when the second region is disposed below the virtual image display region, the second region can be enlarged upward. Thus, the second image adjustment processing can be sequentially applied from the end portion near the virtual image display region which is likely to be incompletely seen.

In the display control method of a fourth embodiment which can be dependent on the first to third embodiments, further comprising: when the vehicle posture change is equal to or greater than a predetermined threshold value, the position of the second virtual image is switched from the second image adjustment processing to the first image adjustment processing and adjusted when the vehicle posture change is less than the predetermined threshold value. This has the advantage of automatically switching to the first image adjustment process for determining that the vehicle is not likely to be completely seen, and significantly reducing the positional shift between the virtual image and the real image due to the change in the posture of the vehicle.

In the display control method of a fifth embodiment which can be dependent on the first to fourth embodiments, further comprising: when the second image position adjustment process is executed, a third position adjustment process is executed for the first and second virtual images, in which adjustment of the position of the virtual image with respect to the posture change of the vehicle is suppressed as compared with the second image position adjustment process, in a case where the second virtual image is changed from a state that is disposed within a second position adjustment range of the virtual image display region to a state that is disposed outside the second position adjustment range. Thus, it is also considered that in the case where the second image position adjustment processing causes incomplete viewing, the adjustment amount of the position of the virtual image with respect to the posture variation of the vehicle is further suppressed, thereby more significantly preventing the incomplete viewing of the virtual image. In addition, in the display control method according to another fifth embodiment, further comprising: when the second image position adjustment process is executed, a third position adjustment process is executed for fixing the positions of the first and second virtual images with respect to the change in the posture of the vehicle in a case where the second virtual image changes from a state in which the second virtual image is disposed within a second position adjustment range of the virtual image display region to a state in which the second virtual image is disposed outside the second position adjustment range. Thus, it is also considered that in the case where the second image position adjustment processing causes incomplete visibility, the position of the virtual image with respect to the posture variation of the vehicle is fixed, thereby completely preventing the incomplete visibility of the virtual image.

The display control method according to a sixth embodiment, which can be applied to the first to fifth embodiments, further includes: disposing the second virtual image so as to be visually observed under the first virtual image;

When the vehicle is tilted forward and the first virtual image offset when the first image position adjustment process is performed is changed from a state of being disposed in the first position adjustment range to a state of being disposed on the upper side of the first position adjustment range, a third image position adjustment process is performed in which the position of the first virtual image is fixed to the upper peripheral edge portion in the first position adjustment range and the position of the second virtual image is fixed to a predetermined position below the upper peripheral edge portion in the first position adjustment range. Thus, it is also considered that the advantage is to prevent one virtual image from being fixed and the other virtual image from continuing the position correction.

The display control method according to a seventh embodiment which can be applied to the first to sixth embodiments further includes: disposing the second virtual image so as to be visually observed under the first virtual image;

When the vehicle is tilted backward and the second virtual image is changed from a state of being disposed in the second position adjustment range to a state of being disposed at the lower side of the second position adjustment range, the third image position adjustment process is performed in which the position of the second virtual image is fixed to the lower peripheral edge portion in the second position adjustment range and the position of the first virtual image is fixed to a predetermined position above the lower peripheral edge portion in the second position adjustment range. Thus, it is also considered that the advantage is to prevent one virtual image from being fixed and the other virtual image from continuing the position correction.

The display control method according to an eighth embodiment which can be applied to the first to seventh embodiments, further includes: the second position adjustment amount is changed according to the position of the second virtual image in the vertical direction in the virtual image display area, and is reduced as the position of the second virtual image is closer to the end of the vertical direction in the virtual image display area. Thus, it is also considered that there is an advantage that the observer does not easily feel uncomfortable due to the difference in the amount of correction of the position between the plurality of virtual images in the virtual image display region.

The display control device according to the ninth embodiment controls a head-up display device that is mounted on a vehicle, includes a display unit that displays an image on a display surface, and visually observes a virtual image by projecting light of the image onto a projection unit so that the virtual image overlaps a virtual image display region overlapping a road surface in front of the vehicle, and includes:

One or more control circuits;

A memory; and

One or more computer programs, stored in the memory, configured in a manner to be executed by the one or more control circuits,

In the control circuit of the present invention,

in order to suppress relative positional displacement of the virtual image and the road surface due to the change in the posture of the vehicle, a first position adjustment amount that dynamically changes is set in association with the posture change amount information, and in the case of the change in the posture of the vehicle,

In the display control apparatus pertaining to the tenth embodiment, which is dependent on the ninth embodiment, information indicating a prediction of the posture change of the vehicle or prediction-related information including a predicted value of the posture change of the vehicle is further acquired,

The control circuit is based on the prediction related information,

In the case where the prediction of the posture variation of the vehicle is the first prediction state,

2) A second image adjustment process of performing adjustment of the position of the second virtual image with respect to the posture change of the vehicle that is suppressed more than the first image adjustment process,

3) A first image adjustment process of adjusting the positions of the first virtual image and the second virtual image in accordance with the first position adjustment amount is performed.

In the display control apparatus pertaining to the eleventh embodiment which can be attached to the ninth embodiment or the tenth embodiment, when the control circuit determines that the posture of the vehicle is changed greatly or that the posture of the vehicle is changed frequently, the number of contents of executing the second image adjustment processing is increased.

In the display control device according to a twelfth aspect which can be applied to the ninth to eleventh aspects, the control circuit changes the position of the second virtual image from the second image adjustment processing to the first image adjustment processing to adjust the position of the second virtual image when the posture change of the vehicle becomes less than the predetermined threshold after the posture change of the vehicle becomes equal to or greater than the predetermined threshold.

In the display control apparatus according to the thirteenth embodiment, which is applicable to the ninth to twelfth embodiments, when the second image position adjustment process is performed, the control circuit further performs a third position adjustment process for suppressing adjustment of the position of the virtual image with respect to the posture change of the vehicle, with respect to the second image position adjustment process, on the first and second virtual images when the second virtual image changes from a state in which the second virtual image is disposed within the second position adjustment range of the virtual image display region to a state in which the second virtual image is disposed outside the second position adjustment range.

In the display control apparatus of the fourteenth embodiment, which may be dependent on the ninth to thirteenth embodiments, the control circuit configures the second virtual image so as to be visually observed under the first virtual image,

When the vehicle is tilted forward and the first virtual image offset when the first image position adjustment process is performed is changed from a state of being disposed in the first position adjustment range to a state of being disposed on the upper side of the first position adjustment range, a third image position adjustment process is performed in which the position of the first virtual image is fixed to the upper peripheral edge portion in the first position adjustment range and the position of the second virtual image is fixed to a predetermined position below the upper peripheral edge portion in the first position adjustment range.

In the display control apparatus of the fifteenth embodiment which can be attributed to the ninth embodiment to the fourteenth embodiment,

The control circuit configures the second virtual image in such a way as to make a visual observation under the first virtual image,

When the vehicle is tilted backward and the second virtual image is changed from a state of being disposed in a second position adjustment range to a state of being disposed at a lower side of the second position adjustment range, a third image position adjustment process is further performed in which a position of the second virtual image is fixed to a lower peripheral edge portion in the second position adjustment range and a position of the first virtual image is fixed to a predetermined position above the lower peripheral edge portion in the second position adjustment range.

In the display control apparatus according to a sixteenth embodiment which can be applied to the ninth to fifteenth embodiments, the control circuit changes the second position adjustment amount in accordance with the position of the second virtual image in the virtual image display region, and decreases the second position adjustment amount as the position of the second virtual image is closer to the end in the vertical direction of the virtual image display region.

The head-up display device according to the seventeenth embodiment includes a display unit mounted on a vehicle and configured to display an image on a display surface, and configured to visually observe a virtual image by projecting light of the image onto a projected unit so that the virtual image overlaps a virtual image display region overlapping a road surface in front of the vehicle, the head-up display device including:

One or more control circuits;

A memory; and

In the control circuit of the present invention,

attitude change amount information indicating a change in the attitude of the vehicle is acquired,

Drawings

Fig. 1 is a diagram showing an example of application of the vehicle display system according to the present embodiment to a vehicle.

Fig. 2 is a diagram showing a structure of the head-up display device.

Fig. 3 is a block diagram of a display system for a vehicle according to several embodiments.

Fig. 4 is a diagram illustrating a model space of configuration content and virtual viewpoints, according to several embodiments.

Fig. 5 is a diagram showing an example of a foreground visually observed by an observer and an image (virtual image) displayed overlapping the foreground while the vehicle is traveling.

Fig. 6 is a diagram showing a virtual image before the position adjustment process, the left diagram showing a relationship between the virtual image and the virtual plane, and the right diagram showing a foreground and the virtual image visually observed when the observer faces forward.

Fig. 7A is a diagram showing a virtual image after the first image adjustment process, the left diagram showing the relationship between the virtual image and the virtual plane, and the right diagram showing the foreground and the virtual image visually observed when the observer faces forward.

Fig. 7B is a diagram showing that the processed virtual image is suppressed from being completely seen, the left diagram shows the relationship between the virtual image and the virtual plane, and the right diagram shows the foreground and the virtual image visually observed when the observer faces forward.

Fig. 8 is a diagram for explaining the limitation of the position adjustment amount.

Fig. 9 is a diagram for explaining the limitation of the position adjustment amount.

Fig. 10 is a diagram showing a flow of display control according to several embodiments.

Fig. 11 is a diagram illustrating an image adjustment process with respect to a posture change, and shows an example in which the first image adjustment process or the second image adjustment process is executed according to the content.

Fig. 12 is a diagram illustrating an image adjustment process with respect to a posture change, and shows an example in which the first image adjustment process or the second image adjustment process is executed according to the content.

Fig. 13 is a diagram illustrating a position adjustment process with respect to a posture change, and shows an example in which the first position adjustment process or the second position adjustment process is executed according to the content.

Fig. 14 is a diagram illustrating a position adjustment process with respect to a posture change, and shows an example in which the first position adjustment process or the second position adjustment process is executed according to the content.

Symbol description

1: A vehicle; 2: a projected section; 6: road surface; 10: a display system for a vehicle; 20: HUD device (head-up display device); 21: a light exit window; 22: a frame; 30: a display control device; 31: an I/O interface; 33: a control circuit; 35: an image processing circuit; 37: a memory; 40: an image display device; 50: a display; 50a: a display surface; 52: an optical layer; 60: a light source unit; 80: a relay optical system; 100: a virtual plane; 200: an eye movement range; 205: a center; 401: a vehicle ECU;403: a road information database; 405: a vehicle position detection unit; 407: an operation detection unit; 409: an eye position detection unit; 411: an off-vehicle sensor; 413: a brightness detection unit; 415: a posture detecting section; 417: a portable information terminal; 419: an external communication device; 510: a drawing module; 520: an image adjustment module; 522: a position adjustment module; 524: a depression angle adjustment module; 526: a size adjustment module; 700: an observer; c: a position adjustment amount; c10: a first position adjustment amount; c20: a second position adjustment amount; CT: a limit; CTd: a limit; CTu: a limit; e: a depression angle adjustment amount; f: a size adjustment amount; m: an image; MP: a target location; PO: a reference position; v10: a first virtual image; v20: a second virtual image; VP: a virtual viewpoint; VS: a virtual image display region; VS01: vertical face; VS02: an inclined surface; VS03: a road surface overlapping surface; VS10: a first region; VS20: a second region; VS21: a second region; VS22: a second region; VT: a position adjustment range; VT1: a first position adjustment range; VT2: a second position adjustment range; alpha: a pitch angle; α0: a reference gesture; αtd2: a gesture threshold; αtu2: a gesture threshold; αth1: a threshold value; αth2: a threshold value; beta: depression angle.

Detailed Description

Hereinafter, in fig. 1 to 14, a description is provided of the structure and operation of an exemplary vehicle display system. The present invention is not limited to the following embodiments (including the contents of the drawings). Naturally, the following embodiments may be modified (including deletion of the constituent elements). In the following description, well-known technical matters are appropriately omitted in order to facilitate understanding of the present invention.

Reference is made to fig. 1. Fig. 1 is a diagram showing an example of a configuration of a virtual image display system for a vehicle. In fig. 1, the left-right direction of the vehicle (an example of a moving body) 1 (in other words, the width direction of the vehicle 1) is defined as the X-axis (the positive direction of the X-axis is the left direction when the vehicle 1 faces forward), the vertical direction along a line segment orthogonal to the ground or the surface corresponding to the ground (in this case, the road surface 6) is defined as the Y-axis (the positive direction of the Y-axis is the upper direction), and the front-back direction along a line segment orthogonal to the left-right direction and the vertical direction is defined as the Z-axis (the positive direction of the Z-axis is the straight direction of the vehicle 1). In this respect, the other figures are also identical.

As shown in the drawings, a vehicle display system 10 provided in a vehicle (moving object) 1 includes: an eye position detection unit (line of sight detection unit) 409 for detecting the positions or line of sight of left eye 700L and right eye 700R of an observer (typically, a driver sitting in the driver seat of vehicle 1); an off-vehicle sensor 411 configured by a camera (for example, a stereo camera) or the like that photographs the front (broadly, the periphery) of the vehicle 1; a posture detecting section 415 that detects a posture of the vehicle 1; a head-up display device (hereinafter, also referred to as HUD device) 20; and a display control device 30 that controls the HUD device 20. In addition, the eye position detection unit (line-of-sight detection unit) 409 and the off-vehicle sensor 411 may be omitted.

Fig. 2 is a diagram showing a configuration of the head-up display device 20. The HUD device 20 is disposed, for example, in a dash panel (symbol 5 of fig. 1). The HUD device 20 has: an image display device (display section) 40; a relay optical system 80; and a housing 22 that houses the image display device 40 and the relay optical system 80, and that has a light exit window 21 through which display light K from the image display device 40 can be emitted from the inside to the outside.

The image display device (display unit) 40 is herein referred to as a parallax type 3D display device. The stereoscopic display device (parallax type 3D display device) 40 is composed of a display 50 and a light source unit 60, and the display 50 is an autostereoscopic display device using a multi-viewpoint image display system capable of controlling depth expression by visually observing left and right viewpoint images; the light source unit 60 functions as a backlight. The image display device (display unit) 40 is not limited to a stereoscopic image display device that displays a 3D image, and may be a display device that displays a 2D image.

The display 50 has: a display surface 50a for modulating the illumination light from the light source unit 60 to generate an image M on the display surface 50 a; and an optical layer (an example of a light separation unit) 52 having, for example, a lenticular lens, a parallax barrier (parallax barrier), or the like, for separating light emitted from the display surface 50a into left-eye display light (fig. 1 symbol K10) such as left-eye light rays K11, K12, and K13, and right-eye display light (fig. 1 symbol K20) such as right-eye light rays K21, K22, and K23. The optical layer 52 includes filters such as lenticular lenses, parallax barriers, lens arrays, and microlens arrays. In the embodiment, the optical layer 52 is not limited to the above-described optical filter, and may include any optical layer disposed in front of or behind the display surface 50 a. However, this is an example and is not limiting.

In addition, the image display device 40 may be configured to emit left-eye display light (fig. 1 symbol K10) such as left-eye light rays K11, K12, and K13 and right-eye display light (fig. 1 symbol K20) such as right-eye light rays K21, K22, and K23 by configuring the light source unit 60 with a directional backlight unit (an example of a light ray separation unit) instead of or in addition to the optical layer (an example of a light ray separation unit). Specifically, for example, when the directional backlight unit irradiates illumination light toward the left eye 700L, the display control device 30 described later causes the display surface 50a to display a left-view image, thereby causing left-eye display light K10 such as left-eye light rays K11, K12, and K13 to face the left eye 700L of the observer, and when the directional backlight unit irradiates illumination light toward the right eye 700R, causes the display surface 50a to display a right-view image, thereby causing right-eye display light K20 such as right-eye light rays K21, K22, and K23 to face the left eye 700L of the observer. However, this is an example and is not limiting.

The display control device 30, which will be described later, controls the manner in which the HUD device 20 displays (perceives) the content FU by performing, for example, an image drawing process (a patterning process), a display driving process, and the like, by directing the original left-eye display light K10, which is the left-viewpoint image V1, to the left eye 700L of the observer, and directing the original right-eye display light K20, which is the right-viewpoint image V12, to the right eye 700R, and adjusting the left-viewpoint image V1 and the right-viewpoint image V2. The display control device 30 described later can control the display (display 50) so as to reproduce a light field in which light rays output in various directions from points or the like existing in a constant space are directly (approximately) reproduced.

The intermediate transfer optical system 80 includes curved mirrors (concave mirrors, etc.) 81 and 82 that reflect light from the image display device 40 and project display light K10 and K20 of an image onto the windshield (projection target portion) 2. But may further have other optical components (refractive optical components such as lenses, diffractive optical components such as holograms, reflective optical components, or a combination of these may be included).

In fig. 1, an image display device 40 of the HUD device 20 displays an image (parallax image) having parallax for each of the left and right eyes. As shown in fig. 1, each parallax image is displayed as V10 imaged on a virtual image display area (virtual image imaging plane) VS. The focal point of each eye of the observer (person) is adjusted to be in focus with the position of the virtual image display area VS. The position of the virtual image display region VS is referred to as an "adjustment position (or imaging position)", and the distance from a predetermined reference position (for example, the center 205 of the eye movement range 200 of the HUD device 20, the viewpoint position of the observer, or the specific position of the vehicle 1) to the virtual image display region VS is referred to as an adjustment distance (imaging distance).

The virtual image display region VS is a virtual (apparent) surface set in the real space in front of the occupant (visual observer such as driver) in correspondence with the display surface 50a of the display 50. The virtual image display region VS includes, for example, an upright surface VS01 perpendicular to the road surface 6, an inclined surface VS02 inclined with respect to the road surface 6, a road surface overlapping surface VS03 overlapping the road surface 6, a surface (not shown) having an upright surface (including a pseudo upright surface) on a side close to an occupant (visual observer) and an inclined surface on a side away from the occupant. In the display using the other surface than the standing surface VS01, the display distance of the virtual image differs depending on the display position on the virtual image display region, so that the depth expression can be performed.

Fig. 3 is a block diagram of a virtual image display system for a vehicle according to several embodiments. The display control device 30 includes: one or more I/O interfaces 31, one or more control circuits 33, one or more image processing circuits 35, and one or more memories 37. Fig. 3 is merely one embodiment, and the illustrated components may be combined with a smaller number of components or may have additional components. For example, the image processing circuit 35 (e.g., a graphics processing unit) may be included in one or more control circuits 33.

As shown in the figure, the control circuit 33 and the image processing circuit 35 are operatively coupled to a memory 37. More specifically, the control circuit 33 and the image processing circuit 35 can control the vehicle display system 10 (the image display device 40), for example, generate and/or transmit image data, by executing a program stored in the memory 37. The control circuit 33 and/or the image processing circuit 35 may comprise at least one general purpose micro control circuit (e.g., a Central Processing Unit (CPU)), at least one Application Specific Integrated Circuit (ASIC), at least one Field Programmable Gate Array (FPGA), or any combination thereof. The memory 37 includes any type of magnetic medium such as a hard disk, any type of optical medium such as a CD and DVD, any type of semiconductor memory such as a volatile memory, and a nonvolatile memory. Volatile memory may include DRAM and SRAM, and non-volatile memory may include ROM and NVRAM.

As shown, the control circuit 33 is operatively coupled to the I/O interface 31. The I/O interface 31 communicates (also referred to as CAN communication) with, for example, a vehicle ECU401 and/or other electronic devices (symbols 403 to 419 described below) provided in the vehicle according to the standard CAN (Controller Area Network). The communication standard used for the I/O interface 31 is not limited to CAN, and includes, for example, a wired communication interface such as CANFD (CAN with Flexible Data Rate), LIN (Local Interconnect Network), ethernet (registered trademark), MOST (Media Oriented Systems Transport: MOST is registered trademark), UART, USB, or the like, or an in-vehicle communication (internal communication) interface such as a Personal Area Network (PAN) such as a Bluetooth (registered trademark) network, a Local Area Network (LAN) such as an 802.11xWi-Fi (registered trademark) network, or the like, which is a short-range wireless communication interface within several tens of meters. The I/O interface 31 may include an off-vehicle communication (external communication) interface such as a wireless wide area network (WWAN 0, IEEE802.16-2004 (WiMAX: worldwide Interoperability for Microwave Access)), a wide area communication network (e.g., an internet communication network) in accordance with a cellular communication standard such as IEEE802.16 e-based (Mobile WiMAX), 4G-LTE, LTE ADVANCED, 5G, and the like.

As shown in the drawing, the control circuit 33 is connected to the I/O interface 31 so as to be capable of mutually operating, and can exchange information with various other electronic devices and the like connected to the vehicle display system 10 (I/O interface 31). The I/O interface 31 is operatively connected to, for example, a vehicle ECU401, a road information database 403, a vehicle position detecting unit 405, an operation detecting unit 407, an eye position detecting unit 409, an off-vehicle sensor 411, a brightness detecting unit 413, an attitude detecting unit 415, a portable information terminal 417, an external communication device 419, and the like. The I/O interface 31 may include a function of processing (converting, calculating, analyzing) information received from other electronic devices or the like connected to the vehicle display system 10.

The image display device 40 is operatively coupled to the control circuit 33 and the image processing circuit 35. Thus, in several embodiments, the image displayed by the display surface 50a may be based on image data received from the control circuit 33 and/or the image processing circuit 35. The control circuit 33 and the image processing circuit 35 control (adjust) the image displayed on the display surface 50a based on the information acquired from the I/O interface 31.

The software components stored in the memory 37 include a drawing module 510 and an image adjustment module 520 (a position adjustment module 522, a depression angle adjustment module 524, and a size adjustment module 526).

The drawing module 510 forms an image M based on information (navigation information, vehicle information, etc.) acquired by the display control device 30, and temporarily stores the formed image M in a buffer (not shown). The image M displayed on the display 50 is visually observed by the observer 700 as a virtual image V20. At this time, the virtual image V20 expresses the content FU.

Fig. 4 is a diagram illustrating a model space configuring the content FU and the virtual viewpoint VP. In fig. 4, the coordinate system of the virtual viewpoint VP has a depth direction of Z1 axis, a left-right direction of X1 axis (corresponding to the width direction X of the vehicle 1), and an up-down direction of Y1 axis (corresponding to the up-down direction Y of the vehicle 1). The rendering module 510 performs calculation of each vertex data of the rendered content FU in each rendering frame. In this case, a model space of each content FU is constructed. Then, data of each drawn vertex is calculated on a "model coordinate system (local coordinate system)" for each virtual object. The rendering module 510 converts the content FU rendered in the model coordinate system into a two-dimensional image by projecting the content FU onto a predetermined projection plane (virtual image display region VS described later) with reference to the virtual viewpoint VP, and sets the two-dimensional image as an image M. The rendering module 510 may also be configured to place each content FU placed in the "model coordinate system (local coordinate system)" in the space of the "world coordinate system". That is, the vertex data of each content FU of the drawing object calculated on the "model coordinate system" may be arranged on the "world coordinate system". Further, some or all of the content FU may not be arranged on the "world coordinate system".

The observer 700 visually observes the virtual image V20 formed on the virtual image display region VS via the projection target portion 2, and perceives the presence of the content FU at the predetermined target position MP. For example, when the content FU is an arrow of the navigation progress path, the arrow of the virtual image V20 is displayed in the virtual image display area VS so that the content FU is visually observed as to be arranged at the predetermined target position MP of the live view when viewed from the virtual viewpoint VP. That is, when an image (here, the virtual image V20) obtained by converting the projection of the content FU into the virtual image display region VS is displayed with the virtual viewpoint VP as a reference, if the observer 700 observes from the same position (for example, the center 205 of the eye movement range 200) as the virtual viewpoint VP, the content FU disposed at the predetermined target position MP can be perceived as if observed from the virtual viewpoint VP as shown in fig. 5.

In general, the virtual plane 100 on which the (set) target position MP where the content FU is arranged is made to coincide with the height of the surface of the foreground (road surface 6) (i.e., the set height is set to 0 m). However, this is an example and is not limiting. In other examples, the target position MP (virtual plane 100) may be set at a position higher than the height of the surface of the foreground (road surface 6) (i.e., the set height may be set to 0.5m or 1 m). In other examples, the target position MP (virtual plane 100) may be set at a position lower than the height of the surface of the foreground (road surface 6) (that is, the set height may be set to-1 m or-2 m). The depression angle β is an angle (a depression angle) between the horizontal direction (Z1-X1 plane) and the content FU (target position MP) when viewed from the predetermined virtual viewpoint VP.

The image adjustment module 520 (position adjustment module 522, depression angle adjustment module 524, size adjustment module 526) of fig. 3 executes a process of adjusting the position of the virtual image V20 displayed in the virtual image display area VS (position adjustment process), a process of adjusting the depression angle (depression angle adjustment process), and a process of adjusting the size (size adjustment process) in conjunction with the posture variation of the vehicle 1.

Fig. 6 is a diagram showing a virtual image before the position adjustment process, the left diagram showing a relationship between the virtual image and the virtual plane, and the right diagram showing a foreground and the virtual image visually observed when the observer faces forward. In fig. 6, the vehicle posture AT10 is set to AT11. The content FU is configured such that a visual observer can observe that the content FU is superimposed on the target position MP (first region 110) of the virtual plane 100. The content FU has a predetermined size and is disposed so as to overlap the first region 110 of the virtual plane 100. In fig. 6, the size of the content FU is set to a first length L10 in the depth direction of the first region 110. The arbitrary virtual plane 100 is a virtual plane on which the content FU is disposed, and is set to be parallel to the front-rear-left-right direction of the vehicle 1 (may be substantially coincident with the road surface 6), for example. The rendering module 510 causes the display 50 to display the original image M as the virtual image V21 in order to display the image (here, the virtual image V21) in which the content FU is projection-converted into the virtual image display region VS with the virtual viewpoint VP1 as a reference. Here, the distance from the virtual viewpoint VP1 to the first region 110 along the virtual plane 100 is set to D0, and the distance (height) from the virtual plane 100 to the virtual viewpoint VP1 is set to h0.

Fig. 7A is a diagram showing a virtual image after the first position adjustment processing, the left diagram showing the relationship between the virtual image and the virtual plane, and the right diagram showing the foreground and the virtual image visually observed when the observer faces forward. In fig. 7A, the vehicle posture AT10 is set to be AT12 of a pitch angle α12 that is smaller than the vehicle posture AT11 before the first position adjustment process. Here, the forward tilting refers to a posture in which the front of the vehicle 1 is lowered (in other words, the rear of the vehicle 1 is raised) with reference to the vehicle posture AT11 shown in fig. 6. When the vehicle 1 is tilted forward from the state of fig. 6 to the state of fig. 7A, the virtual image display region VS observed from the viewpoint of the observer is relatively moved downward (Y-axis negative direction) B12 with respect to the real scene (road surface) 6 due to the posture variation. When the first position adjustment process is not performed, the virtual image V21 shown in fig. 6 is moved by the downward image movement amount B12 to the position G2 in the right view of fig. 7A as the virtual image display region VS moves by the downward image movement amount B12 due to the posture change. That is, the virtual image V21 of the content FU observed from the viewpoint of the observer is shifted from the target position MP1 desired to be arranged. The control circuit 33 in several embodiments corrects the position G2 of the virtual image by the first position adjustment amount C12 (C10) (displays the virtual image V22 after the position adjustment) upward (Y-axis positive direction) so as to suppress (cancel) the image movement amount B12 due to the posture variation by executing the first position adjustment processing. Preferably, the virtual image V20 after the posture change can be maintained in the first region 110 (target position MP 1) by making the first position adjustment amount C12 (C10) equal to the image movement amount B12 (B10) due to the posture change (c10=b10). Thus, the image shift amount B10 due to the posture change is canceled by the first position adjustment amount C10, and is not recognized by the observer. However, the first position adjustment amount C10 may be smaller than the image shift amount B10 as long as the image shift amount B10 due to the posture change can be reduced. This suppresses the positional displacement of the virtual image due to the change in the vehicle posture.

Fig. 7B is a diagram showing that the processed virtual image is suppressed from being completely seen, the left diagram shows the relationship between the virtual image and the virtual plane, and the right diagram shows the foreground and the virtual image visually observed when the observer faces forward. In fig. 7B, the vehicle posture AT10 is set to be AT13 of a pitch angle α13 (> α12) that is relatively large as compared with the vehicle posture AT11 when the first position adjustment process is executed. When the vehicle 1 is tilted forward from the state of fig. 6 to the state of fig. 7B, the virtual image display region VS observed from the viewpoint of the observer is relatively moved downward (Y-axis negative direction) B13 with respect to the real scene (road surface) 6 due to the posture variation. When the position adjustment is not performed, the virtual image V21 shown in fig. 6 moves by the image movement amount B13 to the position G3 in the right view of fig. 7B in accordance with the image movement amount B13 of the virtual image display region VS due to the posture change. In order to cancel the downward image shift amount B13 due to the posture change, the virtual image V20 may be moved upward by the image shift amount B13, but if the virtual image V20 is moved upward by the image shift amount B13, the virtual image is moved out of the virtual image display region VS 13.

In the case where the posture of the vehicle 1 is changed greatly (an example of a predetermined condition described later), the control circuit 33 in several embodiments performs the suppressing process to prevent the vehicle from being completely seen, and corrects the limit CT (the virtual image V23 after the display position adjustment) of the position adjustment amount smaller than the first position adjustment amount C13 (C10) in the first position adjustment process for the upward (Y-axis positive direction) position G3 of the virtual image with respect to the downward image movement amount B13 of the virtual image display area VS due to the posture change. By making the limit CT of the position adjustment amount smaller than the image movement amount B13 (B10) due to the posture variation (CT < B10), the virtual image V23 can be maintained in the virtual image display region VS13 even though the second region 120 (the position different from the target position MP 1) on the side near the first region 110 (the target position MP 1) is offset to a greater extent than the first position adjustment process. Thus, the image shift amount B13 (B10) due to the posture change is recognized by the observer because it is not canceled by the limit CT of the position adjustment amount.

The content FU is disposed so as to have a predetermined angular relationship with respect to the road surface 6. Specifically, for example, the content FU is configured to be visually observable in parallel with the road surface 6. However, when the pitch angle α of the vehicle 1 is changed, the angular relationship between the content FU and the road surface 6 is changed. Specifically, in the case where the virtual image V20 is displayed in parallel with the road surface 6, the parallel relationship of the virtual image V20 and the road surface 6 is shifted by the pitch angle α due to the pitch angle α. The deviation of the angular relationship between the virtual image V20 and the road surface 6 according to the posture variation (pitch) of the vehicle 1 can be corrected by adjusting the angle β (the depression angle) of the virtual image V20) about the lateral direction of the virtual image V20.

The control circuit 33 in several embodiments can adjust the depression angle β of the virtual image V20 (content FU) for the posture variation of the vehicle 1 in the process of suppressing the inability to see completely. The control circuit 33 dynamically increases the depression angle β in response to an increase in the pitch angle α in the forward tilting direction. In contrast, the control circuit 33 dynamically decreases the depression angle β in response to an increase in the pitch angle α in the backward tilting direction.

The control circuit 33 in several embodiments may adjust the depression angle β (first depression angle adjustment process) of the virtual image V20 (content FU) with respect to the posture change of the vehicle 1 when executing the first position adjustment process. As in the case of performing the suppression processing that cannot be seen completely, the control circuit 33 dynamically increases the depression angle β in response to an increase in the pitch angle α in the forward direction. In contrast, the control circuit 33 dynamically decreases the depression angle β in response to an increase in the pitch angle α in the backward tilting direction.

The control circuit 33 in several embodiments can adjust the pitch angle β by the same angle as the change amount α of the pitch angle when executing the first position adjustment process (an example of the first pitch angle adjustment process). Specifically, when the vehicle 1 is tilted forward from the state of fig. 6 to the state of fig. 7A, the change amount α of the pitch angle is α12. The virtual viewpoint VP1 shown in fig. 6 changes the angle of the virtual viewpoint VP2 with respect to the virtual plane 100 by α12 according to the change amount α12 of the pitch angle of the forward tilt shown in fig. 7A. Accordingly, the depression angle β12 of the content FU disposed at the target position MP1 with respect to the virtual viewpoint VP2 in plan view is larger than the depression angle β11 by α12. The control circuit 33 in several embodiments may multiply the variation α of the pitch angle β by a predetermined coefficient to obtain an angle (an example of the first pitch angle adjustment process).

In addition, the control circuit 33 in several embodiments can adjust the pitch angle β to the same angle as the change amount α of the pitch angle when the suppression cannot be completely seen (an example of the first pitch angle adjustment process). Specifically, when the vehicle 1 is tilted forward from the state of fig. 6 to the state of fig. 7B, the change amount α of the pitch angle is α13. The virtual viewpoint VP1 shown in fig. 6 moves to the virtual viewpoint VP3 position according to the amount of change α13 of the pitch angle of the forward tilt shown in fig. 7B, and the angle of the virtual viewpoint VP3 with respect to the virtual plane 100 changes α13. Correspondingly, the control circuit 33 may increase the depression angle β of the content FU expressed by the virtual image V23 by the amount of change α13 in the pitch angle of the forward tilt. The control circuit 33 in several embodiments may multiply the variation α of the pitch angle β by a predetermined coefficient to obtain an angle (an example of the first pitch angle adjustment process).

Preferably, the control circuit 33 in several embodiments may make the adjustment amount (depression angle adjustment amount E20) of the depression angle β with respect to the change amount α of the pitch angle in the depression angle adjustment process (second depression angle adjustment process) performed together with the suppression of the inability to completely view the process larger than the adjustment amount (depression angle adjustment amount E10) of the depression angle β with respect to the change amount α of the pitch angle in the depression angle adjustment process (first depression angle adjustment process) performed together with the first position adjustment process. It is possible to visually observe that the virtual image V23 after the processing is not completely seen from the target position MP1 shifted from the virtual image V22 after the first position adjustment processing. Specifically, when the observer leans forward, the virtual image V23 after the processing is visually observed to be prevented from being completely seen is further moved to a position 120 overlapping with the real scene (road surface 6) on the front side of the observer with reference to the target position MP1 (110) of the virtual image V22 after the first position adjustment processing. When the content FU is arranged near the observer along the virtual plane 100 parallel to the road surface 6, the depression angle β of the content FU with respect to the virtual viewpoint VP in plan view increases. Conversely, when the content FU is disposed away from the observer, the depression angle β of the content FU with respect to the virtual viewpoint VP in plan view becomes smaller. Therefore, the control circuit 33 in more preferred embodiments can set the depression angle adjustment amount (depression angle adjustment amount) by which the depression angle β of the content FU in the depression angle adjustment process (second depression angle adjustment process) performed together with the inhibition process is adjusted to be less than the full view, the depression angle adjustment amount by which the movement of the content FU in the far and near direction of the virtual plane 100 (which is set to be substantially coincident with the road surface 6 as an example) is added to the depression angle adjustment amount by which the pitch angle α of the vehicle 1 is changed. The depression angle β13 in fig. 7B is calculated as a value obtained by correcting the depression angle β11 in fig. 6, which is a reference for the change in posture, by a depression angle adjustment amount obtained by adding the depression angle adjustment amount by which the content FU moves from the first region 110 on the virtual plane 100 to the second region 120 on the near side to the first region, and the depression angle adjustment amount by the change in the pitch angle α of the vehicle 1.

Fig. 8 is a diagram for explaining the position adjustment range. The virtual images in several embodiments set the position adjustment range VT for each virtual image. As shown in fig. 8, if the lower end of the position adjustment range VT is set closer to the reference position PO of the virtual image V1, the limit CTd of the downward position adjustment amount of the virtual image V1 is set shorter. On the other hand, if the upper end of the position adjustment range VT is set farther than the reference position PO of the virtual image V1, the limit CTu of the upward position adjustment amount of the virtual image V1 is set longer. The display control device 30 (image adjustment module 520) may set the position adjustment range VT for each of the plurality of virtual images V1 displayed in the virtual image display region VS, or may set the position adjustment range VT for all the virtual images V1 in common. The position adjustment range VT may be the virtual image display region VS (the entire virtual image display region VS may be set as the position adjustment range VT). The image adjustment module 520 includes table data (not shown) that correlates the posture change (pitch angle α) of the vehicle 1 with the position adjustment amount C of the virtual image V1, and can set, based on the table data, a limit (posture threshold value) αtu of the pitch angle of the recline that is assumed when the position adjustment amount C of the virtual image V1 reaches the limit CTd of the downward position adjustment amount and a limit (posture threshold value) αtd of the pitch angle of the forward tilt that is assumed when the position adjustment amount C of the virtual image V1 reaches the limit CTu of the upward position adjustment amount.

The display control device 30 (control circuit 33) in the present embodiment displays at least a first virtual image displayed in a first region within the virtual image display region and a second virtual image displayed in a second region closer to an end portion of the virtual image display region in the up-down direction than the first region, performs a first image adjustment process for adjusting the position of the first virtual image in accordance with a first position adjustment amount, and performs a second image adjustment process for adjusting the position of the second virtual image in accordance with a second position adjustment amount for suppressing adjustment of the position of the virtual image with respect to change in the posture of the vehicle in comparison with the first image adjustment process.

Fig. 9 is a diagram showing an example of image adjustment processing of a first virtual image and a second virtual image displayed in a virtual image display region in several embodiments. The virtual image display region VS is classified as: a second region VS22 including an upper end; a first region VS10 disposed below the second region VS22; and VS21 including a lower end and disposed below the first region VS 10. The virtual image having the reference position PO in the first region VS10 is referred to as a first virtual image V10, and the virtual image having the reference position PO in the second region VS20 (the second region VS22 in fig. 9) is referred to as a second virtual image V20. The position adjustment range VT may be provided in plurality in the virtual image display area VS for each content. In fig. 9, a position adjustment range VT1 is set for the first virtual image V10, and a position adjustment range VT2 is set for the second virtual image V20. When the first virtual image V10 or the second virtual image V20 moves downward within the position adjustment range VT, the first virtual image V10 or the second virtual image V20 moves dynamically with the posture change before reaching the limit CTd of the lower position adjustment amount, but the position adjusted by the limit CTd of the lower position adjustment amount is fixed with respect to a further large posture change. In contrast, when the first virtual image V10 or the second virtual image V20 moves upward within the position adjustment range VT, the first virtual image V10 or the second virtual image V20 moves dynamically with the posture change before reaching the limit CTu of the upper position adjustment amount, but is fixed at the position adjusted by the limit CTu of the upper position adjustment amount with respect to a further large posture change.

Fig. 10 is a diagram showing a flow of display control in several embodiments of fig. 10. The image adjustment process S1 shown in fig. 10 is implemented by the control circuit 33 executing the image adjustment module 520 stored in the memory 37. In step S110, the image adjustment module 520 (position adjustment module 522) acquires the drawing data generated by the drawing module 510. In step S120, the position adjustment module 522 sets a limit CT of the amount of position adjustment of the virtual image V1 (the original image M as the virtual image V1) based on the information indicating the display position of the image M (the information indicating the reference position PO) before the position adjustment processing included in the acquired drawing data is performed. Specifically, for example, the position adjustment module 522 sets the position adjustment range VT, and sets the limit CT of the position adjustment amount C based on the position adjustment range VT and the reference position PO.

Next, in step S130, the position adjustment module 522 acquires information indicating the posture change of the vehicle 1 (posture change information) from the posture detection unit 415. The posture detecting unit 415 includes one or more sensors such as a gyro sensor, an acceleration sensor, and a height sensor, for example. The posture detecting unit 415 may calculate a vehicle posture such as a pitch angle or a roll angle, or a frequency of change in the vehicle posture, as posture change information, from sensor values such as an angular velocity, an acceleration, and a height of the mobile body, and output the calculated vehicle posture or frequency to the display control device 30. The posture change information may include, in addition to the vehicle posture (pitch angle, roll angle, etc.), the frequency of change in the vehicle posture (vibration frequency), and the like. Part or all of the function of calculating the posture change information by the posture detection unit 415 may be provided in the display control device 30.

In step S140, the position adjustment module 522 calculates a first position adjustment amount C10 used in a first image adjustment process S151 described later. First, the position adjustment module 522 calculates the posture change amount (the amount of angular displacement) of the vehicle 1 based on the posture change information acquired from the posture detection unit 415. For example, the position adjustment module 522 calculates an angle (pitch angle) α of the vehicle 1 about the pitch axis by integrating the angular velocity detected by the attitude detection unit 415. This allows calculation of the amount of displacement (angle) of the vehicle 1 in the rotational direction about the Y axis (pitch axis) shown in fig. 1. In the present embodiment, the pitch angle is calculated, but the yaw angle or the roll angle may be calculated. For example, all angles about the X, Y and Z axes may be calculated. However, some or all of the functions of the position adjustment module 522 for calculating the attitude change amount (the angular offset amount) may be provided with a device different from the display control device 30 that can communicate with the display control device 30, and the display control device 30 may input information indicating the attitude change amount (the angular offset amount) of the vehicle 1 from the different device via the I/O interface 31. That is, several display control apparatuses 30 may omit the function of the position adjustment module 522 to calculate the posture variation amount (the offset amount of the angle).

In step S140, the position adjustment module 522 further calculates a first position adjustment amount C10 for correcting the display position of the virtual image V20 from the posture variation amount (the offset amount of the angle) of the vehicle 1. Specifically, the position adjustment module 522 converts the offset amount of (pitch angle) into a pixel value, and determines an adjustment amount by which the pixel value of the offset amount (the image movement amount B10 due to the posture change) is restored. Preferably, the position adjustment module 522 calculates the position adjustment amount (first position adjustment amount C10) in the opposite direction equal to the image movement amount B10 due to the posture change in order to restore the position shift of the virtual image V20 due to the posture change of the vehicle 1.

In step S150 of several embodiments, the image adjustment module 520 generates image data by performing image adjustment on the drawing data acquired in step S110. In S150, the image adjustment module 520 rearranges the pixels of the drawing data into pixels of the image data according to the position adjustment amount C, the depression angle adjustment amount E, and the size adjustment amount F. In S160, the image adjustment module 520 outputs the (adjusted) image data generated in S150 to the display 50.

In step S150 of several embodiments, the control circuit 33 (position adjustment module 522) executes a first image adjustment process (S151 described later) for adjusting the position of the first virtual image V10 in accordance with the first position adjustment amount C10, and executes a second image adjustment process (S152 described later) for adjusting the position of the second virtual image V20 to suppress a change in posture of the virtual image with respect to the vehicle 1, as compared with the first image adjustment process (S151 described later).

In the first image adjustment processing S151, the position adjustment module 522 performs position adjustment of the first virtual image V10 (the original image M as the first virtual image V10) based on the first position adjustment amount C10 (the position adjustment amount C) dynamically set in step S140 in combination with the posture change acquired in step S130. Preferably, the first position adjustment amount C10 is set so as to cancel out the positional shift of the image due to the posture change.

In the first image adjustment processing S151 according to several embodiments, the image adjustment module 520 (the depression angle adjustment module 524) may dynamically change the first depression angle adjustment amount E10 (the depression angle adjustment amount E) in combination with the posture change acquired in step S130, and perform the depression angle adjustment of the first virtual image V10 (the original image M as the first virtual image V10) based on the first depression angle adjustment amount E10 (the depression angle adjustment amount E), in addition to the above-described position adjustment.

In the first image adjustment processing S151 according to several embodiments, when the first position adjustment amount C10 is not set so as to cancel the positional shift of the image due to the posture change (in other words, when the positional shift of the image due to the posture change occurs), the image adjustment module 520 (the size adjustment module 526) may dynamically change the size adjustment amount F in addition to the above-described position adjustment in accordance with the posture change acquired in step S130, and perform the size adjustment of the first virtual image V10 (the original image M as the first virtual image V10) based on the size adjustment amount F.

In the second image adjustment processing S152, the image adjustment module 520 (depression angle adjustment module 524) dynamically changes the second depression angle adjustment amount E20 (depression angle adjustment amount E) in accordance with the posture change acquired in step S130, and performs depression angle adjustment of the virtual image V20 (the original image M as the virtual image V20) based on the second depression angle adjustment amount E20 (depression angle adjustment amount E). The depression angle adjustment amount E20 in the second image adjustment processing S152 is a depression angle adjustment amount that corrects a depression angle shift of the real (road surface 6) and virtual images due to a posture variation (a change in the pitch angle α). Preferably, the depression angle adjustment amount E20 in the second image adjustment processing S152 may be added to the depression angle adjustment amount for correcting the depression angle shift of the real (road surface 6) and the virtual image due to the posture variation (change in the pitch angle α) to express the movement to the vicinity (or express the movement to the distant direction) due to the posture variation (change in the pitch angle α).

In the second image adjustment processing S152 according to several embodiments, the image adjustment module 520 (position adjustment module 522) may dynamically change the second position adjustment amount C20 (position adjustment amount C) in conjunction with the posture change acquired in step S130 (position adjustment processing in the second image adjustment processing S152). The image adjustment module 520 may include table data, an arithmetic expression, and the like for setting the second position adjustment amount C20 (position adjustment amount C) based on the posture change acquired in step S130.

In the position adjustment process of the second image adjustment process S152, the image adjustment module 520 calculates a dynamically changed second position adjustment amount C20 from the posture change amount of the vehicle 1. For example, the second position adjustment amount C20 is obtained by multiplying the first position adjustment amount C10 determined from the posture variation amount of the vehicle 1 by a coefficient smaller than 1. In addition, the position adjustment module 522 of several embodiments may read the second position adjustment amount C20 stored in the memory 37 without depending on the posture variation amount of the vehicle 1. In the present embodiment, the adjustment amount in the pitch axis direction is calculated, but the adjustment amounts in the yaw axis direction and the roll direction may be calculated. The roll angle is set to a predetermined adjustment amount that restores the offset amount of the roll angle while maintaining the angle. However, some or all of the functions of the image adjustment module 520 for calculating the second position adjustment amount C20 may be provided with a device different from the display control device 30 that can communicate with the display control device 30, and the display control device 30 may input the display parameter (the second position adjustment amount C20) for adjusting the position of the virtual image from the different device via the I/O interface 31.

In the second image adjustment processing S152 according to several embodiments, the image adjustment module 520 may dynamically change the resizing amount F in combination with the posture change acquired in step S130, and perform the resizing processing of the virtual image V20 (the original image M as the virtual image V20) based on the resizing amount F, in addition to the position adjustment. The image adjustment module 520 may include table data, an arithmetic expression, and the like for setting the resizing amount F according to the posture change acquired in step S130.

Fig. 11 is a diagram illustrating an image adjustment process with respect to a posture change, and shows an example in which the first image adjustment process or the second image adjustment process is executed according to the content. The first virtual image V10 is adjusted (first image adjustment processing is performed) by a first position adjustment amount C10 and a first depression angle adjustment amount E10 that dynamically change in accordance with the posture change of the vehicle 1. The first position adjustment amount C10 is a position adjustment amount C that dynamically corrects the position of the virtual image in the virtual image display region VS in the up-down direction in combination with the posture variation (forward tilting of the pitch angle α) so as to significantly suppress (preferably cancel) the displacement of the virtual image in the up-down direction due to the posture variation. Thereby, the positional deviation of the virtual image is preferably canceled (the position of the virtual image is maintained at the target position MP 1). The first depression angle adjustment amount E10 is a depression angle adjustment amount E that dynamically increases the depression angle β of the virtual image in combination with the posture variation (forward tilting of the pitch angle α) so as to suppress (preferably cancel) the depression angle offset between the virtual image and the real image (road surface 6) caused by the posture variation (change of the pitch angle α) with respect to the predetermined reference posture. If the positional deviation of the virtual image is canceled by the first position adjustment amount C10, the virtual image is not deviated from the target position MP1, and no adjustment of the size is required, so the first size adjustment amount F10 is zero.

The second virtual image V20 is adjusted (second image adjustment processing is performed) by the second position adjustment amount C20 and the second depression angle adjustment amount E20 that dynamically change in accordance with the posture change of the vehicle 1. The second position adjustment amount C20 is smaller than the first position adjustment amount C10 that significantly suppresses (preferably cancels) the displacement of the virtual image in the up-down direction due to the posture variation, and is a position adjustment amount C that dynamically corrects the position of the virtual image in the virtual image display region VS in the up-down direction in combination with the posture variation (forward tilting of the pitch angle α). As a result, a downward/upward shift of the virtual image due to the posture change (forward/backward tilt of the pitch angle α) occurs (in other words, the virtual image moves toward the vicinity of the observer due to forward tilt or moves toward the far side of the observer due to backward tilt). The depression angle adjustment amount E is set to a second depression angle adjustment amount E20, and the depression angle adjustment amount for correcting the depression angle shift of the real scene (road surface 6) and the virtual image due to the posture variation (change in the pitch angle α) is added to the depression angle adjustment amount for expressing the movement of the virtual image to the vicinity of the observer, and the depression angle β of the virtual image is dynamically increased in conjunction with the posture variation (forward tilt of the pitch angle α). Here, the second depression angle adjustment amount E20 is larger than the first depression angle adjustment amount E10 because the depression angle adjustment amount that expresses the movement of the virtual image to the vicinity of the observer is added thereto. The second resizing amount F20 is set so that the size-combined posture variation (variation in the pitch angle α) of the virtual image becomes dynamically large in order to express the movement of the virtual image to the vicinity of the observer.

Fig. 12 is a diagram showing an example of image adjustment processing of a first virtual image and a second virtual image displayed in a virtual image display region in several embodiments. In time t11 to t12, pitch angle α is a forward tilt smaller than preset attitude threshold αt2 with respect to predetermined reference attitude α0, and in time t12 to t13, pitch angle α is a backward tilt smaller than preset attitude threshold αtu2. The position adjustment amount C is set to dynamically correct the position adjustment amount C of the virtual image in the virtual image display region VS upward (downward in t12 to t 13) in combination with the posture change (forward tilt of the pitch angle α (backward tilt in t12 to t 13)) so as to significantly suppress (preferably cancel) the displacement of the virtual image downward (upward in t12 to t 13) due to the posture change (the first virtual image V10 is the first position adjustment amount C10, and the second virtual image V20 is the second position adjustment amount C20). Thereby, the positional deviation of the virtual image is preferably canceled (the position of the virtual image is maintained at the target position MP 1). The depression angle adjustment amount E is set to a depression angle adjustment amount E (the first virtual image V10 is the first depression angle adjustment amount E10, and the second virtual image V20 is the second depression angle adjustment amount E20) that dynamically increases (decreases) the depression angle β of the virtual image (the forward tilt of the pitch angle α (the backward tilt in t12 to t 13)) in combination with the posture change (the forward tilt of the pitch angle α) so as to suppress (preferably cancel) the depression angle shift of the virtual image from the real image (the road surface 6) due to the posture change (the change of the pitch angle α). If the positional deviation of the virtual image is canceled by the first position adjustment amount C10, the virtual image is not deviated from the target position MP1, and no adjustment of the size is required, so the first size adjustment amount F10 is zero. The second size adjustment amount F20 is set so that the size-coupled posture variation (change in the pitch angle α) of the virtual image becomes dynamically large in order to express the movement of the virtual image to the vicinity of the observer in t11 to t12, and is set so that the size-coupled posture variation (change in the pitch angle α) of the virtual image becomes dynamically small in order to express the movement of the virtual image to the distant place of the observer in t12 to t 13. The reference attitude α0 is the pitch angle α of the vehicle stored in advance in the memory 37, and typically, the pitch angle of the vehicle is zero (parallel to the road surface 6). Further, the reference attitude α0 may vary. Specifically, when the pitch angle α of the vehicle does not change much for a predetermined time or longer, the pitch angle α stabilized at this time may be set (updated) to the reference attitude α0 and stored in the memory 37.

In time t13 to t14, pitch angle α is a backward tilt greater than attitude threshold αtu2 set in advance. The position adjustment amount C is set to a third position adjustment amount C21 (limit CT of the position adjustment amount) that fixes the position of the virtual image in the virtual image display region VS irrespective of the posture variation (backward tilt of the pitch angle α). As a result, the upward displacement of the virtual image due to the posture change (backward tilting of the pitch angle α) becomes further large (in fig. 12, if the displacement amount by the second position adjustment amount C20 is made to be a broken line, the third position adjustment amount C21 is a thick line). The depression angle adjustment amount E is set to a third depression angle adjustment amount E21, and the depression angle adjustment amount for correcting the depression angle shift of the real scene (road surface 6) and the virtual image due to the posture variation (change in the pitch angle α) is added to the depression angle adjustment amount for expressing the movement of the virtual image to the far side of the observer, and the depression angle β of the virtual image is dynamically increased in combination with the posture variation (backward tilt of the pitch angle α). Here, the third depression angle adjustment amount E21 is larger than the second depression angle adjustment amount E20 because a depression angle adjustment amount that expresses movement of the virtual image to the observer's distance is added thereto. The third size adjustment amount F21 sets the size adjustment amount so that the size of the virtual image is dynamically reduced in association with the posture change (change in the pitch angle α) in order to express the movement of the virtual image to the observer's distance. The image processing at times t14 to t17 is the same as the image processing at times t11 to t13, and the description thereof is omitted.

In several embodiments, the control circuit 33 (the image adjustment module 520) may perform 1) a first image adjustment process of adjusting the position of the first virtual image V10 according to the first position adjustment amount C10 in the case where the prediction of the posture variation of the vehicle 1 is the first prediction state; 2) A second image adjustment process of suppressing adjustment of the position of the virtual image with respect to the posture change of the vehicle 1, which is smaller than the first image adjustment process, is performed on the position of the second virtual image V20, and 3) a first image adjustment process of adjusting the position of the first virtual image V10 and the position of the second virtual image V20 according to the first position adjustment amount C10 is performed in the case where the prediction of the posture change of the vehicle 1 is a second prediction state smaller than the first prediction state. At this time, the position adjustment range VT2 set for the second virtual image V20 does not change (but is not limited thereto). In several embodiments, the posture detection unit 415 may include various software components for performing various operations related to predicting the posture change of the vehicle from information (acceleration or acceleration) capable of predicting the posture change of the vehicle, and may output information indicating a result of predicting the posture change of the vehicle (an example of prediction related information) or a predicted value of the posture change of the vehicle (an example of prediction related information) to the control circuit 33. Specifically, for example, the posture detection unit 415 may input observation information (acceleration or acceleration) capable of predicting the posture change of the vehicle, and predict the next value using one or more conventional observation information by using a prediction algorithm such as a least square method, a kalman filter, an α - β filter, or a particle filter. A part or all of the functions of predicting the posture change of the vehicle may include the control circuit 33 (a prediction module (not shown) as a prediction algorithm may be provided in the memory 37).

In several embodiments, the control circuit 33 (the image adjustment module 520) may execute the first image adjustment process for adjusting the second virtual image based on the first position adjustment amount when the posture variation of the vehicle is less than the predetermined threshold value, and execute the second image adjustment process for adjusting the second virtual image based on the second position adjustment amount for suppressing the adjustment of the position of the virtual image relative to the posture variation of the vehicle 1 when the posture variation of the vehicle is greater than the predetermined threshold value.

Fig. 13 is a diagram showing an example of image adjustment processing of a first virtual image and a second virtual image displayed in a virtual image display region in several embodiments. The control circuit 33 (image adjustment module 520) executes a first image adjustment process for adjusting the second virtual image based on the first position adjustment amount C10 when the pitch angle α (an example of posture fluctuation) of the vehicle 1 is smaller than a predetermined threshold αth1 (times t21 to t22, t23 to t25, and t26 to t 27), and executes a second image adjustment process for adjusting the second virtual image based on the second position adjustment amount C22 (C20) when the pitch angle α (an example of posture fluctuation) of the vehicle 1 is larger than the predetermined threshold αth1.

In several embodiments, the control circuit 33 (the image adjustment module 520) may execute the second image adjustment process for adjusting the second position adjustment amount with respect to the second virtual image, calculate the peak value of the posture change of the vehicle, and switch from the second image adjustment process to the first image adjustment process when the peak value of the posture change of the vehicle is less than the predetermined threshold value and the posture change of the vehicle is in the opposite phase with respect to the reference posture as the base point.

Fig. 14 is a diagram showing an example of image adjustment processing of a first virtual image and a second virtual image displayed in a virtual image display region in several embodiments. At the time t31 to t32, the control circuit 33 (image adjustment module 520) executes second image adjustment processing for adjusting the second position adjustment amount C23 (C20) to suppress the adjustment of the position of the virtual image with respect to the posture change of the vehicle 1 with respect to the first position adjustment amount C10, with respect to the second virtual image V20, calculates the peak value of the vehicle posture change at the time of forward tilting, and determines whether or not the peak value is less than a predetermined threshold value αth2. Since the peak value of the vehicle posture fluctuation is larger than the predetermined threshold value αth2 in the time t31 to t32, the second virtual image V20 performs the second image adjustment process (the same applies to t33 to t 34) that is adjusted according to the second position adjustment amount C23 (C20) even in the opposite phase (backward tilt) of the time t32 to t 33. Since the peak value of the vehicle posture fluctuation is smaller than the predetermined threshold value αth2 in the time t33 to t34, the second virtual image V20 is switched from the second image adjustment processing to the first image adjustment processing in the opposite phase (backward tilt) in the time t34 to t 35.

In the display control methods according to the embodiments, when it is determined that the posture of the vehicle is changed greatly or the posture of the vehicle is changed frequently, the number of contents of the second image adjustment process is increased. For example, when two contents of the first image adjustment process are displayed and two contents of the second image adjustment process are displayed, if it is determined that the posture of the vehicle is large or the posture of the vehicle is frequent, the contents of the first image adjustment process may be one and the contents of the second image adjustment process may be three.

In the display control methods according to the embodiments, when it is determined that the vehicle posture is changed greatly or the vehicle posture is changed frequently, the second region of the second virtual image, which is regarded as the second image adjustment processing, is enlarged toward the center in the vertical direction of the virtual image display region. For example, as shown in fig. 9, when the second region VS21 is disposed below the virtual image display region VS, the second region VS21 can be enlarged upward when it is determined that the posture of the vehicle is large or the posture of the vehicle is frequent. Thereby, the content belonging to the second area VS21 is increased. Accordingly, the second image adjustment processing can be sequentially applied from the end portion near the virtual image display region which is likely to be incompletely seen.

In the display control method of several embodiments, further comprising: when the second image position adjustment processing is executed, if the second virtual image V20 changes from a state of being disposed in the second region VS20 of the virtual image display region VS to a state of being disposed outside the second region, a third position adjustment processing (including a mode in which the adjustment of the virtual image is dynamically performed with respect to the change in the posture of the vehicle) is executed for the first virtual image V10 and the second virtual image V20 to suppress the adjustment of the position of the virtual image with respect to the change in the posture of the vehicle as compared with the second image position adjustment processing. Thus, it is also considered that in the case where the second image position adjustment processing causes incomplete viewing, the adjustment amount of the position of the virtual image with respect to the posture variation of the vehicle is further suppressed, thereby more significantly preventing the incomplete viewing of the virtual image.

In addition, the display control method according to several embodiments further includes: when the second image position adjustment processing is executed, in a case where the second virtual image V20 changes from a state of being disposed in the second region VS20 of the virtual image display region VS to a state of being disposed outside the second region VS20, third position adjustment processing is executed for fixing the positions of the virtual images with respect to the posture change of the vehicle for the first virtual image V10 and the second virtual image V20. Thus, it is also considered that in the case where the second image position adjustment processing causes incomplete visibility, the position of the virtual image with respect to the posture variation of the vehicle is fixed, thereby completely preventing the incomplete visibility of the virtual image.

In the display control method of several embodiments, further comprising: the second virtual image V20 is arranged so as to be visually observed below the first virtual image V10; when the vehicle 1 is tilted forward and the first virtual image V10 that is offset when the first image position adjustment process is performed changes from a state of being disposed in the first position adjustment range VT1 to a state of being disposed on the upper side of the first position adjustment range VT1, the third image position adjustment process is performed in which the position of the first virtual image V10 is fixed to the upper peripheral edge portion in the first position adjustment range VT1 and the position of the second virtual image V20 is fixed to a predetermined position below the upper peripheral edge portion in the first position adjustment range VT 1. Thus, it is also considered that the advantage is to prevent one virtual image from being fixed and the other virtual image from continuing the position correction.

In the display control method of several embodiments, further comprising: the second virtual image V20 is arranged so as to be visually observed below the first virtual image V10; when the vehicle 1 is tilted backward and the second virtual image V20 that is offset when the second image position adjustment process is performed changes from a state of being disposed in the second position adjustment range VT2 to a state of being disposed at the lower side of the second position adjustment range VT2, the third image position adjustment process is performed in which the position of the second virtual image V20 is fixed to the lower peripheral edge portion in the second position adjustment range VT2 and the position of the first virtual image V10 is fixed to a predetermined position above the lower peripheral edge portion in the second position adjustment range VT 2. Thus, it is also considered that the advantage is to prevent one virtual image from being fixed and the other virtual image from continuing the position correction.

In the display control method of several embodiments, further comprising: the second position adjustment amount C20 is changed according to the position of the second virtual image V20 in the vertical direction within the virtual image display region VS, and the second position adjustment amount C20 is reduced as the position of the second virtual image V20 is closer to the end of the virtual image display region VS in the vertical direction. Thus, it is also considered that there is an advantage that the observer does not easily feel uncomfortable due to the difference in the amount of correction of the position between the plurality of virtual images in the virtual image display region.

The display control device 30 according to the present embodiment controls the head-up display device 20, the head-up display device 20 being mounted on the vehicle 1 and having a display 50 for displaying an image on a display surface 50a, and visually observing a virtual image by projecting the light of the image onto the projection target portion 2 so that the virtual image overlaps in a virtual image display region VS overlapping a road surface in front of the vehicle 1, the device including: one or more control circuits 33; a memory 37; and one or more computer programs stored in the memory 37 and executed by the one or more control circuits 33, wherein in the control circuits 33, at least a first virtual image V10 displayed in a first region VS10 within the virtual image display region VS and a second virtual image V20 displayed in a second region VS20 closer to an end portion of the virtual image display region VS in the up-down direction than the second region VS20 are displayed; acquiring posture change amount information indicating a posture change of the vehicle 1; in order to suppress relative positional displacement of the virtual image and the road surface due to the posture variation of the vehicle 1, a dynamically changing first position adjustment amount C10 is set in association with the posture variation amount information, and in the case of the posture variation of the vehicle 1, 1) a first image adjustment process of adjusting the position of the first virtual image V10 in accordance with the first position adjustment amount C10 is performed; 2) The second image adjustment processing of adjusting the position of the second virtual image V20 according to the second position adjustment amount C20, which suppresses adjustment of the position of the virtual image with respect to the change in posture of the vehicle 1, is performed on the position of the second virtual image V20 than the first image adjustment processing. In this way, it is considered that the position adjustment is performed on the second virtual image near the end portion in the virtual image display region according to the posture change of the vehicle, thereby suppressing the reduction of the virtual reality, and the amount of the position adjustment is suppressed, so that it is possible to prevent that a part or all of the second virtual image cannot be completely seen (does not fall into the virtual image display region), while the positional displacement between the virtual image and the real scene due to the posture change of the vehicle in the first virtual image near the central portion in the vertical direction in the virtual image display region is significantly reduced.

The head-up display device 20 described in the present specification includes: any of several embodiments display control means 30; a display surface 50a that emits display light; and a relay optical system 80 for projecting the display light from the display surface 50a to the projected portion 2. In this case, the advantages can be considered to be the same as described above.

The operations of the processing steps described above may be performed by one or more functional blocks of an information processing apparatus such as a general-purpose control circuit or a dedicated chip. Combinations of these modules, and/or well-known hardware combinations of functions in place of these are included within the scope of the invention.

The functional blocks of the display system 10 for a vehicle are arbitrarily executed by hardware, software, or a combination of hardware and software in order to execute the principles of the various embodiments described. Those skilled in the art will appreciate that the functional blocks illustrated in fig. 3 may be arbitrarily combined or separated into more than two sub-blocks to implement the principles of the illustrated embodiments. Thus, the descriptions in this specification optionally support any possible combination or split of the functional blocks described in this specification.

Claims

1. A display control method for controlling a head-up display device having a display unit mounted on a vehicle and displaying an image on a display surface, wherein a virtual image is visually observed by projecting light of the image onto a projected portion so that the virtual image overlaps in a virtual image display area that overlaps with a road surface in front of the vehicle, characterized in that the method comprises:

Displaying at least a first virtual image displayed in a first area within the virtual image display area and a second virtual image displayed in a second area closer to an end of the virtual image display area in a vertical direction than the first area;

obtaining posture change amount information indicating a posture change of the vehicle;

In order to suppress the relative positional deviation between the virtual image and the road surface caused by the posture change of the vehicle, a dynamically changing first position adjustment amount is set in combination with the posture change amount information;

When the posture of the vehicle changes,

1) performing a first image adjustment process on the position of the first virtual image according to the first position adjustment amount;

2) A second image adjustment process is performed on the position of the second virtual image, in which adjustment of the position of the virtual image with respect to a change in the posture of the vehicle is suppressed compared with the first image adjustment process. The second image adjustment process is performed according to a second position adjustment amount.

2. The display control method according to claim 1, further comprising:

further acquiring information indicating a prediction of a posture change of the vehicle or prediction-related information including a predicted value of a posture change of the vehicle;

When the prediction of the posture change of the vehicle is a first prediction state according to the prediction related information,

2) performing a second image adjustment process on the position of the second virtual image to suppress a change in the position of the virtual image relative to the posture of the vehicle compared to the first image adjustment process;

When the predicted change in the posture of the vehicle is a second predicted state smaller than the first predicted state,

3) Performing a first image adjustment process of adjusting the position of the first virtual image and the position of the second virtual image according to the first position adjustment amount.

3. The display control method according to claim 1, further comprising:

When it is determined that the posture change of the vehicle is large or the posture change of the vehicle is frequent, the number of contents of the second image adjustment process to be executed is increased.

4. The display control method according to claim 1, further comprising:

After the posture change of the vehicle becomes equal to or larger than a predetermined threshold, when the posture change of the vehicle becomes less than the predetermined threshold, the position of the second virtual image is adjusted by switching from the second image adjustment process to the first image adjustment process.

5. The display control method according to claim 1, further comprising:

When the second image position adjustment processing is executed, in a case where the second virtual image changes from a state configured within the second position adjustment range of the virtual image display area to a state configured outside the second position adjustment range, a third position adjustment processing is performed on the first virtual image and the second virtual image, which suppresses the change in the position of the virtual image relative to the posture of the vehicle compared to the second image position adjustment processing.

6. The display control method according to claim 1, further comprising:

disposing the second virtual image so as to be visually observed below the first virtual image;

When the vehicle is leaning forward and the first virtual image that is offset changes from a state configured within the first position adjustment range to a state configured on the upper side of the first position adjustment range when the first image position adjustment processing is performed, a third image position adjustment processing is performed to fix the position of the first virtual image at an upper peripheral portion within the first position adjustment range and, at the same time, fix the position of the second virtual image at a specified position below the upper peripheral portion within the first position adjustment range.

7. The display control method according to claim 1, further comprising:

When the vehicle is tilted backward and the offset second virtual image changes from a state configured within the second position adjustment range to a state configured on the lower side of the second position adjustment range when the second image position adjustment processing is performed, a third image position adjustment processing is performed to fix the position of the second virtual image at the lower peripheral portion within the second position adjustment range and, at the same time, fix the position of the first virtual image at a specified position above the lower peripheral portion within the second position adjustment range.

8. The display control method according to claim 1, further comprising:

The second position adjustment amount is changed according to the vertical position of the second virtual image in the virtual image display area, and is reduced as the position of the second virtual image is closer to the vertical end of the virtual image display area.

9. A display control device for controlling a head-up display device, the head-up display device having a display unit mounted on a vehicle and displaying an image on a display surface, wherein a virtual image is visually observed by projecting light of the image onto a projected portion so that the virtual image overlaps in a virtual image display region overlapping with a road surface in front of the vehicle, characterized in that:

one or more control circuits;

Memory; and

one or more computer programs stored in the memory and configured to be executed by the one or more control circuits,

In the control circuit,

In order to suppress the relative positional deviation between the virtual image and the road surface caused by the posture change of the vehicle, a dynamically changing first position adjustment amount is set in combination with the posture change amount information. When the posture of the vehicle changes,

10. A head-up display device, comprising:

A display unit, which is mounted on the vehicle and displays an image on a display surface;

a transfer optical system that projects the display light from the display portion to the projected portion;

one or more control circuits;

Memory; and

The method comprises superimposing a virtual image in a virtual image display area which overlaps with a road surface in front of the vehicle and visually observing the virtual image, wherein:

In the control circuit,

obtaining posture change amount information indicating a posture change of the vehicle,