2.1.

Optics of Light Field VR

In Fig. 1, we observe the generation of an elemental image (EI) array through a lens array using spatial multiplexed LF optics. Unlike non-LF optics, LF technology enables the provision of volumetric virtual images that accurately simulate the correct disparity of each EI, allowing viewers to experience correct eye accommodation. This eliminates the need for VAC for observers. By carefully considering first-order design parameters and boundary conditions, such as FOV, resolution, total thickness, and overlapping ratio with focal length of microlens array (MLA), as depicted in Fig. 2, we can achieve optimal results.

Fig. 1

Multi-depth image via spatial multiplexed LF optics with EI.

Fig. 2

First order design parameters and boundary conditions: (a) FOV, (b) resolution, (c) total thickness, and (d) overlapping ratio with focal length of lens array and object distance (gap between lens array and panel).

In this paper, we focus on a newly developed LCD by INNOLUX, boasting a resolution of $3240\times 2880\text{pixels}$ and a high-pixel density of 1440 PPI (pixels per inch). The effective display area measures $58.32\times 51.84{\mathrm{mm}}^2$. Considering a monocular viewing scenario, the best FOV achieved is $70\mathrm{deg}\times 65\mathrm{deg}$ with an eye relief of 29 mm and an eye box of 6.4 mm. To further enhance the horizontal FOV in binocular vision, a 15-deg tilt between the panels is incorporated, as illustrated in Fig. 3. This design modification expands the binocular FOV to an impressive $100\mathrm{deg}\times 65\mathrm{deg}$, with an angular resolution exceeding 15 pixels per degree (PPD).

Fig. 3

Designed LF (a) optical layout of designed LF and (b) MTF of designed LF optics.

To ensure high-quality image reproduction, the designed modulation transfer function (MTF) across the entire image field surpasses 30% at the Nyquist frequency of 28 line pairs per millimeter (lp/mm), as demonstrated in Fig. 3. This indicates that the display system is capable of accurately reproducing fine details and maintaining image clarity. Through these advancements, our research contributes to the development of LF display technologies, enabling an immersive visual experience with volumetric virtual images, expanded FOV, and improved image quality.

2.2.

High-Resolution VR LCD

2.2.1.

Wide field of view

High resolution (large panel size) provides a wide FOV.¹² High resolution also helps to simplify the optical system design of the VR head-mounted display. However, there are more gate lines in a high-resolution panel than in a low-resolution panel. The more gate lines take more scanning time. This will decrease the refresh rate and increase moving picture response time (MPRT).¹³ Thus an optimization between the resolution and MPRT is needed. INNOLUX proposes a 3K3K resolution 1411 PPI display panel, which provides a virtual image with binocular FOV 100 deg in VR optical system and also maintains the 15 PPD image quality, as shown in Fig. 4.

Fig. 4

FOV improvement from 80 deg to 100 deg.

2.2.2.

Fast moving picture response

MPRT is a general indicator of the motion quality of a display. Generally speaking, the MPRT of a display panel in VR system should be $<1.5\mathrm{ms}$, otherwise, users will see the motion blur when an object moves fast in the screen. To achieve such a fast MPRT in a 3K3K high-resolution display panel, we set the refresh rate at 75 Hz with a 10% duty ratio blinking backlight. We also improve the scanning time of 3000 gate lines within 9 ms and LC response time within 3 ms. The measured MPRT is about 1.1 ms as shown in Fig. 5, which provides a high-quality moving picture without motion blur.

Fig. 5

(a) Addressing timing and (b) LC response time.

2.2.3.

Circuit design

When the panel resolution is increased, the loading of the gate driver circuit (GDC) will become heavier, so it is necessary to increase the size of the components to improve the driving capability, resulting in larger left and right borders. When the resolution is increased, the vertical space of GDC of each pixel will become smaller, so the circuit design can only be laid out in the horizontal space, resulting in larger borders on the left and right sides of the panel. Considering the requirement of interpupillary distance, the left and right borders need to be controlled below 2 mm. Therefore, Innolux developed GDC 1-to-4 driver circuit and short-channel component design to reduce borders in the future to meet the needs of products, as shown in Fig. 6.

Fig. 6

1:4 GDC: (a) GDC 1-to-2 and (b) GDC 1-to-4.

High-resolution and high-frequency driven pixel charging times are short. Therefore, small signal distortion will affect the ability to charge. When the resolution and driving frequency are increased at the same time, the RC load becomes heavier, and it is difficult to realize high-frequency driving. In order to solve the problem of insufficient charging, we use three-terminal input in the circuit design, strengthen the driving ability, optimize the signal line load, and reduce the signal distortion phenomenon. This enables higher-resolution high-frequency drive, provides faster picture refresh rates, and solves dynamic image smearing issues, as shown in Fig. 7.

Fig. 7

DEMUX three-end driving: (a) two-terminal input and (b) three-terminal input.

2.2.4.

High brightness and contrast ratio

The brightness and CR are also important specifications in a VR display. High brightness and high CR bring the immersion to the users.¹⁴ However, when the pixel becomes such a small size as 1411 PPI, many issues come out because of the low aperture ratio. For example, we have to adopt the metal layer as the black matrix to avoid the BM rounding effect, as we mention in our previous paper.¹⁵ The optical design should be optimized to keep the brightness and CR. In this paper, to increase the aperture ratio, we change the pixel arrangement so that two pixels share the black matrix near the TFT device, as shown in Fig. 8. This arrangement decreases the area of black matrix and increases the aperture ratio.

Fig. 8

(a) Traditional pixel arrangement, (b) BM shared pixel arrangement, (c) traditional aperture design, and (d) optimized aperture design.

The LC dark fringe caused by the relief of TFT device is ignorable in a large size pixel. However, it is critical in a small size pixel because the dark fringe results in a low CR. To improve the CR, we optimize the shape of aperture area such that the dark fringe is blocked and the bright fringe is transmitted. As a result, the CR is increased, as shown in Fig. 8.

2.3.

Visual Correction

In order to further simplify computation complexity in LFVR, a ray tracing-based graphic process “corrected eye box mapping” for full visual-corrected function is also included, correction for myopia, hyperopia, and even astigmatism are completed as reported perilously.¹⁸ In order to obtain comprehensive visual-corrected function, spherical power (SPH), cylinder power (CYL), and cylinder axis (AXIS) should all be considered. As shown in Fig. 9, corrected eye box is scaled into different sizes and directions based on the original eye box size. For adjusting the SPH, the corrected eye box size is scaled according to the center, as shown in Fig. 9(a). For adjusting the CYL and AXIS, the corrected eye box size is scaled in the different ratios and rotation angles from the original eye box size in specific axis, as shown in Figs. 9(b) and 9(c). Similarly, in cases of irregular astigmatism, the content will be rearranged based on the corrected eye box size. The simulations conducted in LightTools involve the replication of −4D astigmatism in different rotation angles, as demonstrated in Figs. 10(a) and 10(b). The proposed mapping method is then employed to obtain the corrected results, as illustrated in Fig. 10(c).

Fig. 9

Corrected eye box mapping for visual correction in (a) SPH and astigmatism of the (b) CYL and (c) AXIS.

Fig. 10

Proof of concept of LF vision correction (a) reference without astigmatism; (b) astigmatism from left to right: 0 deg, 45 deg, and 90 deg of AST axis; and (c) corrected with proposed eye box mapping method from left to right: 0 deg, 45 deg, and 90 deg.

The performance of vision correction was also examined using specific prescriptions: (−3 SPH, −1 CYL, and 45 deg AXIS) and (+2 SPH, +2 CYL, and 45 deg AXIS). To simulate myopia or hyperopia, a prescription lens with the opposite power and angle was placed in front of the camera. The resulting blurred images can be seen in Figs. 11(a) and 12(a) and displays the camera’s focus at 500 mm. The LF images in Figs. 11(b) and 12(b) demonstrate the successful correction of myopia, hyperopia, and even astigmatism.

Fig. 11

Captured LF image at 500 mm virtual distance (a) myopia (−3 SPH, −1 CYL, and 45 AXIS) and (b) corrected.

Fig. 12

Captured LF image at 500 mm virtual distance: (a) hyperopia (+2 SPH, +2 CYL, and 45 AXIS) and (b) corrected.

2.4.

Uniformity Improving

In the LF optics, showing in Fig. 13(a), there is a challenge posed by the partial overlap between EIs, which can result in a non-uniform luminance distribution in the final LF image. This often manifests as a noticeable grid or dot array pattern. To address this issue and further enhance the uniformity of the LF image, a compensating luminance distribution is typically employed, often based on a Gaussian model for each EI. This compensation technique aims to reduce the discrepancy between the maximum and minimum luminance values, as depicted in Fig. 13(b).

Fig. 13

Brightness uniformity of LF image (a) original without compensation, (b) Gaussian-based compensation, and (c) dynamic modulated compensation.

However, it is important to note that the pupil size of the observer can vary under different ambient brightness conditions. To account for this dynamic change and further improve the luminance uniformity, a modulated compensation approach coupled with an eye tracker is employed. This dynamic modulation compensates for variations in the pupil size and ensures a more consistent luminance distribution across the LF image, as illustrated in Fig. 13(c).

2.5.

Eye Box Expansion

In VR products, a larger eye box is a crucial requirement due to the automatic scanning nature of the human eye. However, designing a larger eye box in LF optics involves trade-offs with other first-order parameters. To address this challenge, two approaches have been studied and proposed, both utilizing information from an eye tracker: full digital image calculation and LC-based beam steering.

Figure 14 illustrates the results of the full digital image calculation approach. It is evident that the LF image is free from vignetting and can seamlessly combine even with a 3 mm shift in the eye pupil, as depicted in Fig. 14(b). Moreover, through experimentation, an eye box that is $\sim 2$ times larger has been successfully achieved.

Fig. 14

LF image with 3 mm pupil shift: (a) w/o processing and (b) w/ proposed full digital image calculation.

These findings highlight the effectiveness of the full digital image calculation approach in overcoming the trade-offs associated with designing a larger eye box in LF optics. It demonstrates the potential to enhance the user experience by providing a wider viewing area without compromising image quality or coherence.