View synthesis for 3D computer-generated holograms using deep neural fields

Area of Science:

• Optics and Photonics
• Computer Vision
• Artificial Intelligence

Background:

• Computer-generated holography (CGH) reconstructs images by simulating light wave propagation and interference.
• Current CGH methods are often limited to specific viewpoints, hindering immersive applications like virtual and augmented reality.
• The need exists for holographic systems that enable free-viewpoint observation, mimicking natural human vision.

Purpose of the Study:

• To develop a novel neural network-based representation for light wave propagation.
• To enable the generation of holograms viewable from arbitrary six-degree-of-freedom (6DoF) viewpoints.
• To transform sparse 2D photographic data into dynamic, multi-view 3D holographic representations.

Main Methods:

• Training a neural network to learn wavefield representations from sparse 2D images.
• Encoding light intensity from photographs into a neural network model.
• Implicitly learning amplitude and phase surrogates of light waves.
• Predicting continuous complex wavefronts for arbitrary view synthesis.

Main Results:

• Introduction of the Neural Holographic Field (NHF), the first AN N-based representation for free-space light wave propagation.
• Successful transformation of sparse 2D photos into 3D holograms with arbitrary viewpoint accessibility.
• Demonstration of visualizing smartphone-captured scenes from 6DoF viewpoints on a prototype holographic display.

Conclusions:

• The Neural Holographic Field offers a new paradigm for generating view-independent holograms.
• This method bridges the gap between 2D image capture and realistic 3D holographic display.
• The technology has significant potential for enhancing immersive experiences in virtual, augmented, and mixed reality applications.