Researchers have developed a new deep learning method that simplifies the creation of holograms, allowing 3D images to be generated directly from 2D photos taken with standard cameras. This technique, involving a sequence of three deep neural networks, not only streamlines the hologram generation process, but also outperforms current high-end GPUs in speed. It does not require expensive equipment such as RGB-D cameras after the training phase, making it cost-effective. With potential applications in high-fidelity 3D displays and in-car holographic systems, this innovation marks a significant advance in holographic technology.
Researchers propose a new method that uses deep learning to create three-dimensional holograms from two-dimensional color images.
Holograms provide a three-dimensional (3D) view of objects, offering a level of detail that two-dimensional (2D) images cannot match. Their realistic and immersive display of 3D objects makes holograms incredibly valuable in a variety of sectors, including medical imaging, manufacturing and virtual reality.
Traditional holography involves recording three-dimensional data of an object and its interactions with light, a process that requires high computing power and the use of specialized cameras to capture 3D images. This complexity has limited the widespread adoption of holograms.
In-depth tutorial on generating holograms
Recently, many deep learning methods have also been proposed to generate holograms. They can create holograms directly from 3D data captured using RGB-D cameras that capture an object’s color and depth information. This approach circumvents many computational challenges associated with the conventional method and represents an easier approach to generate holograms.
Revolutionary holography with an innovative approach
Now, a team of researchers led by Professor Tomoyoshi Shimobaba from the Graduate School of Engineering, Chiba University, proposes a new approach based on deep learning that further streamlines the generation of holograms by creating 3D images directly from ordinary 2D color images captured with ordinary cameras . Yoshiyuki Ishii and Tomoyoshi Ito of the Graduate School of Engineering, Chiba University were also part of this study, which was recently published in the journal Optics and lasers in technology.
Explaining the rationale behind this research, Prof. Shimobaba says, “There are several issues in realizing holographic displays, including the acquisition of 3D data, the computational cost of holograms, and the transformation of holographic images to match the characteristics of a holographic display device. We undertook this study because we believe that deep learning has developed rapidly in recent years and has the potential to solve these problems.”
The three-step process of deep learning
The proposed approach uses three deep neural networks (DNNs) to transform a regular 2D color image into data that can be used to display a 3D scene or object as a hologram. The first DNN uses a color image captured using an ordinary camera as input and then predicts the associated depth map, providing information about the 3D structure of the image.
Both the original RGB image and the depth map created by the first DNN are then used by the second DNN to generate a hologram. Finally, the third DNN refines the hologram generated by the second DNN, making it suitable for display on various devices.
The researchers found that the time taken by the proposed approach to process data and generate a hologram was better than that of a state-of-the-art GPU.
“Another notable advantage of our approach is that the reproduced image of the final hologram can represent a natural 3D reproduced image. Furthermore, since depth information is not used during hologram generation, this approach is inexpensive and does not require 3D imaging devices such as RGB-D cameras after training,” adds Prof. Shimobaba while discussing the results further.
Future Applications and Conclusion
In the near future, this approach may find potential applications in heads-up and head-mounted displays to generate high-quality 3D displays. Likewise, it could revolutionize the generation of a holographic head-up display in the car, which may be able to present the necessary information about people, roads and signs to passengers in 3D. Thus, the proposed approach is expected to pave the way for increasing the development of ubiquitous holographic technology.
Well done to the research team for this remarkable achievement!
Reference: “Multi-Depth Hologram Generation from 2D Images Using Deep Learning” by Yoshiyuki Ishii, Fan Wang, Harutaka Shiomi, Takashi Kakue, Tomoyoshi Ito, and Tomoyoshi Shimobaba, 2 Aug 2023. Optics and lasers in technology.
DOI: 10.1016/j.optlaseng.2023.107758