In a groundbreaking development, Stanford engineers have seamlessly integrated advanced display technologies, holographic imaging, and artificial intelligence to create an augmented reality (AR) experience that overlays full-colour, 3D moving images onto the natural world through a regular pair of glasses.
Pioneering Spatial Computing
The new AR headset prototype, developed by researchers in spatial computing, employs holographic imaging to project vibrant, full-colour 3D images directly onto the lenses. Unlike current bulky AR headsets, this innovation promises a comfortable, compact, and aesthetically pleasing form factor that you would be proud to wear all day long.
“Our headset looks like an ordinary pair of glasses to others, but the wearer sees an enriched world with dynamic, full-colour 3D imagery,” explains Gordon Wetzstein, an associate professor of electrical engineering at Stanford and a leading expert in spatial computing. Wetzstein and his team detailed their breakthrough in a recent paper published in Nature.
Although still in the prototype stage, this technology has the potential to revolutionise industries such as gaming, entertainment, training, and education. For instance, Imagine a surgeon using these glasses for complex surgical planning, an aeroplane mechanic learning to work on the latest jet engine, or a student exploring ancient civilisations in a virtual history lesson. “The possibilities are endless,” says Manu Gopakumar, a doctoral student and co-first author of the paper.
Overcoming Technical Challenges
This novel approach successfully navigates the engineering complexities that have traditionally resulted in cumbersome headsets or subpar 3D experiences that often cause visual fatigue or nausea. The researchers overcame these challenges by combining AI-enhanced holographic imaging with innovative nanophotonic devices, a combination that has not been used in AR systems before.
“No other AR system currently offers the same compact form factor or matches our 3D image quality,” says Gun-Yeal Lee, a postdoctoral researcher and co-first author of the paper.
The researchers overcame technical barriers by combining AI-enhanced holographic imaging with innovative nanophotonic devices. Traditional AR systems often require complex optical setups where users see a digitized approximation of the real world. These setups involve cameras capturing real-time imagery, combined with computed imagery and projected to the user’s eye, necessitating bulky equipment.
Innovative Holographic Imaging
The Stanford team advanced beyond these limitations by using holography, a technique developed in the late 1940s, to create more realistic 3D images. This is a significant breakthrough in holographic imaging, which has historically faced challenges in portraying accurate 3D depth cues, sometimes resulting in nausea-inducing visuals. However, the team employed AI to enhance these depth cues and leveraged advances in nanophotonics and waveguide display technologies to project holograms onto the glasses’ lenses without bulky optics.
The waveguide, constructed with nanometer-scale patterns etched onto the lens surface, allows small holographic displays mounted at each temple to project imagery through these patterns. The light bounces within the lens before reaching the viewer’s eye, creating a blended view of the natural world and full-colour 3D images.
Achieving True-to-Life 3D
This approach delivers a superior 3D effect by combining stereoscopic (a technique that creates the illusion of depth in an image) and holographic techniques, providing a lifelike visual experience. “Holography gives you the full 3D volume in front of each eye, enhancing the lifelike quality of the images,” explains Brian Chao, a doctoral student and co-author of the paper.
The culmination of these waveguide display techniques and improvements in holographic imaging results in a true-to-life 3D experience that is visually satisfying and free from the fatigue associated with earlier methods. “Holographic displays have long been seen as the ultimate 3D technology, but they’ve never quite made a commercial breakthrough. Perhaps we’ve found the killer app they’ve needed,” says Wetzstein.