Envision the bustling activity of a city street during rush hour, where vehicles and bicycles zip past, and pedestrians weave through the crowded sidewalks. Amidst the glare of shop windows and the shadows of underpasses, our brains seamlessly process these dynamic scenes. This effortless human capability presents a significant challenge for machines equipped with 3D-sensing systems, such as those used in self-driving cars.
In a promising development, researchers from the Computational 3D Imaging and Measurement Lab at the University of Arizona have made strides in improving machine 3D vision. Their work, published in Nature Communications, aims to equip machines with “superhuman 3D vision,” according to lab director Florian Willomitzer, an associate professor at the Wyant College of Optical Sciences.
The research team is not merely replicating human 3D vision but is enhancing 3D sensors to capture images with higher resolution and speed, even under challenging conditions like reflective surfaces. “Humans already have a built-in 3D camera system – the stereo vision of our two eyes,” Willomitzer explained. “Our goal is to enable computers and machines to see in 3D better than any human, which is crucial for tasks such as reliable navigation of self-driving cars and accurate guidance during robotic surgery.”
However, 3D imaging technology faces a persistent issue: current sensors are optimized for either “diffuse” (matte) or “specular” (reflective) surfaces. Real-world environments, such as a car interior or a surgical site, often include a mix of these surface types, posing a challenge for existing 3D imaging systems.
Willomitzer’s team has built upon deflectometry, a technique that measures specular surface shapes by observing deformation in reflected patterns. Traditional methods require large screens, leading to costly and inflexible setups. Instead, the team has innovatively used the surrounding environment as a “virtual screen” for measurements. Aniket Dashpute, a doctoral student at Rice University and the study’s first author, described how their approach uses a laser scanner to capture everything in a room, transforming the entire space into a virtual display for deflectometry.
Rather than using conventional cameras, the researchers employ a neuromorphic event camera, which captures only essential measurement parts at high time resolution. This allows for high frame rate 3D video capture of mixed reflectance scenes with moving objects. “The event camera can handle vastly different light levels – from very dim to extremely bright,” noted co-author Jiazhang Wang, a postdoctoral research associate at the Wyant College of Optical Sciences. This capability ensures accurate measurement of object surfaces regardless of their reflectivity.
The research has so far been demonstrated in a laboratory setting, but Willomitzer emphasizes the scalability of the technology. “Scalability is an important requirement for the wide spectrum of 3D imaging applications,” he said, highlighting potential uses from surgical imaging to room digitization.
Contributors to the study include James Taylor, a doctoral student at the Wyant College of Optical Sciences; Oliver Cossairt, an adjunct associate professor at Northwestern University; and Ashok Veeraraghavan, a professor at Rice University. The research received support from the National Science Foundation and the Defense Advanced Research Projects Agency.
Read More Here
Comments are closed.