Background
Non-line-of-sight (NLOS) imaging is an emerging field of research with applications that include autonomous vehicle collision avoidance, search and rescue operations, industrial inspection, and endoscopy. Because NLOS objects are outside of the line-of-sight (LOS) of both the camera and illumination source(s), computational algorithms are required to reconstruct the scene from indirect light paths.
Recent successes in NLOS imaging has relied on measurement of the time-of-flight of light to distinguish between NLOS from LOS path contributions. These techniques include pulsed lasers and streak cameras, and single-photon avalanche diodes (SPADs). Using time-resolved measurements, backpropagation and optimization can reconstruct NLOS scenes, typically under the assumption of a flat LOS wall to avoid LOS indirect light. Although these time-resolved detectors achieve superior results, they are costly, consume more power than conventional CMOS image sensors, and require specialized optics. To alleviate these issues, research has begun to focus on NLOS imaging using conventional cameras.
Invention Description
Researchers at Arizona State University have developed a new data-driven approach for NLOS 3D localization requiring only a conventional camera and projector. Greater than 90% accuracy is achieved in localizing a NLOS object to a 5cm x 5cm voxel. An adaptive lighting algorithm generalizes to line-of-sight (LOS) scenes with non-planar surfaces. Based on radiosity, this algorithm uses convolutional neural network (CNN) architecture to identify and illuminate scene patches in the LOS which most contribute to the NLOS light paths. Use of this adaptive lighting method improved accuracy by 6%-15% with a non-planar LOS wall, demonstrating the advantage of combining the physics of light transport with active illumination for data-driven NLOS imaging.
Potential Applications
• Visual tracking
• Robotics
• Biomedical imaging
Benefits and Advantages
• Cost-Effective – Achieves NLOS localization without expensive time-resolved detectors, using only a conventional 2D camera and a projector
• Accurate – Demonstrates >90% localization accuracy to a 5cm x 5cm voxel
• Innovative – To the best of the inventors’ knowledge, this is the first published work to perform NLOS imaging on non-planar LOS scenes
Laboratory Homepage of Professor Suren Jayasuriya