GoKawiilLidar can be used to see objects hidden around corners. However, until now, such a feat required lab-grade devices. A new study reveals off-the-shelf smartphone-grade lidar, which costs less than US $100, can also help see around corners.
The advance may have a host of potential applications. “In autonomous driving, around-the-corner sensing could help self-driving cars detect other vehicles, cyclists, or pedestrians before they come into direct view, improving safety at blind intersections or obstructed roads,” says Siddharth Somasundaram, a doctoral student at MIT’s Media Lab. “In robotics, it could help robots navigate cluttered or partially hidden environments.”
More broadly, “we think the most important implication is the democratization of the technology,” Somasundaram says. “When technologies like this become accessible, people often discover applications far beyond what the original researchers imagined.” The scientists have publicly released the code required to perform such work.
How lidar sees around corners
Lidar is increasingly giving 3D scanning capabilities to autonomous vehicles, drones, robots, and smartphones. A lidar sensor uses light much as radar uses radio waves—it shines a laser onto a location and analyzes how long reflected pulses take to return in order to calculate distances and generate a 3D map of a place.
By analyzing laser pulses that bounce off reflective surfaces, lidar can see items obscured from their direct line of sight, such as something hidden behind a corner. However, the first examples of such non-line-of-sight imaging “relied on extremely specialized scientific equipment often costing [US] $0.5 million to $1 million,” Somasundaram says. “These systems were large, expensive, and confined to laboratory environments.”
Still, over the years, higher-grade sensors—such as single-photon detectors—began appearing in consumer hardware, Somasundaram says. “Once we started experimenting with these sensors, we realized that even off-the-shelf devices were actually capturing faint around-the-corner signals,” he explains.
But such findings did not necessarily mean these sensors could actually prove useful in non-line-of-sight imaging. Consumer lidar typically captures images that are full of noise, due in part to the low-power lasers it uses because of eye safety concerns. In addition, the consumer nature of these devices meant the sensors are often relatively low resolution. Moreover, movements of the camera and target objects could result in blurry pictures.
“There were definitely moments where we weren’t sure whether meaningful imaging would even be possible,” Somasundaram says.
To overcome these challenges, instead of attempting non-line-of-sight imaging based on the data within individual pictures, the researchers analyze multiple images at once. They note they were inspired by how smartphone cameras often take bursts of photos in quick succession to improve their merged quality and how synthetic aperture radar in satellite imaging mixes signals from multiple antennas to capture images with the quality of a single large antenna.
... continue reading