In 2010, when Microsoft unveiled the Kinect, it pitched the camera as a revolutionary new gaming device. Swing an imaginary lightsaber and that would be translated onscreen. Throw a football and it would be caught on your TV. Fifteen years later, we know the Kinect as an expensive failure. Microsoft overestimated the demand for playing games with your body. But the Kinect did still turn out to be revolutionary — just not for gaming.
Now, we understand the Kinect is anything but a gaming device. It became a robotics game changer, enjoyed a brief dalliance with pornography, and is now upsold as a ghost hunting toy. None of which would have been possible had a community of hackers not come together to fashion open source drivers for the Kinect, freeing it from the limitations of being locked to the Xbox 360 and opening new frontiers of experimentation, creative expression, and commercial advancement.
“Technically, nothing the Kinect did was entirely new,” says Memo Akten, an artist working with code, data, and AI and an assistant professor at the University of California, San Diego. The small camera projected a grid of infrared dots and read deformities in that pattern to discern depth. In an early example of machine learning, it recognized human limbs and gestures. “Those capabilities existed in research and industrial systems for many years,” he adds. Those systems cost in the region of $5,000 to $12,000. Here was Microsoft selling a variation of the technology for $150.
“What had previously required very expensive equipment and/or complex multi-camera setups with manual alignments, calibration, and correspondence was now available off the shelf,” Akten continues.
Kyle Machulis, CEO of Nonpolynomial and founder of buttplug.io — an open source project for controlling sex toys — was working on $250,000 mapping systems not dissimilar to the Kinect in 2010. He quickly recognized the peripheral as an opportunity to “democratize that technology.”
He recalls heading out on November 4th to pick up a Kinect to reverse engineer. An hour later, New York-based DIY electronics producer Adafruit announced OpenKinect: a bounty of $1,000 — a prize that it would raise to $3,000 — for whomever offered evidence of the Kinect working on any operating system.
“Imagine being able to use this off the shelf camera for Xbox for Mac, Linux, Win, embedded systems, robotics, etc.,” Adafruit wrote in its announcement. “We know Microsoft isn’t developing this device for FIRST Robotics, but we could! Let’s reverse engineer this together, get the RGB and distance out of it and make cool stuff!”
Doing so was not a simple case of taking the Kinect apart or plugging it in. Though it could connect to a PC via USB, the way they communicated was unknown and the only way to get at it was to watch the Kinect and Xbox 360 speaking to one another. “Since the Kinect didn’t have PC drivers, we needed this piece of hardware called a USB sniffer,” Machulis tells The Verge. A colloquial term for a protocol analyzer, a USB sniffer is a tool that could record the data passed between the Kinect and Xbox 360. In 2010, that cost $1,200 and, Machulis says, “I really didn’t want to buy it.”
Some information could be gleaned by simply connecting the Kinect to the PC, but it was mostly unhelpful — power consumption, packet sizes, and confirming the Kinect is, in fact, a camera. Hackers could start sending random packets and possibly work something out, but it was just as liable to brick the Kinect completely.
Hackers and reverse engineers around the world were raring to go. But it appeared that whoever got their hands on a sniffer would win the bounty almost by default. That race wasn’t just for the money, however, but also the cachet of being the first to hack such a high-profile device. With the community stalled over the massive expense — almost half the bounty — it opened the door for someone outside the community to potentially snatch the glory away.
... continue reading