Imagine silently mouthing something to yourself, and your AI assistant knows what you're trying to say. It could be via your glasses or your earbuds or your phone's camera. Apple just purchased a company, called Q.ai, that's trying to do this exact thing. That sounds weird and sci-fi, and yet to me, as someone who's been looking at smart glasses and wearables for a long time, it sounds very familiar, too.
Apple's investment in this Israeli startup isn't small at all. The acquisition cost around $2 billion, according to the original report by the Financial Times and news outlets like Reuters. It's more than any other Apple move since its splashy acquisition of Beats a decade ago. Unlike with Beats, though, nobody knows about Q.ai. At least, not yet. Still, the possibilities for new interfaces could be very strong. Another key piece is being added to the ever-expanding puzzle of future personal tech interfaces.
Q.ai isn't a company I've ever met with or gotten a demo from, but one of its founders, Aviad Maizels, also created PrimeSense, the infrared-based technology that powered the room-scanning 3D capabilities of Microsoft's Kinect camera for the Xbox years ago. PrimeSense was acquired by Apple in 2013. That tech became the TrueDepth camera array for Face ID, and it also lives in Apple's Vision Pro for near-range hand tracking.
Judging by what's been reported about its patents, Q.ai allows tracking of small facial movements and emotional expressions with optical sensors and could enable silent command input to an AI interface or the recognition of other subtle facial cues. The Israeli site GeekTime goes into a bit more detail, saying the tech would measure muscle and lip movements and might need to be near your mouth.
CNET reached out to Apple and Q.ai for comment, but neither responded immediately.
The Vision Pro can already track face movements, but it can't convert lip movements into speech. Numi Prasarn/CNET
Part of a new interface system for wearables and glasses?
I just wrote about how Apple is already showing signs of moving toward an ecosystem of connected AI wearables: pins, glasses, earbuds, watches or some combination thereof. Any of these wearables could potentially use what Q.ai is developing. It sounds like headphones and glasses are the two most likely areas, though, and with reports that the next generation of AirPods will have infrared cameras onboard, the pieces look even more ready to connect.
Even mixed-reality headsets like the Vision Pro could leverage Q.ai's tech. The Vision Pro can already recognize facial expressions with its eye-tracking cameras, downward-facing cameras and infrared sensors. But interacting with Vision Pro is still a little awkward to me. I use my eyes to gaze and my hands to pinch things, but I have to say "Hey Siri" to make audio requests. I'd rather my interactions feel more natural and subtle. Maybe this new acquisition could help.
As noted augmented reality artist and researcher Helen Papagiannis notes in her recent newsletter, "Apple's rumoured AI pin makes sense less as a standalone product and more as a node in Apple's ecosystem, drawing on shared sensing, intelligence, and context across devices working in concert with AirPods and, eventually, glasses."
... continue reading