GoKawiilTL;DR The Pixel 11 Pro Fold supports transcribing American Sign Language (ASL) into text.
It uses the front camera to see what the user is signing, and displays the translation on the cover screen as well.
The feature currently only supports ASL and is available in select countries and regions.
Google today announced the Pixel 11 series, including its newest foldable: the Pixel 11 Pro Fold. While it’s easy to get distracted by the foldable Pixel’s sleek design and new 48MP camera, the phone also comes with a feature that could prove to be very useful.
The Pixel 11 Pro Fold supports transcribing American Sign Language to text. The phone will use its camera to see and understand ASL gestures and transcribe them into text directly on the screen. This could make it easier for ASL users to communicate with others, even if they don’t know sign language.
Users can simply open the Translate app and tap the sign language icon in Gboard’s top toolbar to start. The feature will use the Pixel 11 Pro Fold’s front camera to recognize what the user is signing, and the transcription will also be displayed on the cover screen, so the other person can easily read it and respond.
The feature uses Google’s new sign-language-to-text (SL2T) translation model. The model can “see” and understand physical movement, including the movement of a user’s hands, arms, torso, head, and face. It uses this information to understand the meaning being conveyed through the sign language and translate it into English. Google says SL2T was trained on over 100,000 hours of sign language data across more than 50 languages, with roughly a quarter of the data in ASL.
The new foldable Pixel also supports transcribing speech to text, which could be a useful accessibility feature, especially for those who are hard of hearing or deaf. The feature provides a text field for typing responses or questions for the other person to read.
ASL transcription is currently available on the Pixel 11 series in select countries and only supports American Sign Language-to-English translation. However, Google has confirmed that support for more devices and additional languages is coming soon.
... continue reading