Apple’s latest AI model listens for what makes speech sound ‘off’, here’s why that matters

As part of its fantastic body of work on speech and voice models, Apple has just published a new study that takes a very human-centric approach to a tricky machine learning problem: not just recognizing what was said, but how it was said. And the accessibility implications are monumental. In the paper, researchers introduce a framework for analyzing speech using what they call Voice Quality Dimensions (VQDs), which are interpretable traits like intelligibility, harshness, breathiness, pitch monotony, and so on. These are the same attributes that speech-language pathologists pay attention to when evaluating voices affected by neurological conditions or illnesses. And now, Apple is working on models that can detect them too. Teaching AI to hear and to listen Most speech models today are trained primarily on healthy, typical voices. This means they tend to break or underperform when users sound different. This is obviously a huge accessibility gap. Apple’s researchers trained lightwe ... Read full article.