GoKawiilSign up to see the future, today Can’t-miss innovations from the bleeding edge of science and tech Email address Sign Up Thank you!
Amazon is catapulting us into the final boss of sub vs dub debates.
On Thursday, its Prime Video streaming service announced the launch of a new AI-powered feature that matches actors’ mouth movements to dubbed audio tracks — which we’re sure won’t result in nightmarish hallucinations of facial disfigurement.
For now, the company says that the AI-powered lip syncing will exclusively be available on the English dub of the German Series “Maxton Hall,” though there are plans to expand it to other titles in the future.
“At Prime Video, we’re constantly innovating with new tools that enhance the viewing experience for our customers,” Raf Soltanovich, vice president of technology at Prime Video and Amazon MGM Studios, said in a statement, per the New York Times. “With lip-syncing, we’re creating a more seamless and immersive way for customers to enjoy global content.”
Right now, it’s unclear how much of the feature is automated by AI, and there’re plenty of questions to be asked about how it’ll be deployed. Such as: in the future, will it be available on all titles, and will the rightsholders get a say in it? And isn’t this an overreach on Amazon’s part in how much it’s determining how a work of art is being presented? Will an actor’s unique facial movements be smoothed down into something generic? And lest we forget the most obvious question mark: how convincing will it be? (It feels telling that Amazon hasn’t posted any clips of its tech in action.)
It’s not the first time Amazon Prime Video has tried something like this. Last year, it rolled out an experimental “AI-aided” dubbing feature for a dozen movies and shows in English and Spanish. It also experimented with showing AI-generated recaps of TV shows, and an AI tool that picked movie and show recommendations based on similar character arcs and plot points. Amazon, in other words, is trying to make consuming the content it provides as frictionless as possible.
There are plenty of valid reasons for preferring dubs, especially for those with accessibility issues. But the fact that actors in a movie might be speaking a different language from your own is being treated as something that needs to be erased and covered up is a bit worrying — at least in our humble opinion.
More on art: Gal Godot Starring in Heavily AI-Generated Movie About the Glories of Bitcoin