GoKawiilAliyah, a light-skinned Black woman dressed in country-western gear, is struggling to sell metal buckles she handmade on TikTok. In a video for the social media platform from March, she cries to the camera and pleads for views: “Even as a black woman, I have more faith that white women will stay 13 seconds [on this video] to save my belt buckle business,” the onscreen text reads. She wipes a tear off her cheek.
But Aliyah isn’t real, and neither are her supposedly handmade products — she’s one of many AI-generated influencers created to sell mass-produced products via dropshipping on TikTok, Facebook, and Instagram. Identical belt buckles — sunflower design, detachable knife inlay, and all — are sold on the fast-fashion site Shein, and for a quarter of the price.
Aliyahsbuckles on TikTok (left), and Amaya’s buckles on TikTok (right)
There are some clues to spot to determine that this video is AI-generated. Aliyah’s voice is robotic and emotionless, which doesn’t match the crying face on the screen. In one clip, she is sewing a leather belt where there wouldn’t usually be sewing at all. When she wipes a tear off her face, the stream of liquid below where she wipes also disappears. And lastly, there are dozens of uncannily comparable videos, but with different AI-generated characters, circulating on TikTok. One, a profile for an account called “Aliyahsbuckles,” features an identical background, tabletop, and spool of twine.
The Verge found dozens of accounts on TikTok with similar narratives and a variety of dropshipping products, including belt buckles, mugs shaped like cowboy boots, crochet bags, and cardigans. Some of these videos are labelled as AI-generated. Similar accounts are also active on Instagram and Facebook. Nearly all aspects of the accounts appear to be AI-generated — from the “person” in the video to automated responses to comments, which in some cases attempt to mimic African American vernacular — and experts warn scams like this are growing every day.
“It’s massive,” Jeremy Carrasco, a researcher of AI-generated media and director of Riddance.ai, an organization that focuses on AI video detection, told The Verge of AI-generated videos connected to e-commerce stores. “Most of them aren’t coordinated. Some of them are coordinated. A lot of the time they’ll run a single [AI-generated] actor, or a couple actors will run all sorts of shops,” he explained. Those AI-generated avatars pretend to make the items, go to fairs to display their products, and “respond” to comments through automation. “What we’re seeing right now are these retail scams where they’ll link to the Shopify websites.”
Carrasco estimates his research team is finding up to 100 accounts that attempt to sell products via AI-generated avatars every day. Most of the accounts found by The Verge were created in the last two months and contain videos about small businesses owned by marginalized individuals struggling to make a sale; these videos are incredibly similar, with only slight variations in their scripts. While we also found Native American, Hispanic, and white women characters, the most viewed and engaged-with AI-generated characters found by The Verge are Black women. Aliyah’s account alone has 40,000 followers.
Previous Next
... continue reading