An AI-powered gun detection system hooked up to a Baltimore County high school’s cameras mistook a bag of Doritos chips as a weapon — and called the cops on a 16-year-old student.
As local news station WBAL-TV 11 News reports, Taki Allen was enjoying the snack while sitting outside of Kenwood High School after football practice.
Twenty minutes later, he was visited by a small army of heavily-armed police officers.
“It was like eight cop cars that came pulling up for us,” he told WBAL-TV 11 News. “At first, I didn’t know where they were going until they started walking toward me with guns, talking about, ‘Get on the ground,’ and I was like, ‘What?'”
“They made me get on my knees, put my hands behind my back, and cuffed me,” Allen added. “Then, they searched me and they figured out I had nothing.”
“I was just holding a Doritos bag — it was two hands and one finger out, and they said it looked like a gun,” the student said.
The incident highlights glaring shortcomings with current gun detection systems, which are being rolled out at schools across the country. That’s not to mention the problematic privacy concerns of monitoring students with flawed AI tech or the outsized role law enforcement plays in public schools.
Besides false positives, gun identification software has proven unable to prevent deadly shootings, such as the one at Antioch High School in suburban Nashville earlier this year.
Other systems focused on gun detection have previously been accused of furthering racial biases, raising the possibility that Black students, like Allen, could be facing AI-facilitated discrimination while spending time at school.
The Baltimore County Public Schools system rolled out Virginia-based startup Omnilert’s gun detection tech last year. Once hooked up to public cameras, it can scan surveillance footage and alert police to potential weapons in real time.
... continue reading