GoKawiilFlock cameras may have accuracy problems. Here’s why that matters. Justin Sullivan/Staff/Getty
Flock Safety can’t seem to stay out of the news recently, from the latest wave of attacks on Flock surveillance cameras to claims that Flock’s gunshot detection microphones are also picking up private home conversations. But one specific report has caught my attention — a study showing a startlingly high failure rate for the surveillance system AI.
Read more: Everything You Should Know About Flock Surveillance and How it Works
Following up on complaints about Flock Safety flagging vehicles incorrectly, the police department of Roseville, California, ran its own analysis of the cameras, studying nearly 1,500 Flock reports. They found an astonishing 71% failure rate, where the cameras would mistake one license plate number for another.
That’s not just a snafu. Mistaking one number for another, like seeing a 7 as a 1, can lead Flock cameras to incorrectly identify an innocent vehicle as one that was previously stolen. That’s exactly what happened in the California situation, where a citizen complained that the cameras were repeatedly flagging his car to police as stolen because the AI technology interpreted a 9 as an 8.
This is a stark contrast to Flock’s own claims — sort of. The company reports that independent evaluations show its cameras have a 96% vehicle count accuracy rating, while correctly categorizing 92% into vehicle types. But accurately recognizing vehicles is a far cry from accurately reading license plates, which Flock doesn’t provide much data on. I’ve reached out to Flock to learn more, and I’ll let you know if the company has additional details to provide.
Why this Flock failure rate could cause trouble
If Flock camera accuracy isn’t reliable, what reason do cities have to use them? Bloomberg/Contributor/Getty
It’s important to note that this analysis was conducted between 2023 and 2024. Many AI Large Language Models can improve over time with better training and more access to material, which is one reason AI companies are literally devouring old books these days. So Flock’s cameras may have improved in accuracy by now. However, we’ve also seen the opposite happen: AI models can eat up bad data, especially “recursive” data that was created by another AI, and start becoming dumber, less accurate when performing their tasks.
Flock has said that the camera setup Roseville used wasn’t typical, but didn’t provide more details. It’s possible that where cameras are placed and when they are activated (at night, in the rain, etc.) can have a significant impact on accuracy. So can license plate covers, dirty license plates and unusual license plate placement.
... continue reading