This Is Flock’s AI Search Tool for Cops
Original reporting by Wired
Flock Safety’s latest search tools are an AI-powered surveillance system that allows police to find individuals across a network of cameras using written descriptions, rather than just license plates. These powerful capabilities, designed to enhance law enforcement investigations, have quickly become central to dozens of police-stalking cases, prompting widespread backlash and contract cancellations nationwide. Facing intense scrutiny—including allegations of serious privacy violations and misuse by officers—Flock responded with a package of changes, promising shorter data retention, mandatory case codes for searches, and automated auditing systems to catch abuse, all slated to be mandatory by year's end.
The Code's Verdict
WIRED investigated the code underlying these new "safeguards" that Flock claims will enhance accountability. Our analysis reveals that while the company's guardrails may discourage some abusive uses and record them for review, they fundamentally fail to prevent them. Crucially, the system's AI, which moderates search requests for sensitive content, allows queries related to "political, social, and cultural expression" to proceed with only a warning. Experts warn this category is among the most protected under the First Amendment, creating a significant vulnerability. This leaves a powerful surveillance apparatus with insufficient checks, raising serious concerns about potential misuse and chilling effects on civil liberties, as the risk of inaccuracies and abuse ultimately falls on the officer.
The investigation into Flock Safety’s surveillance tools reveals a critical juncture in the deployment of AI in law enforcement. Despite a growing backlash and the company’s recent pledges of enhanced safeguards, WIRED’s analysis shows that these measures largely serve to record potential misuse rather than fundamentally prevent it. The system's continued allowance of searches on protected categories like political expression, often with only a minor warning, underscores a persistent vulnerability that experts contend directly conflicts with constitutional rights. The inherent opaqueness of Flock’s AI, coupled with the company’s reluctance to disclose its inner workings and the specific parameters of its moderation, further compounds concerns, leaving a profound accountability gap.
Broader Implications
This dynamic positions Flock’s technology as a microcosm of the larger debate surrounding algorithmic policing. The promise of efficiency in crime-fighting must be stringently weighed against the demonstrated risks of inaccuracy, bias, and widespread abuse at an unprecedented scale. As ubiquitous camera networks become the norm across public spaces, the unchecked deployment of such powerful, yet flawed, search capabilities threatens to fundamentally alter the relationship between citizens and the state, eroding privacy and chilling free expression in the digital age. Moving forward, the imperative for robust, independent oversight, mandated transparency in AI design, and truly preventative technological safeguards—not merely auditing after the fact—will be paramount to responsibly navigate the societal impact of mass surveillance and safeguard civil liberties against structural erosion. This ongoing tension demands far more than corporate assurances; it requires systemic change and public deliberation to ensure fundamental rights are not sacrificed for perceived security.
Frequently asked questions
- How do police departments use Flock Safety's AI-powered camera surveillance systems?
- Flock Safety provides law enforcement with AI tools to search extensive camera networks. Police can track vehicles by license plate or attributes and find people using written descriptions. Features include continuous "watchlist" monitoring in designated areas for individuals matching criteria, and "People Detection Alerts" for activity in specific camera views, expanding surveillance capabilities beyond traditional vehicle tracking.
- What privacy and reliability issues are associated with Flock Safety's AI surveillance technology?
- Concerns include widespread police misuse for personal reasons, often targeting ex-partners or acquaintances. Experts also highlight the AI's unreliability at scale, particularly for person descriptions, and the lack of transparency regarding its accuracy. The system's content moderation allows officers to proceed with searches related to constitutionally protected expression after a warning, raising First Amendment implications.
- What safeguards has Flock Safety implemented to address misuse of its police surveillance tools?
- Flock Safety has introduced several reforms: shortening default data retention, requiring case codes for all searches, and implementing automated auditing to flag suspicious activity. The system also includes an AI model to moderate search queries, blocking some sensitive categories like race or religion. However, it only warns for political expression, allowing officers to proceed and noting the action for administrators.