British Police Developed an Extensive Crime Forecasting System, but Some Outcomes Were Unreliable.
In response to WIRED’s public records inquiries, Avon and Somerset Police released extensive performance data for 13 risk models applied between 2017 and 2024—covering predictions related to missing persons, antisocial behavior, and potential offenders or victims of crime. This data, alongside additional context regarding the police’s data science initiatives, was given to the independent auditing firm Eticas for assessment. The findings were alarming.
“Most of these models exhibit low precision scores, indicating that many individuals flagged as risks are misidentified,” concluded the data review. A model designed to forecast burglary incidents had a precision rating below 10% for over three years, as per police data. As detailed by Eticas, this implied fewer than one in ten identified as high risk would actually commit an offense. Additional concerns were raised regarding fluctuating performance metrics for various models. “This is not typical behavior for well-managed models in operational use,” remarked the audit.
A representative from Avon and Somerset Police informed WIRED that the division opted not to implement certain models, including the burglary-related one. When questioned about the existence of years’ worth of audit data for models not in use, the representative stated the audit process was “automated” and that data was drawn from a “static file that wasn’t deleted after the decision not to deploy the model.”
The police force declined requests for interviews regarding its data science initiatives and did not completely address a detailed series of questions. “Every model is evaluated based on its performance, and if issues are identified, they will be updated or disabled,” the Avon and Somerset Police spokesperson mentioned in a statement, adding that each model undergoes review by a subject matter expert before deployment.
It remains unclear what actions Avon and Somerset Police undertook to mitigate the risks highlighted by its ethics committee in the initial stages of its data science efforts. According to disclosure records, the committee seemingly did not address predictive analytics after 2017. While Avon and Somerset Police claims that “each project and product” within its data science operations is evaluated by a dedicated ethics group, the spokesperson informed WIRED that “no meeting has yet occurred,” as “no model has been created that presents potential ethical dilemmas.”
In response to a public records request, Avon and Somerset Police provided a screenshot of a “bias check app” purportedly monitoring and comparing average risk scores between white individuals and people of color, concluding that there was “no significant disparity between the two.” The Eticas review stated: “Simply tracking ethnicity as a monitoring variable does not equate to examining whether the model results in discriminatory effects,” noting the lack of thorough testing across ethnicity, gender, and socioeconomic conditions as “a considerable oversight.”
When asked whether he believes predictive analytics has a role in policing or social work, Davies expressed that more effort is necessary. “In our attempts, we were aiming to accomplish this for the right reasons and in the right manner, yet we lacked the capacity that it truly required.” He believes part of the work should aim to enable workers to utilize risk models without leading them to predetermined conclusions. “There is a danger that staff may rely solely on computer outputs and not exercise their own judgment.”
Predictive analytics continues to have a significant influence on policing and public services in the region. Bristol City Council still employs a risk-scoring model to evaluate the likelihood of children disengaging from education, employment, or training. According to the latest audit data from Avon and Somerset Police, released in July last year, the model utilized by the Offender Management App accurately predicts only one in three individuals who actually commit offenses, while one in four individuals flagged as likely offenders do not.
Rajat Sharma
https://in.linkedin.com/in/rajat-mediaHelping D2C Brands Scale with AI-Powered Marketing & Automation 🚀 | $15M+ in Client Revenue | Meta Ads Expert | D2C Performance Marketing Consultant