Negotiating Risk Boundaries in AI for Policing Through Mixed-Stakeholder Deliberation

arXiv:2608.05418 · cs.AI · Submitted 2026-08-05 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Next we'll be talking about the paper "Negotiating Risk Boundaries in AI for Policing Through Mixed-Stakeholder Deliberation".

Jane: The paper was written by Mackenzie Jorgensen, Jo Reilly, Alex Sutherland and Miri Zilka from Northumbria University and University of Cambridge and University of Oxford.

Tom: Stay tuned as we take you through the paper and discuss its implications.

Title: Tom: We're talking about Negotiating Risk Boundaries in AI for Policing Through Mixed-Stakeholder Deliberation.

Jane: It's quite a long title, but Mackenzie Jorgensen and her team are really asking how we decide if police tech is actually safe for everyone.

Tom: I think you hit the nail on the head, Jane, because they aren't just looking at whether the software runs smoothly or has high accuracy.

Jane: Right, they're looking at whether it's actually fair when it hits real-world communities.

Tom: And they didn't just stick to talking with computer scientists to get those answers.

Jane: That was the most interesting part for me, because they brought in community members alongside police officers and academics.

Tom: It's a much more holistic way of looking at things, isn't it?

Jane: It really is, since it includes people who have lived experience with how policing works on the ground.

Lu: That kind of collaboration is exactly what we need to avoid building tools that just automate old prejudices!

Meng: I do wonder if it's actually possible to get all those different groups to agree on anything during a workshop.

Lu: The disagreement is actually the most useful part because it forces us to think about things we usually ignore.

Meng: If we can take those intense debates and turn them into actual design requirements, it would save so much trouble during deployment.

Lalam: Moving toward that kind of collective decision-making helps build a culture where technology supports human dignity instead of just acting as a surveillance tool.

Tom: It's a massive shift in how we think about technological progress.

Jane: And that leads us to the actual results they found from those discussions.

Summary: Jane: They evaluated thirteen different use cases using a color-coded system for risk levels.

Tom: Most people were actually fine with things like AI helping with administrative reports or managing data.

Jane: But they were very firm about rejecting tools that predict if someone might reoffend.

Tom: That's because the objection wasn't even about the technology itself, but about the idea of predicting crime in that way.

Jane: It sounds like they were questioning the very premise of those risk assessment models.

Meng: That makes sense since those models often rely on arrest data, which is already skewed by racial bias.

Lu: It creates this feedback loop where you just end up over-policing the same neighborhoods forever!

Meng: If we change what we're asking the software to do, we could avoid that whole cycle.

Lu: Exactly, like using data to see where people need more social support or better housing instead of just more patrols.

Lalam: That kind of thinking turns a tool of suspicion into something that actually builds community trust.

Tom: It's such a different way to look at what "predictive" means in this context.

Jane: And the researchers found that focusing on these equity risks actually made the whole conversation better.

Improvements: Tom: They found that foregrounding racial bias doesn't narrow the discussion, it actually makes it more thorough.

Jane: They use a concept called the "curb-cut effect" to explain why this happens.

Tom: You know, like those sidewalk ramps that were made for wheelchairs but ended up helping people with strollers too?

Jane: That's a perfect analogy for what they're seeing in these workshops!

Lu: It's the same for AI design because if we build systems that are incredibly robust for the most vulnerable, they become better for everyone.

Meng: I see how that works, but how do we translate those social values into something a developer can actually use?

Lu: We could use these mixed-stakeholder workshops to set our testing parameters before we even write the first line of code.

Meng: That would definitely make our deployment pipelines much more resilient to real-world complications.

Lalam: Prioritizing those inclusive principles helps us move toward a digital culture that reinforces human rights by default.

Tom: It sounds like they want ethics to be the foundation of the engineering process rather than just an afterthought.

Jane: And that brings us to our final wrap-up for today.

Conclusion: Tom: We've spent a lot of time with Negotiating Risk Boundaries in AI for Policing Through Mixed-Stakeholder Deliberation today.

Jane: It really shows that we can't just leave these big decisions to the technical teams if we want them to work for everyone.

Tom: I think the biggest lesson is that being inclusive actually helps us build more reliable products.

Jane: It forces us to ask if a tool is even worth making before we start the whole development process.

Tom: Let's hear from the rest of the team one last time before we sign off. Lu, what's your final thought?

Lu: I am so inspired by how this model could be applied to other high-stakes areas like medicine!

Meng: And I'll be thinking about how to turn these qualitative values into concrete technical specifications for my next project.

Lalam: My vision is a future where technology is built alongside people to support our shared humanity.

Jane: That is such a beautiful way to end the show, Lalam.

Tom: Thanks for joining us, everyone; we'll see you next time!

Northumbria University · University of Cambridge · University of Oxford

cs.AI

Submitted: 2026-08-05

Updated: 2026-09-12

Comments: Accepted to AIES 2026

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 80/100

The gist: This paper presents results from a "mixed-stakeholder deliberation workshop" designed to assess the risks of AI adoption in policing.

Key concepts

Mixed-stakeholder deliberation: This approach involves bringing together diverse groups, including community members with lived experience, police officers, and academics, to discuss AI use cases. These workshops help identify equity risks and translate social values into design requirements or testing parameters before technology is developed or deployed.
Curb-cut effect: This concept suggests that designing systems to be robust for the most vulnerable populations ultimately benefits everyone. Much like sidewalk ramps designed for wheelchairs also help people with strollers, building AI that addresses specific equity risks creates more reliable and effective technology for all users.
Risk assessment models: These are tools used to predict if an individual might reoffend. The researchers found these are often rejected because they rely on arrest data, which can be skewed by racial bias, potentially creating feedback loops that lead to the continuous over-policing of certain neighborhoods.

Terminology

Accountability — 17× in this paper · explained in 2 episodes
Accuracy — 10× in this paper · explained in 1 episode
Artificial Intelligence — 7× in this paper
Live Facial Recognition — 7× in this paper
Core — 5× in this paper · explained in 2 episodes
Police Race Action Plan — 5× in this paper
Actor — 3× in this paper · explained in 3 episodes
Verification — 3× in this paper · explained in 2 episodes
Adam — 2× in this paper · explained in 2 episodes
Algorithmic Transparency Recording Standard — 2× in this paper
Artificial Intelligence (AI) — 2× in this paper · explained in 2 episodes
Algorithmic Bias — 1× in this paper · explained in 1 episode
Consistency — 1× in this paper · explained in 3 episodes
Cooperation — 1× in this paper · explained in 2 episodes
Credit assignment — 1× in this paper · explained in 3 episodes
Hallucination — 1× in this paper · explained in 4 episodes
Human-in-the-Loop — 1× in this paper · explained in 3 episodes
Hybrid System — 1× in this paper · explained in 2 episodes

Summary

This paper presents results from a mixed-stakeholder deliberation workshop designed to assess the risks of AI adoption in policing. It matters because while racial bias is a well-documented risk, representatives of affected communities are rarely included in decisions about AI adoption, and this research explores how to negotiate acceptable risk limits during the conception stage of technology deployment.

The Deliberation Framework

The study utilizes a mixed-stakeholder deliberation workshop involving 30 participants, including community representatives, police officers, and academics, to assess 13 AI use cases. The assessment was structured around four core themes derived from the UK’s Police Race Action Plan (PRAP):

Culture & Workforce
Powers & Procedures
Trust & Reconciliation
Safety & Victimisation

Participants evaluated each use case using a red, amber, and green risk framework to determine acceptable deployment. The risk classifications were defined as follows:

Green – low risk, can be deployed with minimal checks.
Amber – medium risk, deploy only in a specific context and with rigorous checks.
Red – unacceptable risk, do not deploy.

Findings on AI Use Cases

The analysis revealed that participants were broadly open to AI adoption, rejecting only three use cases outright. While many discussions focused on technical implementation, certain tools faced fundamental opposition. For example, objections to recidivism risk assessment targeted the premise rather than the implementation, with participants arguing that bias issues are baked into this.

Other key findings regarding specific technologies included:

Live Facial Recognition (LFR) was deemed acceptable only under strict conditions, such as limiting use to high-harm and/or sexual offenders.
Predictive policing (hotspot mapping) was viewed positively for resource deployment, provided it avoids feedback loops that compound over-policing in minority communities.
AI-assisted processing of incident reports was viewed as a helpful assistant, provided it avoids a false sense of efficiency and maintains accountability.

The Integrated Reasoning Process

A central finding is that foregrounding racial equity did not narrow the deliberation. Instead, it led to an integrated reasoning process where discussions gravitated toward fundamental questions about utility and fairness. This phenomenon is described as being reminiscent of the curb-cut effect in inclusive design, suggesting that designing with marginalized communities in mind produces better outcomes for everyone.

The researchers identified three backengineered questions that participants implicitly used to arrive at their risk classifications:

Does it work? This involved questioning if a tool is actually capable of doing what it claims and whether it assists or displaces human judgment.
Will it deliver genuine benefit? Participants scrutinized whether tools provide real efficiency or merely create a false sense of efficiency, questioning if the time saved in generation is offset by the time needed for verification.
Will it deliver benefit for everyone? This layer of scrutiny asked if benefits would be distributed equitably and if the technology would alleviate the current structural problems or worsen them.

Improvements for AI systems

1. Re-engineering Predictive Objectives from Recidivism to Support Needs

Improvement: Transition predictive modeling architectures from Risk of Re-arrest/Reoffending to Proactive Rehabilitation and Intervention Need Prediction.
Capability: Instead of generating a high-risk score that triggers punitive measures, the system identifies specific socio-economic stabilizers (e.g., housing requirements, employment gaps, or mental health service needs) required to prevent future harm, shifting the output from a classification of danger to a roadmap for support.

2. Implementation of Multi-Agency Data Triangulation

Improvement: Replace biased arrest-record proxies with integrated datasets from non-law enforcement sources (e.g., hospital admissions, ambulance/emergency medical services, and social service records).
Capability: The system identifies crime hotspots or victimisation risks by triangulating actual harm and medical emergencies rather than relying on historical arrest patterns, thereby breaking the feedback loop of over-policing in marginalized communities caused by biased arrest data.

3. Transition from Automated Decision-Making to Augmented Sensemaking Co-pilots

Improvement: Redesign AI interfaces from Automated Classification/Decision tools to Human-in-the-Loop Sensemaking Assistants.
Capability: In tasks like incident report analysis or investigation advice, the system provides summarized evidence, highlights potential cultural or contextual nuances that may be missed by humans, and suggests investigative directions, but requires explicit human verification and accountability for every output to prevent automation bias or the displacement of professional judgment.

4. Integration of Curb-Cut Equity Auditing in Design Pipelines

Improvement: Shift from a post-hoc Ethical Checklist approach to an integrated, participatory design framework that treats racial equity as a core performance metric during the ideation stage.
Capability: The system development lifecycle includes mandatory deliberation with marginalized community representatives to ensure the tool is optimized for accuracy and benefit across all demographics; this ensures that solving for the most vulnerable populations (e.g., detecting cultural nuances in victim statements) inherently improves the tool's robustness for the general population.

5. Conditional, High-Harm Restricted Biometric Deployment

Improvement: Implement hard-coded operational constraints and demographic parity triggers within Live Facial Recognition (LFR) systems.
Capability: The system is restricted to searching only specific high-harm/prolific offender watchlists (excluding minor offenses) and features an automated safety halt mechanism that disables the system if real-time monitoring detects a statistically significant divergence in error rates across different ethnic groups.

Abstract

AI tools are being increasingly adopted in policing in the UK and worldwide. Racial bias is a known and well-documented risk, yet representatives of affected communities are rarely included in decisions about AI adoption. We present results from a mixed-stakeholder deliberation workshop bringing together 30 community representatives, police officers, and academics to assess the risks of 13 AI use cases in policing, with an explicit focus on racial bias. We found that participants were broadly open to AI adoption, rejecting only three use cases outright -- most notably recidivism risk assessment, where objections targeted the premise rather than the implementation. Our analysis reveals that foregrounding racial equity did not narrow the deliberation. Instead, discussions gravitated toward a fundamental set of questions: does this tool actually work, will it deliver genuine benefit, and will that benefit extend to everyone? This integrated reasoning---reminiscent of the curb-cut effect in inclusive design---highlights the benefit of incorporating the racial bias lens into the risk-benefit analysis of AI use cases from the outset.