Negotiating Risk Boundaries in AI for Policing Through Mixed-Stakeholder Deliberation

summary

Video file (mp4)

The gist

This paper presents results from a "mixed-stakeholder deliberation workshop" designed to assess the risks of AI adoption in policing.

In short

This episode discusses a paper exploring how to determine if policing AI is safe and fair through mixed-stakeholder deliberation involving community members, police, and academics. Researchers found that while administrative AI is acceptable, tools predicting reoffending are rejected due to racial bias. Inclusivity helps build more reliable, ethical technology.

Key concepts

Mixed-stakeholder deliberation
This approach involves bringing together diverse groups, including community members with lived experience, police officers, and academics, to discuss AI use cases. These workshops help identify equity risks and translate social values into design requirements or testing parameters before technology is developed or deployed.
Curb-cut effect
This concept suggests that designing systems to be robust for the most vulnerable populations ultimately benefits everyone. Much like sidewalk ramps designed for wheelchairs also help people with strollers, building AI that addresses specific equity risks creates more reliable and effective technology for all users.
Risk assessment models
These are tools used to predict if an individual might reoffend. The researchers found these are often rejected because they rely on arrest data, which can be skewed by racial bias, potentially creating feedback loops that lead to the continuous over-policing of certain neighborhoods.

Terminology used across episodes

This episode discusses

The paper

Negotiating Risk Boundaries in AI for Policing Through Mixed-Stakeholder Deliberation · Read on arXiv

Northumbria University · University of Cambridge · University of Oxford

AI tools are being increasingly adopted in policing in the UK and worldwide. Racial bias is a known and well-documented risk, yet representatives of affected communities are rarely included in decisions about AI adoption. We present results from a mixed-stakeholder deliberation workshop bringing together 30 community representatives, police officers, and academics to assess the risks of 13 AI use cases in policing, with an explicit focus on racial bias. We found that participants were broadly open to AI adoption, rejecting only three use cases outright -- most notably recidivism risk assessment, where objections targeted the premise rather than the implementation. Our analysis reveals that foregrounding racial equity did not narrow the deliberation. Instead, discussions gravitated toward a fundamental set of questions: does this tool actually work, will it deliver genuine benefit, and will that benefit extend to everyone? This integrated reasoning---reminiscent of the curb-cut effect in inclusive design---highlights the benefit of incorporating the racial bias lens into the risk-benefit analysis of AI use cases from the outset.

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Next we'll be talking about the paper "Negotiating Risk Boundaries in AI for Policing Through Mixed-Stakeholder Deliberation".

Jane: The paper was written by Mackenzie Jorgensen, Jo Reilly, Alex Sutherland and Miri Zilka from Northumbria University and University of Cambridge and University of Oxford.

Tom: Stay tuned as we take you through the paper and discuss its implications.

Title: Tom: We're talking about Negotiating Risk Boundaries in AI for Policing Through Mixed-Stakeholder Deliberation.

Jane: It's quite a long title, but Mackenzie Jorgensen and her team are really asking how we decide if police tech is actually safe for everyone.

Tom: I think you hit the nail on the head, Jane, because they aren't just looking at whether the software runs smoothly or has high accuracy.

Jane: Right, they're looking at whether it's actually fair when it hits real-world communities.

Tom: And they didn't just stick to talking with computer scientists to get those answers.

Jane: That was the most interesting part for me, because they brought in community members alongside police officers and academics.

Tom: It's a much more holistic way of looking at things, isn't it?

Jane: It really is, since it includes people who have lived experience with how policing works on the ground.

Lu: That kind of collaboration is exactly what we need to avoid building tools that just automate old prejudices!

Meng: I do wonder if it's actually possible to get all those different groups to agree on anything during a workshop.

Lu: The disagreement is actually the most useful part because it forces us to think about things we usually ignore.

Meng: If we can take those intense debates and turn them into actual design requirements, it would save so much trouble during deployment.

Lalam: Moving toward that kind of collective decision-making helps build a culture where technology supports human dignity instead of just acting as a surveillance tool.

Tom: It's a massive shift in how we think about technological progress.

Jane: And that leads us to the actual results they found from those discussions.

Summary: Jane: They evaluated thirteen different use cases using a color-coded system for risk levels.

Tom: Most people were actually fine with things like AI helping with administrative reports or managing data.

Jane: But they were very firm about rejecting tools that predict if someone might reoffend.

Tom: That's because the objection wasn't even about the technology itself, but about the idea of predicting crime in that way.

Jane: It sounds like they were questioning the very premise of those risk assessment models.

Meng: That makes sense since those models often rely on arrest data, which is already skewed by racial bias.

Lu: It creates this feedback loop where you just end up over-policing the same neighborhoods forever!

Meng: If we change what we're asking the software to do, we could avoid that whole cycle.

Lu: Exactly, like using data to see where people need more social support or better housing instead of just more patrols.

Lalam: That kind of thinking turns a tool of suspicion into something that actually builds community trust.

Tom: It's such a different way to look at what "predictive" means in this context.

Jane: And the researchers found that focusing on these equity risks actually made the whole conversation better.

Improvements: Tom: They found that foregrounding racial bias doesn't narrow the discussion, it actually makes it more thorough.

Jane: They use a concept called the "curb-cut effect" to explain why this happens.

Tom: You know, like those sidewalk ramps that were made for wheelchairs but ended up helping people with strollers too?

Jane: That's a perfect analogy for what they're seeing in these workshops!

Lu: It's the same for AI design because if we build systems that are incredibly robust for the most vulnerable, they become better for everyone.

Meng: I see how that works, but how do we translate those social values into something a developer can actually use?

Lu: We could use these mixed-stakeholder workshops to set our testing parameters before we even write the first line of code.

Meng: That would definitely make our deployment pipelines much more resilient to real-world complications.

Lalam: Prioritizing those inclusive principles helps us move toward a digital culture that reinforces human rights by default.

Tom: It sounds like they want ethics to be the foundation of the engineering process rather than just an afterthought.

Jane: And that brings us to our final wrap-up for today.

Conclusion: Tom: We've spent a lot of time with Negotiating Risk Boundaries in AI for Policing Through Mixed-Stakeholder Deliberation today.

Jane: It really shows that we can't just leave these big decisions to the technical teams if we want them to work for everyone.

Tom: I think the biggest lesson is that being inclusive actually helps us build more reliable products.

Jane: It forces us to ask if a tool is even worth making before we start the whole development process.

Tom: Let's hear from the rest of the team one last time before we sign off. Lu, what's your final thought?

Lu: I am so inspired by how this model could be applied to other high-stakes areas like medicine!

Meng: And I'll be thinking about how to turn these qualitative values into concrete technical specifications for my next project.

Lalam: My vision is a future where technology is built alongside people to support our shared humanity.

Jane: That is such a beautiful way to end the show, Lalam.

Tom: Thanks for joining us, everyone; we'll see you next time!

More episodes

← Home