Co-constructing sociotechnical AI governance: participatory system mapping using algorithm registers

This paper investigates the role of municipal algorithm registers in providing transparency and facilitating accountability for algorithmic systems in public services, using a case study of a Dutch city's register and a decision-support tool called Avola used for welfare benefits eligibility. The authors ask two research questions: what do algorithm registers reveal (and occlude) about the sociotechnical systems governing algorithmic systems, and how can diverse stakeholder perspectives inform a more pluralistic system-theoretic safety analysis?

The study employs interviews, surveys, and participatory system mapping workshops with municipal staff, civil society organisations (CSOs), and municipal Ombudsmen (N=8). These stakeholders' contributions inform a System-Theoretic Process Analysis (STPA) that situates the register within a wider sociotechnical governance structure. The authors find that the algorithm register raises more questions than it answers for external actors, and that the information it reveals is too abstract to be meaningful for external actors. For example, the register declares that risk assessments like FRAIA and DPIA have been conducted but withholds details about who was involved and what they found, giving the public a false sense of security.

The participatory mapping workshops revealed that different stakeholder groups contributed distinct insights based on their positionality. CSO staff focused on technical aspects, governance instruments, and complaints channels, questioning who conducted risk assessments and whether they informed each other. Municipal staff contributed knowledge about political bodies governing the system, such as Aldermen and City Council. Ombudsman staff added the least to the maps, noting that our work is more about the human interest and this is more the systems [point of view], and that the citizen-caseworker interaction was the only human connection on the map.

The STPA analysis identified potential safety hazards that would not have been visible using the algorithm register alone, including benefits eligibility denial, system performance deterioration, and inability to contest wrongful decisions. The authors present four loss scenarios spanning the human-in-the-loop, the complaints procedure, the Ombudsman's oversight mandate, and political pressure and the public sphere. For instance, the register's assertion of caseworker discretion may prove beyond the scope of the register's capacity and may hide issues such as algorithmic bias, opacity, and organisational pressures. The FRAIA documentation was found to be limited, with one answer stating that errors happen, but it is so marginal that no number can be attached to it, preventing external observers from mounting credible legal cases.

The authors conclude that algorithm registers are an important step but need to be better aligned with the information needs of different publics. They encourage greater inclusion of expert communities such as CSOs, Ombudsmen, independent researchers, and journalists, while cautioning external actors to be mindful of public organisations' resource challenges. They also reflect on the normative dimensions of algorithm governance efforts, noting that mapping and documentation practices are inherently political and raising the critical question of who has a say in what gets documented. The paper contributes an empirical assessment of a municipal algorithm register, a participatory schema for mapping algorithmic systems, and reflections on the political dimensions of system safety analysis.

Improvements to AI Systems:

1. Context-Aware Transparency Generation: AI systems can be enhanced to generate algorithm register entries that are audience-specific, automatically translating abstract compliance data (e.g., DPIA conducted) into plain-language, actionable summaries for external actors—including who performed the assessment, what specific risks were found, and how they were mitigated—rather than binary yes/no declarations.

2. Occlusion Detection and Gap-Filling: AI can be trained to detect information occlusions in existing registers by cross-referencing documented claims (e.g., caseworker discretion exists) against actual system logs, policy documents, and complaint data, flagging discrepancies where the register's abstraction hides operational realities like algorithmic bias or organizational pressure.

3. Pluralistic Stakeholder Simulation: AI systems can be improved to simulate diverse stakeholder perspectives (e.g., CSO technical experts, municipal staff, Ombudsmen) during system safety analysis, generating synthetic positionality-aware inputs that highlight different risk facets—such as technical failure modes, political oversight gaps, or human-interaction breakdowns—that a single-view analysis would miss.

4. Safety Hazard Inference from Register Data: AI can be enhanced to automatically infer latent safety hazards from register content alone, using patterns from this study (e.g., FRAIA with no error quantification → hazard of unverifiable legal claims; human-in-the-loop asserted → hazard of rubber-stamping) to produce a prioritized list of potential loss scenarios for further investigation.

5. Participatory Mapping Co-Pilot: AI can be developed to facilitate participatory system mapping workshops in real-time, prompting stakeholders with positionality-specific questions (e.g., As an Ombudsman, what human-interest touchpoints are missing?) and automatically integrating their inputs into a live STPA hazard model, reducing the manual burden observed in this study.

6. Political Sensitivity Flagging: AI systems can be improved to detect and flag the political dimensions of documentation choices—such as who decided what to include or exclude in a register—by analyzing metadata, revision histories, and stakeholder feedback, and then alerting oversight bodies to potential power asymmetries in the governance process.

What the Improved AI System Can Do:

• For Municipalities: Automatically draft register entries that meet both legal compliance and public comprehensibility, while proactively identifying and disclosing gaps in risk assessment documentation.

• For Civil Society Organizations: Scan municipal registers and generate targeted, evidence-based challenge letters or legal case files by extracting missing details (e.g., who ran the FRAIA, what error rates were found) and comparing them against public records.

• For Ombudsmen and Auditors: Receive AI-generated blind spot reports that highlight where system maps lack human-interest perspectives, enabling more focused investigations into citizen-caseworker interactions.

• For Researchers and Journalists: Query an AI that reconstructs the full sociotechnical governance structure from fragmented register data, including political bodies, complaints channels, and informal decision points, and simulates what-if scenarios for policy changes.

Algorithm registers have been championed as a means of providing transparency on the use of algorithms in public services. Yet potential publics differ in their expectations of what should be made transparent and how, as well as in their interest in and ability to parse the information currently published in the registers. Moreover, it remains unclear how these instruments can represent the sociotechnical systems in which these algorithms are embedded, and how system-level transparency can facilitate accountability. In this paper, we ask, what do algorithm registers reveal (and occlude) about the sociotechnical systems governing algorithmic systems, and how can diverse stakeholder perspectives inform a more pluralistic system-theoretic safety analysis? To do this, we probe the municipal algorithm register of a Dutch city through a case study of a decision-support tool for caseworkers' assessment of citizens' welfare benefits eligibility based on legal automation through a business rule engine. Through interviews, surveys, and participatory system mapping workshops (with municipal staff, civil society organisations, and ombudsmen, N=8), we seek to understand to what extent the register allows stakeholders to map the algorithmic system in question. These maps inform a System-Theoretic Process Analysis (STPA) that situates the register within a wider sociotechnical governance structure. Participants' contributions allow us to identify potential safety hazards which would not have been possible to see using the algorithm register alone, including benefits eligibility denial, system performance deterioration, and inability to contest wrongful decisions. By engaging both direct and indirect stakeholders, we reflect on the normative dimensions of algorithm governance efforts and how politics shape the practice of system safety analysis.