Short paper: Models in the dark -- Rectification and erasure under GDPR in ML supply chains

arXiv:2606.05946 · cs.LG · Submitted 2026-06-04 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Next we'll be talking about the paper "Short paper: Models in the dark -- Rectification and erasure under GDPR in ML supply chains".

Jane: The paper was written by Henrik Graßhoff, Malte Hansen, Meiko Jensen and Sara Ramezanian from Karlstad University, Karlstad, Sweden..

Tom: Stay tuned as we take you through the paper and discuss its implications.

Introduction to Models in the Dark: Tom: We’ve been reading a fascinating paper titled "Short paper: Models in the dark – Rectification and erasure under GDPR in ML supply chains," and it really sets a high bar for how we should be thinking about data privacy.

Jane: It’s clear right from the the title that this isn't just another legal discussion; it's about understanding the entire lifecycle of machine learning systems, which is huge for us.

Lu: The authors are framing this problem by looking at complex supply chains, not just in isolation, and that perspective is crucial because AI development is never a single event.

Meng: I’m particularly interested in how this relates to the various actors involved; it suggests that our current regulatory frameworks are struggling to keep up with distributed development.

Lalam: The concept of "models in the dark" immediately tells us that when a simple end-user sees an AI, they may have no idea how many layers of other models built their capabilities, which is a massive cultural shift we need to address.

Tom: But as we move beyond just that understanding, what do you think the core problem is in implementation?

Jane: The paper shows that the existing methods are not quite equipped to handle these specific rights because traditional concepts of remembering and forgetting don't apply cleanly to complex statistics learned by algorithms.

Lu: It’s not enough for a simple deletion; we have to deal with the fact that data can be implicitly encoded in parameters, which makes simple solutions ineffective against this complexity.

Meng: That brings up a massive engineering challenge: if the data is baked into the parameters, how do we even locate it when trying to fulfill a legal request?

Lalam: Finding that achieving compliance is incredibly difficult means we have to think about rethinking our entire approach to building and deploying these systems.

Summary of Implementation Challenges: Tom: The summary section really lays out the technical hurdles, and it’s easy to see why satisfying data subject rights becomes such a complicated task in the real world.

Jane: It’s not just that we haven't tried hard enough; the paper suggests that many GDPR requirements simply cannot be technically met given current state of mind regarding how machine learning functions.

Lu: The findings highlight a gap between this legal requirement and the technical reality, which is a major red flag for anyone overseeing these systems.

Meng: From an engineering view, it's concerning to see such a lack of practical solutions; we need methods that can scale with these complex supply chains without breaking under pressure.

Lalam: It’s not just about ticking boxes; it’s about ensuring that the entire system is trustworthy and ethically sound in every interaction, which requires more than just quick fixes.

Tom: That leads us to look at the specific challenges—the model-intrinsic issues versus the supply chain problems.

Jane: The paper divides these issues into two distinct types, which helps us see where we need specialized solutions for each area of impact.

Lu: Intrinsic problems—like determining if a specific data point is actually stored in the model's learned parameters—are fundamentally different from the dealing with the vast network of actors involved.

Meng: The difficulty quantifying that influence makes me wonder how much effort we are really asking controllers to put into audits that might not be feasible at scale across multiple parts of an AI system.

Lalam: This forces us to confront the idea that our technology isn't just a simple black box; it’s a complex ecosystem with inherent vulnerabilities that we must address for ethical reasons.

Tom: It feels like we are at a critical juncture where the technical limitations are meeting the legal demands.

Suggested Improvements and Solutions: Tom: Even though the challenges are significant, there's hope in the suggested improvements, which is a bit of a relief to see that there might be paths forward for compliance.

Jane: The paper suggests an interdisciplinary approach to bridge the gap between legal demands and actual technical implementation within machine learning systems.

Lu: I like that focus; it implies this isn't just a purely technical hurdle but an organizational and legal one too, forcing different fields to collaborate on a solution.

Meng: From a practical standpoint, this suggests we need standardized protocols for model updates across the supply chain to make those rights enforceable in a real-world setting.

Lalam: And I hope these improvements lead towards creating AI that feels fundamentally responsible in how it handles human data, moving beyond just compliance toward trust.

Tom: That brings up the idea of propagating an updated version of the model through the system, which is one way to address those issues.

Jane: It’s important to remember that this update strategies vary based on whether or not distributing a model involves third parties at all, depending on legal obligations.

Lu: The authors point out that if we are dealing with an immense number of derivatives in the supply chain, a simple update might not be enough because of how cascading effects work.

Meng: We also need to consider the practical burden of informing recipients; distributing multi-gigabyte model updates sounds like a massive network traffic problem.

Lalam: It’s not just about sending data; it’s about ensuring that the entire system is updated consistently so that our cultural integrity isn't compromised by outdated or uncorrected information.

Tom: So, how do we make sure those changes actually reach every part of every model?

Conclusion and Final Thoughts: Tom: We’ve covered a lot of ground, from the "models in the dark" to the practical difficulties in implementing rectification and erasure under GDPR.

Jane: It seems like a fair assessment; that it is not yet feasible to implement these rights comprehensively across both technical and systemic levels.

Lu: I think the future lies in finding ways to make this complex, perhaps even through developing decentralized governance structures for these systems.

Meng: I’m looking forward to seeing what practical solutions emerge for managing model updates across all relevant actors, because that is where the real work needs to happen.

Lalam: We need a future where AI is not just powerful, but also fundamentally responsible and respects our personal data at every step of every interaction.

Tom: It’s clear that the paper "Short paper: Models in the dark – Rectification and erasure under GDPR in ML supply chains" leaves a lot of open issues for research.

Jane: We’ve seen that this is a problem that requires both deep technical knowledge and real-world policy changes, which is a powerful message.

Lu: The authors have really highlighted the complexity of how many different types of models exist downstream, making it impossible to ignore the cascading effects.

Meng: I think we need to keep pushing for standardized procedures so that the actual flow of data is as clear as possible for accountability.

Lalam: As we wrap up this discussion, we hope that all striving towards building AI that feels fundamentally trustworthy and respects our cultural values in every interaction.

Karlstad University, Karlstad, Sweden.

cs.LG

Submitted: 2026-06-04

Updated: 2026-06-04

Comments: accepted for presentation at Annual Privacy Forum 2026

License: http://creativecommons.org/licenses/by-nc-nd/4.0/

Importance score: 83/100

The gist: The paper presents a holistic survey of challenges in implementing the rights to rectification and erasure under the General Data Protection Regulation (GDPR) within machine learning systems,

Key concepts

Models in the Dark: This concept describes a situation where an end-user interacting with an AI system has no visibility into how many layers of other underlying models contributed to its final capabilities. This lack transparency requires a significant cultural shift in how we view the entire AI development process.
ML Supply Chains: AI development is not a single event; it involves a complex, distributed network of various actors and stages. The supply chain perspective is crucial because current regulatory frameworks are struggling to keep up with this multi-layered and interconnected process.
Rectification and Erasure (GDPR): These are legal rights allowing data subjects to have their personal data corrected or forgotten. The discussion highlights that traditional methods of 'remembering' and 'forgetting' do not apply cleanly to the complex statistical patterns learned by algorithms.

Terminology

Actor — 22× in this paper · explained in 3 episodes
Machine Unlearning — 15× in this paper · explained in 2 episodes
Accountability — 7× in this paper · explained in 2 episodes
General Data Protection Regulation — 5× in this paper
European Data Protection Board — 3× in this paper
Federated Learning — 3× in this paper · explained in 14 episodes
Hallucination — 3× in this paper · explained in 3 episodes
Security and Privacy — 3× in this paper
Accuracy — 2× in this paper · explained in 1 episode
Adaptation — 2× in this paper · explained in 1 episode
Core — 2× in this paper · explained in 2 episodes
Diffusion Model — 2× in this paper · explained in 8 episodes
Diffusion Models — 2× in this paper · explained in 6 episodes
Generative Model — 2× in this paper · explained in 2 episodes
Approximate Unlearning — 1× in this paper · explained in 1 episode
Artificial Intelligence (AI) — 1× in this paper · explained in 2 episodes
Artificial Intelligence Act — 1× in this paper
Continual Learning — 1× in this paper · explained in 2 episodes
Differential Privacy — 1× in this paper · explained in 4 episodes
Differentially Private — 1× in this paper · explained in 3 episodes
Generative AI — 1× in this paper · explained in 2 episodes
Generative Models — 1× in this paper · explained in 2 episodes
Robustness — 1× in this paper · explained in 5 episodes
Verifiability — 1× in this paper · explained in 2 episodes
Verification — 1× in this paper · explained in 2 episodes

Summary

The paper presents a holistic survey of challenges in implementing the rights to rectification and erasure under the General Data Protection Regulation (GDPR) within machine learning systems, specifically addressing issues arising from complex ML supply chains.

Problem Statement and Scope:

The core issue is that existing work has "largely addressed these rights from either a legal or a technical perspective in isolation and disregards the fact that models are produced in complex supply chains involving multiple actors across development, distribution, and deployment." The paper aims to bridge this gap by investigating challenges related to the GDPR’s right to rectification (RTR) and right to erasure (RTE).

The ML Supply Chain Context:

The paper first defines the relevant data flows in an ML supply chain. This process involves several actors and stages, including:

Data collection: Raw data is collected from diverse sources.
Preprocessing: Data sanitization techniques may be used to reduce personal data presence.
Data analysis/Model training: The prepared data is used to train the ML model, typically performed by a single Model developer, though this can be outsourced or collaborative (e).
Distribution: The resulting ML model is delivered from the Model developer to the Model distributor and finally offered as an ML service by the Model host to a Model user.

The paper notes that an ML supply chain thus leads to a multitude of data objects held by different controllers, and that DSR requests can be filed against both data used for model training or adaptation or against the models themselves.

Model-Intrinsic Challenges:

Section 3 details challenges inherent to the ML models themselves:

Necessity to assess models, not training data: A key insight is that a model is not equivalent to its training data. Any assessment of what a model has stored must be based on the model itself rather than its training data.
Identification of the data subject within a model: Quantifying the influence of a specific training data point on a model remains an open problem, making unambiguous and complete identification of data subjects difficult, if not impossible. The controller must verify stored data on a case-by case basis rather than relying solely on training data.
Technical methods for rectification and erasure:
Fully retraining a model is the most GDPR-compliant way but is very resource-intensive.
Less resource-intensive approaches involve unlearning a specific subset of data. However, these methods introduce issues: Unlearnt models typically perform worse than retrained ones, with performance potentially deteriorating exponentially, leading to catastrophic unlearning.
Inference-time interventions: These modify input/output during interactions but are acknowledged that they would not satisfy definitions of unlearning since the data is not technically rectified or erased from the model.
Continually learning models: Models may learn continually, creating a ramified directed graph of continuously updated models, each of which depends on its predecessors and therefore, implicitly, on all previous training data.

Supply-Chain Challenges:

Section 4 introduces the concept of complexity arising from the network:

The notion of models in the dark: ML models are rarely used in isolation; they are the result of a larger supply chain which involves multiple actors... creating a complex network of subsequent downstream derivatives. This phenomenon is termed models in the dark, referring to every derivative model built from an upstream one.
Locating controllers under supply chain opacity: Data subjects may lack information about whether and how their data is present in an ML model. If the controller is far upstream, personal data may have been propagated to an unknown number of downstream models in the dark.
Updating models along the supply chain: Addressing models in the dark requires propagating an updated version of the model. This process involves legal obligations (e.g., informing recipients under Art. 19 GDPR) and practical difficulties: Bandwidth constraints, limited enforcement, and the rise of open-weight models remain obstacles.

Conclusion:

The paper concludes that both domains—model-intrinsic challenges and supply-chain challenges—contain significant hurdles that make the effective enforcement of RTR and RTE infeasible. The study provides a taxonomy of these challenges and identifies key limitations in current technical approaches. Future work is planned to evaluate unlearning techniques, minimize network traffic during model distribution, and investigate communication flows for GDPR-compliant data sharing.

Improvements for AI systems

*(The following analysis synthesizes the critical requirements derived from GDPR, the AI Act, and advanced research into Machine Unlearning/Privacy Attacks. The proposed improvements focus on creating a legally auditable, privacy-preserving lifecycle for Large Language Models (LLMs).)

We must move beyond standard model fine-tuning and adopt a multi-layered, governance-aware architecture. The core improvements integrate advanced unlearning mechanisms with formal differential privacy guarantees to ensure compliance with the Right to Erasure (GDPR) while maintaining functional integrity.

Mechanism: Integrate a dedicated, verifiable module that executes Variational Bayesian Unlearning techniques (as proposed by Nguyen et al.) directly into the model’s inference pipeline, rather than relying solely on pre-training dataset scrubbing. This layer must be callable via an explicit API endpoint (/api/unlearn).
Implementation Detail: The HUL does not just forget data; it generates a Proof of Unlearning—a mathematically verifiable report detailing the parameters and weights that have been adjusted or isolated to nullify the influence of the specified data points (D erased).
Improved Capability: The system can achieve Guaranteed Erasure Compliance. If a user invokes an unlearning request, the system provides immediate API confirmation and a corresponding audit trail proving that the influence of their personal data (D user) has been mathematically neutralized across all model layers, satisfying stringent legal requirements like GDPR's Right to Be Forgotten.
Mechanism: Implement two distinct levels of privacy protection:

Training/Fine-Tuning Layer (Input Privacy): Use Differential Privacy during all fine-tuning stages (e.g., implementing the methodology seen in VaultGemma). This ensures that the model parameters cannot be reverse-engineered to reveal specific training examples, mitigating potential privacy attacks (Rigaki & Garcia).
Inference Layer (Output Privacy): Apply a constrained differential privacy mechanism to the model's output logits when processing sensitive queries. This prevents the LLM from leaking sensitive training information or generating patterns that correlate too closely with protected classes, even if the core model is compromised.

Implementation Detail: The system must accept a configurable epsilon (epsilon) budget parameter for both layers, allowing the deploying organization to precisely balance privacy risk against model utility according to jurisdictional legal requirements.
Improved Capability: The system achieves Resilient Privacy. It can operate with high sensitivity on protected data while providing quantifiable, mathematically rigorous proof (epsilon) that the output cannot be linked back to a specific training record or individual, satisfying both regulatory and security mandates.
Mechanism: Overhaul the standard Continual Learning (CL) loop. Instead of simply adapting weights to new data (D new), every adaptation cycle must first pass through a formal Data Governance Gate. This gate checks D new against three criteria: 1) Existing unlearning obligations, 2) Differential Privacy budget exhaustion, and 3) Compliance with the latest AI Act restrictions.
Implementation Detail: The model maintains a segmented memory structure. New knowledge is stored in an ephemeral, verifiable Knowledge Patch rather than overwriting core weights immediately. Only after the Knowledge Patch passes all governance checks (including a simulated unlearning test against old data points) is it merged into the main model weights.
Improved Capability: The system achieves Legally Compliant Adaptability. The model can evolve and learn new skills over years (Continual Learning) without violating prior privacy commitments or losing critical, regulated knowledge. It provides an auditable history of why and how a specific piece of knowledge was integrated, drastically reducing legal liability associated with model drift.

Abstract

The rights to rectification and erasure, as established under the General Data Protection Regulation (GDPR), are central to protecting individuals' privacy. However, their effective enforcement in machine learning (ML) systems remains challenging. Existing work has largely addressed these rights from either a legal or a technical perspective in isolation and disregards the fact that models are produced in complex supply chains involving multiple actors across development, distribution, and deployment. This paper presents a holistic survey of challenges in implementing the rights to rectification and erasure in ML models. Drawing on academic literature and guidance from data protection authorities, we find that many GDPR requirements cannot yet be technically met in practice. Our findings further suggest that issues arising in ML supply chains are insufficiently addressed in research. To tackle this gap, we introduce the notion of models in the dark -- derived models created further downstream in an ML chain without sufficient transparency or traceability -- and analyse the urgent challenges posed by this phenomenon. By adopting an interdisciplinary perspective, this work contributes to bridging the gap between legal requirements and the technical implementation of data subject rights in ML, ultimately supporting the development of trustworthy artificial intelligence.