LLM Anonymization Against Agentic Re-Identification

arXiv:2605.30848 · cs.CR, cs.CL · Submitted 2026-05-29 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Security Radio. Generated commentary on the latest security and cryptography papers.

Nadia: I'm Nadia, and with me are Elias and Priya, guest researcher.

Elias: Today's paper: "LLM Anonymization Against Agentic Re-Identification".

Nadia: Agentic LLMs with web search change the anonymization problem because rich contextual details can become cross-referenceable evidence, yet those same details often carry significant downstream analytic value.

Elias: First, who's behind it and why it matters.

Paper summary: Nadia: We've seen that the paper introduces AURA as an LLM-powered mask-reconstruct framework specifically designed to address the new threat posed by agentic web search, where contextual details can become cross-referenceable evidence. The core thesis of "LLM Anonymization Against Agentic Re-Identification" is that existing defenses are insufficient because they don't account for this specific re-identification threat.

Elias: What the paper claims is that the central tension is between resisting these new agentic web search re-identification threats and keeping the downstream analytic utility of the text intact, since those contextual details are often valuable in their own right.

Priya: From my perspective, what this means practically for researchers is that we need a method that doesn't just aggressively scrub data but one that understands which parts of the context are truly sensitive versus which parts still carry meaningful research insight.

Nadia: Exactly, and AURA proposes a three-phase process: Phase zero initializes the system by inferring a privacy scope using web search to identify potential re-identification attributes <ref:2605.30848#pg0>. This is followed by Phase one Masking Convergence, where the transcript is iteratively rewritten based on that feedback until no more attributes can be inferred <ref:2605.30848#pg0>.

Elias: The paper claims this iterative rewriting process is how they handle the leakage through masking, resulting in a masked template with `MASKi + mask map M` after that convergence phase. This focuses heavily on reducing attribute leakage through this iterative rewriting guided by those privacy inferences.

Priya: I'm interested in the input for that process because Phase zero also involves extracting an "insight profile P," which summarizes the transcript's research value across eight utility dimensions, which seems like a vital step to quantify what we are trying to protect or preserve <ref:2605.30848#pg0>.

Nadia: That insight profile P is critical because it summarizes the research value in those eight dimensions, and that summary then guides Phase two Reconstruct, Evaluate, and Select <ref:2605.30848#pg0>. This phase generates several candidate rewrites for the masked spans before they get rigorously tested.

Elias: In Phase two each candidate rewrite gets assessed by an "attribute inference attacker" to determine privacy severity S and a "utility keeper" to measure utility loss L across those dimensions <ref:2605.30848#pg0>. This sets up the final selection step where they prioritize candidates based on specific criteria.

Priya: So it sounds like the paper is building a sophisticated system that doesn't just anonymize; it’s actively measuring its own effectiveness against both privacy and utility metrics simultaneously during the reconstruction phase.

Nadia: Precisely, and the final selection process involves selecting candidates that meet a specificity cap C less than or equal to C max first, and then choosing the one that minimizes both privacy severity S and utility loss L. This ensures the final sanitized transcript is optimized for both goals.

Elias: That optimization step is where I see the core technical contribution, as they decouple where to intervene from how to rewrite, giving us a flexible mechanism rather than a fixed redaction rule. This decoupling is really what makes this framework different from prior work in this area.

Priya: That decoupling sounds like it gives researchers the necessary control over the anonymization process that's often missing in current text processing methods when trying to balance these competing needs.

Nadia: So, the main point we took away is that AURA provides a framework for studying and tuning that separation between privacy and utility preservation in a way that's both adaptive and empirically validated against real transcripts.

Elias: It sounds like a very robust system because it’s not just relying on one static approach but dynamically adapting to the inferred threat landscape of the text.

Priya: And when we consider the results, it seems they show this adaptive variant can keep utility recovery rates up to eighty point three percent for the API-powered version, which is a solid performance number that validates its ability to maintain high analytic value while resisting these specific agentic attacks <ref:2605.30848#pg0>.

Conclusion: Nadia: To wrap up the paper "LLM Anonymization Against Agentic Re-Identification," the authors are presenting AURA as their primary contribution, which is a framework that tackles the operating region between resistance to agentic web-search re-identification and utility retention.

Elias: They introduce AURA as an LLM-powered mask-reconstruct framework that decouples where to intervene from how to rewrite, and they validated this by testing it against both adversarial privacy attacks and utility retention checks.

Priya: And the paper empirically characterizes how scope design influences resistance to re-identification while reconstruction preserves utility while maintaining privacy, showing a clear relationship between the two factors.

Nadia: Essentially, the implication is that effective anonymization needs to be a dedicated process rather than just a single redaction step, and that scope design acts as a practical control surface for users to adapt based on their specific release risk or analytic needs.

Elias: And they also pointed out that stronger or differently aligned attackers might expose residual risks, suggesting operators must treat anonymization as a multi-stage risk-management process.

Priya: It seems like the final message is that the framework offers a practical way to study and tune the separation between privacy and utility preservation in this complex LLM context.

Nadia: And it pushes that frontier by showing how to push that trade-off for LLM text release in a way that's empirically grounded.

Ziwen Li, Jianing Wen, Tianshi Li

Khoury College of Computer Sciences, Northeastern University

cs.CR, cs.CL

Submitted: 2026-05-29

Updated: 2026-10-01

Comments: 40 pages, 10 figures

Code: https://github.com/AaronLi43/AURA

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 91/100

The gist: Agentic LLMs with web search change the anonymization problem because rich contextual details can become cross-referenceable evidence, yet those same details often carry significant downstream

Key concepts

Agentic Re-identification Threats
This threat arises when an LLM agent uses web search to connect seemingly anonymous text snippets. Rich contextual details in transcripts can be cross-referenced online, allowing an attacker to re-identify individuals even if direct identifiers are removed.
AURA Framework
AURA is a three-phase LLM system for anonymization. It first infers privacy scope via search, then iteratively rewrites the text (Masking Convergence), and finally selects the best rewrite by balancing privacy risk against utility loss (Reconstruct, Evaluate, and Select).
Privacy-Utility Frontier
This concept maps the trade-off between how much protection you get for privacy versus how much useful information you keep. AURA demonstrates that different anonymization settings move along this frontier, showing that some methods offer better privacy at comparable levels of utility loss.
Masking Convergence
This is Phase 1 of AURA where the system repeatedly rewrites parts of the text based on privacy feedback from LLMs. The goal is to iteratively reduce attribute leakage until no more identifiable information can be inferred, resulting in a stable, masked template.

Terminology

Summary

Agentic LLMs with web search change the anonymization problem because rich contextual details can become cross-referenceable evidence, yet those same details often carry significant downstream analytic value. The core finding is that existing defenses are inadequate for agentic re-identification threats, necessitating a framework that decouples privacy localization from utility-preserving reconstruction.

How it works

The proposed solution is AURA (Anonymization with Utility-Retention Adaptation), an LLM-powered mask-reconstruct framework designed to address the tension between resistance to agentic web-search re-identification and utility retention. This framework operates in three distinct phases: Phase 0: Initialization, Phase 1: Masking Convergence, and Phase 2: Reconstruct, Evaluate, and Select. The process begins with inferring a privacy scope using web search capabilities to identify potential re-identification attributes (Phase 0). This is augmented by extracting an insight profile P summarizing the transcript's research value across eight utility dimensions.

Phase 1: Masking Convergence

In Phase 1, the system iteratively rewrites the transcript based on privacy-inference feedback from LLMs. The process continues until no further attributes can be inferred or a stopping condition is met. This results in a converged rewrite, which is then used to generate a masked template T̂ with [MASKi] + mask map M. This phase focuses on reducing attribute leakage through iterative rewriting conditioned on the current privacy inferences.

Phase 2: Reconstruct, Evaluate, and Select

Phase 2 involves generating N candidate rewrites for masked spans and then rigorously evaluating them. Each candidate is assessed by an attribute inference attacker to determine privacy severity (S), a utility keeper to measure utility loss (L), and a specificity check. The selection process prioritizes candidates satisfying the specificity cap, denoted as V = [n Cn ≤ Cmax], and then selects the one that minimizes privacy severity S and utility loss L. This ensures the final sanitized transcript T∗ is optimized for both goals.

Key Contributions

The paper makes three main contributions to the field: first, it is the first to optimize and evaluate LLM text anonymization in the operating region between resistance to real-world agentic web-search re-identification and retention of downstream analytic utility. Second, AURA propose[s] an LLM-powered mask-reconstruct framework that decouples where to intervene from how to rewrite, and is evaluated with both agentic re-identification attacks and utility-retention checks. Third, it empirically characterizes how privacy and utility change across anonymization settings, showing that attribute scope primarily influences resistance to re-identification while reconstruction preserves utility while maintaining privacy.

Evaluation and Results

The framework was evaluated on 27 real-user interview transcripts from the Anthropic Interviewer dataset using three attacker models (GPT-5.1, GPT-5.4-mini, and Gemini-3-Flash). The results demonstrate that AURA's adaptive variants reduce agentic re-identification to 0–5/27 transcripts, substantially outperforming baselines like NER (13–21/27) and prior LLM anonymizers (6–7/27), while retaining high utility recovery rates, such as 80.3% for the API-powered adaptive variant. The analysis shows that AURA's adaptive variants sit closest to the upper-right corner of the privacy-utility frontier, indicating they achieve substantially higher privacy than non-DP baselines at comparable utility levels.

Implications

The findings suggest that effective anonymization requires a dedicated process rather than a single redaction step. The paper concludes that scope design [is] a practical control surface, allowing users to adapt the privacy scope based on release risk or analytic needs. AURA provides a practical framework for studying and tuning the separation between privacy and utility preservation, pushing the privacy-utility tradeoff frontier for LLM-era text release. Furthermore, it highlights that stronger or differently aligned attackers may expose residual risks, suggesting that operators must treat anonymization as a multi-stage risk-management process.

Limitations

The evaluation acknowledges limitations, noting that utility metrics are based on model-based recoverability judgments rather than human raters, and privacy counts rely on a direct-intent re-identification protocol rather than formal privacy guarantees. The study also emphasizes that privacy evaluation must avoid becoming a new source of exposure when using real participant transcripts. The results are controlled stress tests, not absolute anonymity guarantees.

References

[1] Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774 (2023).

Improvements for AI systems

Here are the specific improvements and capabilities derived from the AURA framework for enhancing AI systems:

  1. The ability to perform surgical, span-level anonymization rather than full transcript rewriting allows for high-fidelity preservation of domain-specific jargon and qualitative nuance (e.g., preserving specific technical terms related to sensor physics or research methodologies) while neutralizing re-identification risks associated with quasi-identifiers.

  2. The system can decouple privacy localization from utility reconstruction, meaning it can first identify the precise contextual cues that an agentic LLM exploits for re-identification (Phase 0) and then selectively mask/reconstruct only those specific spans (Phase 2), rather than applying a blanket perturbation or redaction to the entire document.

  3. The framework enables adaptive privacy scope expansion, allowing a data steward to dynamically adjust the level of anonymization based on external threat intelligence or release context. This means systems can move from high-privacy/low-utility modes (for sensitive internal discussions) to high-utility/moderate-privacy modes (for general publication drafts) by simply changing the scope parameter.

  4. The system can generate Risk Maps via the masked template and mask map, providing an auditable visualization of exactly which parts of a document are currently treated as privacy risks before any reconstruction is finalized, facilitating a more transparent data governance workflow.

  5. The framework supports on-device deployment using smaller open-weight models (like Qwen3.5) for the reconstruction phase while leveraging larger API models for complex inference tasks (like initial attribute profiling), enabling scalable and secure implementation in resource-constrained environments without sacrificing high utility recovery rates (up to 80% unit-level).

  6. The system optimizes the privacy-utility frontier by employing a dual selection criterion: prioritizing adherence to a specificity cap (limiting the number of attributes leaked) before minimizing privacy severity, ensuring that too specific data is aggressively generalized before focusing on minimizing direct re-identification risk.

  7. The framework provides robust cross-attacker resilience; by testing against multiple LLM attacker models (GPT-5.1, GPT-5.4-mini, Gemini-3), the resulting anonymized output is less susceptible to bias toward a single model's blind spots, leading to more reliable privacy guarantees in real-world scenarios.

  8. The utility evaluation is sophisticated, measuring recovery at multiple levels (Interviewee Profile Facts and Codebook Facts) and combining them into a Utility Grid, which accurately reflects the downstream analytical requirements of qualitative research—capturing not just who the person is, but also what they do and how they work (e.g., pairing a health services researcher with an AI delegation criteria).

Abstract

Agentic LLMs with web search change the threat model for text anonymization: weak contextual cues can become cross-referenceable evidence for re-identification, yet those same details also carry downstream analytic value of the text. Existing defenses either remove explicit identifiers, perturb text for formal privacy, or test rewritten text against non-web inference models, leaving underexplored the operating region between resistance to agentic web-search re-identification and utility retention. We introduce AURA (Anonymization with Utility-Retention Adaptation), an LLM-powered mask-reconstruct framework that decouples privacy localization from utility-preserving reconstruction and selects candidates with adversarial privacy and utility-retention checks. We evaluate AURA on real-user interview transcripts using re-identification attacks carried out by web-search agents, along with a utility evaluation based on interviewee-profile facts, codebook facts, and the joint contextual utility grid. Our results show that adaptive-scope AURA yields the lowest agentic re-identification counts under each of three attacker models among the non-DP methods, and that at matched scope and backbone, AURA's mask-reconstruct design retains more contextual utility than the prior LLM anonymizer (+6.4 pp unit-grid recovery) at comparable privacy. Source Code: https://github.com/AaronLi43/AURA

Sources

Related papers