It's My Agent, Not Me: Personal Privacy Boundaries in AI-Delegated Information Sharing
summary
The gist
As a diligent and fastidious researcher, I have meticulously analyzed the provided text snippets from arXiv to synthesize a comprehensive and detailed summary of the paper, "It's My Agent, Not Me:
In short
Researchers developed an AI-powered method to uncover nuanced personal privacy boundaries by having participants complete a task involving information sharing delegated to an AI agent. The study found that communication roles and whether sharing was delegated to an AI significantly influence how sensitive users are to disclosing details, proving that privacy preferences are highly context-dependent.
Key concepts
- AI-powered Elicitation Approach
- This is a method where participants complete a specific task designed to reveal their private boundaries. Instead of asking direct questions, the task forces users to share information in ways that expose their hidden trade-offs between sharing and privacy, which the AI then analyzes.
- Communication Roles
- Participants were assigned different roles during the information exchange, such as sender or recipient. The study found that an individual's role directly changed how sensitive they were to revealing specific details, depending on whether they were sending or receiving private data.
- AI Delegation Condition
- This refers to the scenario where a human shares information versus when an AI agent is tasked with sharing it. Findings showed that delegating sharing to an AI generally increased caution regarding the disclosure of identifiable information across all tested contexts.
Terminology used across episodes
This episode discusses
- It's My Agent, Not Me: Personal Privacy Boundaries in AI-Delegated Information Sharing · Paper Radio
- Reducing Privacy Risks in Online Self-Disclosures with Language Models
- Operationalizing Contextual Integrity in Privacy-Conscious Assistants
- User Privacy Harms and Risks in Conversational AI: A Proposed Framework
- Are Large Pre-Trained Language Models Leaking Your Personal Information?
- Artificial Intelligence in Deliberation: The AI Penalty and the Emergence of a New Deliberative Divide
- Exploring User Security and Privacy Attitudes and Concerns Toward the Use of General-Purpose LLM Chatbots for Mental Health
- Contextual Integrity in LLMs via Reasoning and Reinforcement Learning
- Eliciting Human Preferences with Language Models
- Privacy Checklist: Privacy Violation Detection Grounding on Contextual Integrity Theory
- LLM-as-a-Judge for Privacy Evaluation? Exploring the Alignment of Human and LLM Perceptions of Privacy in Textual Data
- Can LLMs Keep a Secret? Testing Privacy Implications of Language Models via Contextual Integrity Theory
- You Are What You Write: Preserving Privacy in the Era of Large Language Models
- Aligning Language Models with Demonstrated Feedback
- A Roadmap to Pluralistic Alignment
- Towards Aligning Personalized Conversational Recommendation Agents with Users' Privacy Preferences
- "Ghost of the past": identifying and resolving privacy leakage from LLM's memory through proactive user interaction
- Privacy Leakage Overshadowed by Views of AI: A Study on Human Oversight of Privacy in Language Model Agent
The paper
It's My Agent, Not Me: Personal Privacy Boundaries in AI-Delegated Information Sharing · Read on arXiv
BINGCAN GUO, ERYUE XU, ZHIPING ZHANG, TIANSHI LI
University of Washington · University of Illinois Urbana-Champaign · Northeastern University
Aligning AI systems with human privacy preferences requires understanding individuals' nuanced disclosure behaviors beyond general privacy norms. Yet eliciting these boundaries remains challenging because they involve context-dependent trade-offs. We present an elicitation approach that probes individuals' privacy boundaries through a discriminative task. Through a between-subjects study with varying communication roles and delegation conditions, we collected 1,650 privacy boundaries from 169 participants across 60 scenarios. We examined how communication roles, AI delegation, and individual differences influence boundaries. Results show that communication roles significantly influence sensitivity to disclosure granularity or identifiability, depending on whose private information is shared; AI delegation and individuals' need for privacy heighten sensitivity to disclosed identifiers. Our analysis further reveals how AI delegation shifts privacy preferences across contexts, highlighting new complexities in AI-mediated communication. We advocate eliciting nuanced privacy boundaries within real-world data flows by considering roles and AI delegation, and enabling user involvement in guiding privacy-aware AI behavior.
Transcript
Introduction to the show: ident: Security Radio. Generated commentary on the latest security and cryptography papers.
Nadia: Today's paper: "It's My Agent, Not Me".
Elias: As a diligent and fastidious researcher, I have meticulously analyzed the provided text snippets from arXiv to synthesize a comprehensive and detailed summary of the paper, "It's My Agent,
Nadia: First, who's behind it and why it matters.
Paper summary: Nadia: So this paper, "It's My Agent, Not Me: Personal Privacy Boundaries in AI-Delegated Information Sharing," it’s about figuring out how to actually understand people’s private boundaries when they let an AI handle the sharing.
Elias: Basically, the core idea is that we need to look past just general privacy rules because those rules don't capture how context really changes what someone is willing to share.
Nadia: They use this new AI-powered elicitation approach where they set up a task for people that forces them to define these boundaries by showing them different versions of the same information.
Priya: So, what do they claim this method actually achieves? Is it just a survey?
Elias: No, it’s framed as a discriminative task instead of generative, which means we’re not asking people to create new privacy rules from scratch. Instead, we show them nine different versions of the same private data and ask them to label each one as acceptable or unacceptable.
Nadia: That lets them specify boundaries based on granularity and identifiability levels, which is what they want to know about how much detail people are okay sharing.
Priya: And how many people did they study for this? How big was their sample size for seeing if this method actually worked?
Elias: They conducted a between-subjects study with one hundred sixty-nine participants across sixty-one scenarios, which resulted in one thousand six hundred eighty-one boundary specifications <ref:2509.21712#pg1,conducted a between-subjects study>. That’s a pretty substantial set of data to work with.
Nadia: The main point they want to make is that we need to examine how communication roles and AI delegation actually influence the boundaries people set up in these situations.
Elias: They also looked at individual differences, checking how traits like the inherent need for privacy affect what boundaries a person specifies.
Priya: So, what’s their big takeaway regarding the context? Does it mean that if you change who is sending or receiving information, those boundaries shift significantly?
Nadia: Exactly. The study found that communication roles—like being the sender versus being the recipient—significantly changed how sensitive people were to sharing specific details or making things identifiable.
Elias: And when you introduce AI delegation into that mix, it seems to heighten sensitivity, especially when it comes to disclosed identifiers, according to their results.
Conclusion: Nadia: So, looking at "It's My Agent, Not Me: Personal Privacy Boundaries in AI-Delegated Information Sharing," the authors are really pointing out that privacy isn't just a static setting you choose once; it’s this messy interaction between who you are talking to and what tool you’re using.
Elias: They’re arguing that we can’t just look at one person and assume their privacy needs will stay the same no matter the situation, which is why they focused on those contextual factors like roles and delegation.
Nadia: The implication for us as people building these AI systems is that if we want an AI agent to be truly helpful without being intrusive, we can’t rely on a single set of privacy settings.
Elias: We have to design the system so it can adapt its disclosure level based on the specific communication role and whether it’s acting as a human or an AI delegate in that moment.
Priya: It sounds like they are saying that for real-world AI alignment, we need to stop looking for one universal privacy truth and start designing systems that respect this variety of individual needs.
Nadia: Precisely. If we don't account for how delegation changes those boundaries across different scenarios, the AI might be sharing too much or being unnecessarily restrictive in a way that doesn't match what the user actually wants in a specific interaction.
More episodes
- 2604.02837-Towards Secure Agent Skills: Architecture, Threat Taxonomy, and Security Analysis
- 2605.10998-Few-Shot Truly Benign DPO Attack for Jailbreaking LLMs
- 2606.09549-SecureClaw: Clawing Back Control of LLM Agents
- 2606.30587-Words Speak Louder Than Code: Investigating Cognitive Heuristics in LLM-Based Code Vulnerability Detection
- 2605.02255-On the Privacy of LLMs: An Ablation Study
- 2506.03467-Differentially Private Distribution Release of Gaussian Mixture Models via KL-Divergence Minimization
- 2605.05251-Identifier-Free Code Embedding Models for Scalable Search
- 2309.17186-Unaware, Unfunded and Uneducated: A Systematic Review of SME Cybersecurity
- 2505.15051-An Empirical Analysis of EOS Blockchain: Architecture, Contract, and Security
- 2601.13031-Post-Quantum Secure Aggregation via Code-Based Homomorphic Encryption