From One Query to Many: A Critical Survey of Privacy Accounting in RAG

summary

The gist

The gist The survey critically examines how privacy accounting evolves from single-query generation to multi-query RAG systems.

In short

This survey examines how privacy accounting changes when moving from generating a single response to running many queries in Retrieval-Augmented Generation (RAG) systems. It categorizes existing methods into three types: query-time privatization, multi-query accounting, and one-time corpus privatization. The authors propose a new framework for evaluating DP guarantees based on privacy granularity, accounting scope, and release boundaries.

Key concepts

Retrieval-Augmented Generation (RAG): RAG allows large language models to use external knowledge bases during inference. While useful for specialized domains, it poses privacy risks because the retrieved data might contain sensitive information that is then processed by the model.
Differential Privacy (DP): DP is a formal mathematical technique used to provide quantifiable privacy guarantees when using data or models. The paper highlights that existing DP methods struggle to compare their guarantees across different scenarios, especially when assessing repeated queries.
Privacy Accounting Evolution: This refers to the shift in how privacy loss is measured as RAG systems move from answering one question to handling many. The survey argues that a privacy budget reported for a single answer does not accurately reflect the total privacy loss accumulated over a stream of repeated queries.
Query-time Privatization: This approach spends the privacy budget during each individual query. It involves splitting retrieved documents, running the model once per group, and releasing only a noisy aggregate output for that specific query. The main limitation is that the fidelity of the answer is fixed and cannot adapt to subsequent queries.

Terminology used across episodes

Reinforcement Learning (RL) — explained in 25 episodes
Sample Complexity — explained in 20 episodes
Federated Learning — explained in 17 episodes
Flow Matching — explained in 17 episodes
Quantum Fisher Information (QFI) — explained in 16 episodes
Interpretability — explained in 15 episodes

This episode discusses

The paper

From One Query to Many: A Critical Survey of Privacy Accounting in RAG · Read on arXiv

Zhiyuan Zhang, hollyzhang03@ucla.edu

University of California, Los Angeles

Retrieval-augmented generation (RAG) enables large language models to leverage external knowledge but introduces privacy risks when the retrieved corpus contains sensitive information. Differential privacy (DP) offers formal protection, yet interpreting and comparing privacy guarantees across existing DP-RAG methods remains challenging. In particular, a privacy budget reported for a single response does not directly characterize privacy loss over repeated queries, and even identical values of epsilon may represent different guarantees depending on the protected data unit, accounting horizon, and observable outputs. This survey critically examines how privacy accounting evolves from single-query generation to multi-query RAG systems. We categorize existing approaches into query-time privatization, multi-query accounting, and one-time corpus privatization. We analyze their underlying assumptions and privacy--utility trade-offs. Rather than ranking methods by epsilon alone, we propose a framework that interprets a DP-RAG guarantee along three dimensions: privacy granularity, accounting scope, and release boundaries. We conclude that an is a claim only together with the three slots: the protected unit, the accounting scope, and the release boundary.

Transcript

Introduction to the show: ident: Security Radio. Generated commentary on the latest security and cryptography papers.

Nadia: I'm Nadia.

Elias: And I'm Elias, here with Priya.

Nadia: Today's paper: "From One Query to Many".

Elias: The gist The survey critically examines how privacy accounting evolves from single-query generation to multi-query RAG systems.

Nadia: First, who's behind it and why it matters.

Paper summary: Nadia: We're talking about this paper "From One Query to Many: A Critical Survey of Privacy Accounting in RAG" again. The main idea is that we need a better way to track privacy when we move from just one query to handling many queries in RAG systems.

Elias: It sets out the thesis that the standard way of reporting a privacy budget for a single response is insufficient because it doesn't capture the total privacy loss over repeated interactions.

Priya: So what they claim is that this issue isn't just about having a bigger epsilon number, but about understanding how that epsilon changes based on other factors like the data unit or how many queries you are planning to run.

Nadia: They highlight that even if two systems report the same epsilon, they might be protecting completely different things or exposing different information to an adversary.

Elias: The paper specifically points out that a single-response budget doesn't tell you much about the privacy loss when you serve a whole stream of answers, and this gap depends on the number of queries.

Priya: And they show that identical epsilon values can represent different guarantees depending on the protected data unit, the accounting horizon, and what actually gets released to someone who isn't running the system.

Nadia: The paper organizes existing methods into three categories: query-time privatization, multi-query accounting, and one-time corpus privatization. These are how they frame the problem of privacy accounting in RAG today.

Elias: They then analyze the underlying assumptions for each of those families and look at the trade-offs between privacy and utility in each approach.

Priya: It seems like this survey is important because it forces us to stop just looking at one number and start considering the whole system's behavior over time.

Nadia: The core contribution is proposing that we should interpret a DP-RAG guarantee using three dimensions: privacy granularity, accounting scope, and release boundaries.

Elias: So instead of just chasing a higher epsilon, the authors suggest focusing on those three specific aspects to really understand the privacy protection offered.

Priya: That gives us a concrete way to evaluate different systems beyond just looking at one number alone.

Conclusion: Nadia: So we've talked about this survey, "From One Query to Many: A Critical Survey of Privacy Accounting in RAG," and the authors are Zhiyuan Zhang at ucla.edu, and that’s a big picture summary of the work.

Elias: The implication here is that for anyone building or using DP-RAG systems, the way you define your privacy guarantee needs to be much more specific than just picking an epsilon value.

Priya: In simple terms, it means we have to stop treating privacy accounting like a simple checkbox and start thinking about the context of the entire interaction stream.

Nadia: It's about moving beyond just reporting a single number to understanding how that number plays out across different scenarios—different queries, different data being protected.

Elias: The paper suggests that any real evaluation of these systems has to consider privacy granularity, accounting scope, and the release boundary as the three crucial lenses.

Priya: That shifts the focus from just achieving a certain epsilon to designing a system with clear rules about what stays protected and what gets shown at every stage.

More episodes

← Home