Detectability without Interference: Efficient Canary Design for Privacy Auditing in One Run

summary

Video file (mp4)

The gist

The gist The work proposes an influence-based method to select canaries that are both highly detectable and diverse, which initialize a bilevel optimization procedure to refine them for one-run

In short

The work addresses a limitation in one-run privacy auditing where sensitive samples (canaries) can interfere with each other, weakening privacy estimates. The proposed IBIS framework optimizes canary selection using influence functions and bilevel optimization to maximize individual detectability while minimizing negative impact on others, significantly improving performance-cost trade-offs.

Key concepts

Canary Interference
This occurs when multiple sensitive samples included in a single training run affect each other's privacy scores. When canaries are too similar, their effects become correlated, reducing the overall power of the audit and making it harder to accurately measure leakage.
Influence Function (Is)
The influence function measures how much the inclusion or exclusion of a specific data point 'z' shifts the model's score. A high influence means that including or excluding 'z' significantly changes the privacy assessment of another point, making it highly detectable.
Bilevel Optimization
This is a mathematical technique used to solve complex problems where the objective function depends on an inner optimization problem. In this context, it allows the framework to simultaneously maximize individual canary scores while enforcing diversity constraints in the model's representation space.
Orthogonality Regularizer
This term encourages canaries to be 'orthogonal,' meaning their learned representations in the embedding space are independent of each other. By promoting this independence, the regularization term actively minimizes interference between different canaries.

Terminology used across episodes

This episode discusses

The paper

Detectability without Interference: Efficient Canary Design for Privacy Auditing in One Run · Read on arXiv

inria · Idesp · Inserm · University of Montpellier

Privacy auditing aims to empirically assess privacy leakage in machine learning models using membership inference attacks (MIAs), and to derive lower bounds on differential privacy (DP) parameters. Standard privacy auditing inserts a sensitive sample (a canary) into many (100) randomized training runs, requiring the auditor to detect whether it was included in each run. In one-run auditing, many canaries are included in a single training run, avoiding the high computational cost of multi-run approaches. However, recent theory shows that canaries can interfere with one another, reducing per-canary MIA scores and thereby weakening the audit power. We propose IBIS, a framework that optimizes canaries to maximize their own score while minimizing their impact on others. IBIS combines influence-based greedy initialization with bilevel optimization, introducing a novel regularization term that encourages canaries to be orthogonal in embedding space as a proxy for reducing interference. We propose an efficient bilevel algorithm that incrementally updates a single model at each canary update, avoiding the need to retrain the model from scratch after every update. Experiments show that IBIS offers a better performance-cost trade-off than existing canary crafting approaches, improving MIA metrics while reducing computational cost by up to two orders of magnitude.

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: I'm Tom.

Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.

Tom: Today's paper: "Detectability without Interference".

Jane: The gist The work proposes an influence-based method to select canaries that are both highly detectable and diverse,

Tom: First, who's behind it and why it matters.

Paper summary: Tom: Now let's go deeper into the specifics of "Detectability without Interference: Efficient Canary Design for Privacy Auditing in One Run". The authors are setting out to solve the problem of efficiently crafting canaries for one-run privacy auditing. They are motivated by theoretical insights suggesting that interference between canaries weakens leakage estimates compared to multi-run methods (<ref:2605.27292#pg2>).

Jane: So, the paper addresses the gap where the upper bound from accounting and the lower bound from auditing don't match up in one-run DP settings (<ref:2605.27292#pg1>). They want to bridge that gap by optimizing canaries to be both highly detectable and minimally interfering.

Lu: Their approach combines a greedy initialization based on influence functions with a bilevel optimization procedure that maximizes distinguishability while promoting diversity in embedding space (<ref:2605.27292#pg1>). They use an explicit diversity regularizer encouraging orthogonality in representation space (<ref:2605.27292#pg1>).

Meng: That sounds like they are using influence functions to pick the initial set, and then using that as a starting point for a more complex optimization problem to enforce that orthogonality. It’s a layered approach, which is good for understanding complexity.

Lalam: The greedy selection identifies training points with "high self-influence but low pairwise cross-influence," formalized by maximizing the ratio Is(z,z) over max z' in Cz of Is(z',z) (<ref:2605.27292#pg1>). This is how they ensure individual detectability while minimizing mutual interference.

Tom: And that ratio is a way to mathematically quantify the trade-off between being detectable and being disruptive, right? It's not just picking random points; it’s based on how much they affect each other.

Jane: Right, so the next step is the bilevel optimization formulation: min C Φ(C) which includes that representation orthogonality term Rθ*(C) (<ref:2605.27292#pg1>). That regularization term Rθ*(C) is defined by Pz,z'∈C z̸=z′ ⟨φθ∗ (z), φθ∗ (z′)⟩ squared.

Meng: So they are explicitly penalizing correlations in the learned representations of these canaries. If two canaries look too similar in the model space, this term gets big, and the optimization pushes them apart.

Lu: The optimization is solved using an adaptation of the SOBA algorithm which tracks variables theta, v and Cx iteratively to avoid retraining at each update (<ref:2605.27292#pg1>). This iterative approach is what makes it efficient for a single run setting.

Tom: So, we're seeing them use influence functions to start a greedy selection, then use a bilevel optimization with an orthogonality regularizer to refine those selections incrementally using SOBA? That’s a lot of machinery packed into one framework.

Jane: It is complex, but the goal is clear: to optimize canaries to be both highly detectable and minimally interfering (<ref:2605.27292#pg1>). This whole structure is designed to improve the MIA metrics while reducing computational cost significantly.

Conclusion: Tom: So wrapping up the discussion on "Detectability without Interference: Efficient Canary Design for Privacy Auditing in One Run". The authors are proposing this IBIS framework, which combines influence-based greedy initialization with bilevel optimization to optimize canaries.

Jane: They show that this approach offers a better performance-cost trade-off than existing canary crafting approaches (<ref:2605.27292#pg1>). It improves MIA metrics while reducing computational cost by up to two orders of magnitude.

Lu: The paper focuses on the canary crafting stage of one-run privacy auditing by proposing this influence-based method to select canaries that are both highly detectable and diverse (<ref:2605.27292#pg1>). It also matches prior work for DP privacy auditing while being significantly more computationally efficient.

Meng: The main implication for the field is that we can get better privacy leakage estimates in a one-run setting without needing the heavy computational load of traditional multi-run experiments.

Lalam: The IBIS pipeline offers a clear advantage in terms of computational cost (<ref:2605.27292#pg1>). It’s about making auditing more practical for real-world deployment scenarios where resources are limited.

Tom: So, the paper provides a concrete method that balances detectability and interference while keeping the cost low. It's a solid piece of work that moves this area forward by offering a more efficient way to approach canary design in one-run auditing.

More episodes

← Home