Topology-Aware Differential Privacy in Federated Learning

arXiv:2506.19260 · cs.CR, cs.DC, cs.LG · Submitted 2026-08-19 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Next we'll be talking about the paper "Topology-Aware Differential Privacy in Federated Learning".

Jane: The paper was written by Murtaza Rangwala, Richard O. Sinnott and Rajkumar Buyya from Quantum Cloud Computing and Distributed Systems (qCLOUDS) Lab and School of Computing and Information Systems and University of Melbourne, Australia.

Tom: Stay tuned as we take you through the paper and discuss its implications.

Jane: We also have Lu with us today — senior AI researcher at Tsinghua.

Tom: We also have Meng with us today — lead engineer at a mysterious AI startup.

Jane: We also have Lalam with us today — the in-house Large Language Model.

Tom: Alright, let's get started.

Summary: Tom: Moving past the title, let’s look at what this paper titled "Topology-Aware Differential Privacy in Federated Learning" actually finds when it runs its analysis. The authors introduce TADI, which is their new framework for breaking down the overall leakage into these manageable channels.

Jane: They explain that TADI is a learned regressor that performs four specific ablations to measure exactly how much information comes from each source, parameter, structural position, or organizational label.

Lu: These four channels are designed to isolate the marginal contribution of each information stream, allowing us to see precisely where the leakage originates in terms structure and parameters.

Meng: Those ablations allow us to pinpoint where the leaks are coming from in practice without needing to guess; it provides a measurable, empirical breakdown of the threat.

Lalam: It allows us to map this theoretical risk onto a real-world deployment scenario, providing tangible evidence of where privacy risks exist within our data systems.

Tom: So, when we look at this paper titled "Topology-Aware Differential Privacy in Federated Learning," TADI gives us that empirical breakdown showing leakage isn't just coming from the parameters.

Jane: It’s revealing that the structural and organizational parts of the network are carrying a significant amount of information about sensitive class concentrations, even when noise is present.

Lu: And this is where it gets really interesting, because I see these channels as providing a clear map of where we need to focus our defensive resources in an AI system.

Meng: TADI confirms the additive nature of the leakage, meaning that knowing how much information comes from structure versus parameters gives us a very clear path forward for targeted fixes.

Lalam: It’s about confirming that this leakage is real and measurable, offering a visual representation of where privacy risks are within the data ecosystem.

Improvements: Tom: So, we have TADI giving us the map; now we need to talk about the solution presented in "Topology-Aware Differential Privacy in Federated Learning." The authors propose FULCRUM, which is a new noise allocation strategy that directly addresses the asymmetric risks identified by TADI.

Jane: Instead of applying uniform noise like standard DP-SGD, FULCRUM uses what they call structural leverage scores to guide how much protection goes to each client based on their position.

Lu: It’s about tailoring the defense so that clients who are in more exposed positions receive stronger protection than those in symmetric spots.

Meng: This means we can finally move away from a "one size fits all" approach and apply a tailored defense based on how much leverage a specific client has within the network structure.

Lalam: The idea is that we should give more noise where the network structure is most likely to leak information, which makes sense for improving our ethical deployment of AI.

Tom: This paper titled "Topology-Aware Differential Privacy in Federated Learning" suggests FULCRUM, a closed-form min-max allocation that strictly improves upon uniform DP-SGD whenever the structural leverage scores are non-uniform.

Jane: It only applies this tailored approach when the network structure is asymmetric, which is what most real life systems are, allowing us to avoid over-protecting clients who don't need it.

Lu: And I appreciate that they didn't just make it a blanket fix; Meng mentioned the proxies—group size, degree, and dataset size—which are practical ways to estimate that structural leverage for FULCRUM.

Meng: Those proxies mean we can actually use this in a real system without having to calculate complex mathematical proofs every time, making it highly deployable.

Lalam: It’s about using these tools to ensure that the more vulnerable organizations or clients in our network get the benefit of better protection.

Conclusion: Tom: We've seen how TADI maps the leaks and FULCRUM provides a solution, so let's wrap up our discussion on "Topology-Aware Differential Privacy in Federated Learning." The results are incredibly positive, showing that this tailored approach is effective across different real world datasets.

Jane: The findings confirm that this tailored approach is effective across datasets like Fed-ISIC2019 and show significant privacy gains, proving it works where the network structure is complex.

Lu: I'm particularly excited to see the theoretical work; the proof that the parameter channel is bounded by DP-SGD while achieving a real prior-coupling floor demonstrates how robust this entire framework is.

Meng: The key practical finding for me—and I think it’s critical—is that these privacy gains, up to one point nine six seven nats, happen at absolutely no measurable utility cost, which is a massive win for adoption.

Lalam: This means we can be much more confident that AI systems will be both highly accurate in their predictions and deeply protective of sensitive data.

Tom: So, in summary, this paper titled "Topology-Aware Differential Privacy in Federated Learning" provides a principled defense that works whether or not the federation is perfectly symmetric.

Jane: It shows we finally have a way to account for the structural asymmetry that was previously ignored when moving toward a practical deployment of AI.

Lu: I'm looking forward to seeing how this guides our research into more complex, dynamic network topologies in future projects.

Meng: This provides the clear roadmap we need to implement FULCRUM in our production environments, ensuring we address structural risk directly.

Lalam: I hope that the implementation of "Topology-Aware Differential Privacy in Federated Learning helps us build a digital future where fairness and privacy are guaranteed for every single person.

Conclusion: Tom: So, we’ve spent time digging into this paper and discuss its core findings, but it’s important to bring it all together one last time as we wrap up our discussion on "Topology-Aware Differential Privacy in Federated Learning."

Jane: It really shows that the authors managed to bridge a huge gap—the assumption of perfect symmetry in FL was simply no longer enough for real-world applications.

Lu: The fact that they formally separated the leakage into controllable and uncontrollable parts is a massive theoretical leap, giving us a framework to manage complexity that we previously ignored.

Meng: And it’s not just theory; I think the practical application of FULCRUM, especially when using those leverage proxies, makes this highly actionable in enterprise systems.

Lalam: It's about recognizing that the way information physically moves across a network is a critical factor in how we ensure privacy and trust.

Tom: That’s exactly it—the paper provides a robust defense that works whether or not your federation has structural symmetry, which is what makes it such an important read.

Jane: I agree, Tom; the results are incredibly strong and practical, providing that much-needed assurance to stakeholders who don't want to sacrifice accuracy for privacy.

Lu: The ability to prove that this tailored allocation strictly improves upon uniform DP-SGD when the leverage is asymmetric confirms that mathematically sound the this entire approach is.

Meng: It proves we can implement a better, more precise mechanism without running into huge utility costs, which is what matters most when deploying AI at scale.

Lalam: This advancement helps us build digital systems that are not only powerful but also ethically responsible for every single person involved in the network.

Tom: Truly a breakthrough, Jane. We're excited to see how this guidance leads to real-world deployment, and we'll be back next time to explore another fascinating paper on AI ethics and design.

Murtaza Rangwala, Richard O. Sinnott, Rajkumar Buyya

Quantum Cloud Computing and Distributed Systems (qCLOUDS) Lab · School of Computing and Information Systems · University of Melbourne, Australia

cs.CR, cs.DC, cs.LG

Submitted: 2026-08-19

Updated: 2026-08-20

Comments: 27 pages, 5 figures, 8 tables. Data from the experiments and source code can be found here: https://doi.org/10.5281/zenodo.20507155

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 92/100

The gist: " Federated learning (FL) allows collaborative model training without centralizing raw data, relying on the premise that an adversary observing only model updates learns little about the underlying

Key concepts

TADI: TADI is a new framework designed to break down overall data leakage into manageable channels. It functions as a learned regressor that performs ablations to measure how much information comes from different sources, such as parameters or structural positions.
FULCRUM: FULCRUM is a proposed noise allocation strategy that directly addresses asymmetric risks identified by TADI. Instead of using uniform noise like standard DP-SGD, it uses structural leverage scores to guide protection based on a client's position in the network.

Terminology

Federated Learning — 41× in this paper · explained in 15 episodes
Aggregation — 26× in this paper · explained in 1 episode
Differential Privacy — 25× in this paper · explained in 4 episodes
AUROC — 17× in this paper · explained in 2 episodes
Ablation — 16× in this paper · explained in 1 episode
Benchmark — 7× in this paper · explained in 2 episodes
Calibration — 7× in this paper · explained in 7 episodes
Consistency — 6× in this paper · explained in 3 episodes
Accuracy — 5× in this paper · explained in 1 episode
Decentralized Federated Learning — 4× in this paper · explained in 2 episodes
Differentially Private — 4× in this paper · explained in 3 episodes
Alignment — 3× in this paper · explained in 1 episode
Core — 3× in this paper · explained in 2 episodes
Topology-Aware Distributional Inference — 3× in this paper
Federated Learning (FL) — 2× in this paper · explained in 7 episodes
Non-IID Data — 2× in this paper · explained in 2 episodes
Two One-Sided Tests — 2× in this paper
Adversarial Model — 1× in this paper · explained in 1 episode
Clustering — 1× in this paper · explained in 2 episodes
Decentralized Learning — 1× in this paper · explained in 2 episodes
Generative Model — 1× in this paper · explained in 2 episodes
Grounding — 1× in this paper · explained in 3 episodes
Loss Function — 1× in this paper · explained in 2 episodes
Personalization — 1× in this paper · explained in 2 episodes
Trace — 1× in this paper · explained in 2 episodes

Summary

Federated learning (FL) allows collaborative model training without centralizing raw data, relying on the premise that an adversary observing only model updates learns little about the underlying client data. While standard defenses like differentially private SGD (DP-SGD) bound content-level leakage (e.g, gradient inversion or membership inference), They do not account for what the communication topology of the federation itself reveals.

The paper formalizes a threat where a passive adversary possesses knowledge of the communication topology (G) and organizational structure (omega). This adversary has access to two information channels that DP-SGD leaves unaddressed:

The structural position of each client in the communication graph.
The organizational membership encoded by the deployment.

The core insight is that The information an adversary can extract about a client’s data separates additively into two terms: a controllable mechanism term... and an uncontrollable prior-coupling term.

Formal Definitions:

System Model: A federation consists of n clients P = P 1,, P n arranged in a communication topology G. Each client P i has a local class distribution i.
Adversarial Goal: The inference goal is to recover the sensitive-class concentration p i = i(C s).
Structural Leverage (i): This is defined as the maximum possible prior coupling between a client's sensitive-class concentration and the rest of the federation’s data, i:= P in FG, omega I(p i; D-i).

The paper introduces two principal contributions: TADI (Topology-Aware Distributional Inference) and FULCRUM (a principled noise allocation mechanism).

A. TADI Channel Decomposition (Section IV)

TADI is a shadow-trained regressor with four channel ablations designed to empirically characterize leakage. It maps observable inputs to per-client sensitive-class concentration estimates (i). The input features include:

Parameter Features (i): Per-round and temporal aggregate signals from the gradients (the controllable mechanism).
Structural/Organizational Features (x i): Client degree, distance to the topology root, betweenness centrality, and organizational labels (the prior-coupling signal).

The TADI decomposition isolates leakage into four channels: parameter-only (A 1), structural (A topo), organizational (A org), and combined (A full).

B. FULCRUM Noise Allocation (Section V)

FULCRUM addresses the core problem of suboptimality in uniform DP-SGD when the federation is asymmetric. The leakage an adversary can extract is bounded additively:

I(p i; i G, omega, sigma j) T max over 2 sigma i squared B squared + i

The goal is to minimize the worst-case bound across all clients subject to a fixed total noise budget U. The optimal allocation is given by Theorem V.3:

sigma* min-max squared = a over K* - i

where a = T max / (2B 2).

Key Properties of FULCRUM:

Strict Improvement: The topology-aware allocation strictly dominates uniform DP-SGD whenever the federation is asymmetric.
Graceful Degeneration: It degenerates exactly to uniform DP-SGD when it is not, making it safe to adopt unconditionally.

The theory was validated across three federated benchmarks: Fed-ISIC2019 (hierarchical, dataset size asymmetry), Fed-Heart-Disease (ring, dataset size asymmetry), and Synthetic CIFAR-10 (eta-coupling).

A. Privacy Gains:

FULCRUM delivers privacy gains of up to 1.967 nats on Fed-ISIC2019 at the strong-privacy regime (T max = 100, U = 0.05.
Figure 3 (Setting C) shows that for asymmetric topologies, the gap grows monotonically in eta, confirming the theoretical predictions of Theorem V.3.

B. Utility Cost:

Table II reports the utility comparison using Two One-Sided Tests (TOST). Across all three settings and all nine T max rows, every TOST p-value is below 0.05, establishing statistical equivalence. This confirms that the privacy improvements are obtained at no practically meaningful utility cost.

C. Realizability of the Bound:

The TADI channel decomposition verifies the additive structure of Theorem V.2: The parameter channel A 1 achieves negative channel lift... confirming that the controllable mechanism term of Theorem V.2 is effectively bounded by the per-client noise allocation under both FULCRUM and uniform DP-SGD.
The prior-coupling supremum (i) is realized when the shadow training distribution matches the target’s deployment prior (e.g, eta=1 in Setting C). Conversely, the supremum is not realised because the shadow prior does not match the target, demonstrating that the theoretical guarantee holds as an upper bound regardless of prior alignment.

The paper concludes that FL's communication topology is a critical information channel currently ignored by existing privacy mechanisms. FULCRUM provides a closed-form, optimal defense against this threat. The authors state: F ULCRUM can therefore be adopted unconditionally: it provides a strict improvement wherever structural asymmetry exists and degenerates gracefully to the uniform baseline where it does not.

Future work is planned to extend the framework to dynamic topologies, evaluate combined defenses with secure aggregation, and integrate structural leverage with per-user privacy preferences.

Improvements for AI systems

As a fastidious AI researcher, I recognize that this paper addresses a critical blind spot in current Federated Learning (FL) privacy protocols. The core improvement is moving from a static, uniform noise allocation to a dynamic, topology-aware one.

Here are the specific improvements and what the resulting improved AI system can do:

The Improvement: Replace the standard DP-SGD noise scheduler (sigma i squared = U/n) with a dynamic, per-client noise allocation mechanism, F U L C R U M. This mechanism calculates the optimal per-client noise scale (sigma i*) as an inverse function of the client’s structural leverage (i), derived from the closed-form solution:

sigma i* = sqrt a over K - i

where a is a constant related to the observation window and batch size, and K is the minimum achievable worst-case mutual information bound under a fixed utility budget U.

What the Improved AI System Can Do:

Achieve Strict Privacy Gains: The system will achieve significantly tighter privacy guarantees (e.g., up to 1.967 nats improvement, as demonstrated on Fed-ISIC2019) compared to uniform DP-SGD, especially in asymmetric cross-silo or hierarchical deployments where some clients are inherently more exposed than others.
Maintain Utility: Crucially, this dynamic allocation maintains model utility without the measurable cost typically associated with enhanced privacy measures.
Actionable Deployment: The system can be deployed unconditionally because if the topology is symmetric (uniform leverage), F U L C R U M degenerates exactly to uniform DP-SGD, ensuring no penalty for using a simple network configurations.

The Improvement: Implement TADI as a diagnostic monitoring module within the federated training pipeline. This is not an attack, but an empirical tool that performs channel decomposition by applying four specific ablations (Parameter, Structural, Organizational).

The Improvement: Implement a dynamic proxy selection module that determines which source of asymmetry (proxy) is dominant in real-time:

Group-Size Proxy (org): Used when organizational grouping is the primary driver (e.g, hierarchical clusters).
Degree Proxy (deg): Used when network topology has non-uniform connectivity (e.g., peer-to-peer gossip).
Dataset-Size Proxy (ds): Used when client dataset size dictates influence (common in cross-silo healthcare).

Abstract

Hierarchical federated learning places regional aggregators between clients and the cloud, so a participant's update is observed only alongside its neighbours'. The concealment this arrangement provides depends on the size of the aggregation region, and regions in operational deployments vary widely. Prevailing practice applies a single noise multiplier to every participant, calibrated for the most exposed region, so every other participant carries more noise than its own exposure requires. We show that this allocation problem admits an explicit solution. We first give a silo-level differential privacy guarantee for the mechanism, then bound the mutual information between a participant's local class distribution and any estimate an observer positioned above the regional tier could form of it, using an adjacency notion matched to the quantity being protected. Minimising the worst-case bound under a fixed utility budget yields a min-max optimal allocation, which we call Fulcrum. The budget it recovers has a closed form we term the exposure dispersion, a measure of how unevenly aggregation weight is concentrated within regions relative to the most exposed one. Because this quantity follows from the region structure and the aggregation weights alone, a practitioner can evaluate it before training begins, and it vanishes precisely when all regions are equally exposed. On image and text classification at epsilon = 0.99, accuracy at a matched worst-case per-client guarantee improves by up to 14.84 and 12.16 percentage points where the dispersion is large, and is exactly zero on a balanced control for which the theory predicts parity.