FedRP: A Communication-Efficient Approach for Differentially Private Federated Learning Using Random Projection

arXiv:2509.10041 · cs.LG · Submitted 2026-08-23 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Today's paper: "FedRP: A Communication-Efficient Approach for Differentially Private Federated Learning Using Random Projection".

Jane: FedRP:

Tom: First, who's behind it and why it matters.

Title and authors: Jane: The paper "FedRP: A Communication-Efficient Approach for Differentially Private Federated Learning Using Random Projection" summarizes a very clever mechanism to solve that core tension we just talked about. It’s not just applying noise, which is the standard approach in differential privacy.

Tom: No, they are using this combination of random projection and ADMM optimization. That's the part that sounds like a mathematical power move.

Lu: The paper explains that by applying random projection, they are mapping those high-dimensional model parameters into a much smaller, lower-dimensional space before transmitting them to the center.

Meng: And this is where the "communication-efficient" part hits the mark; we're reducing the volume of data sent dramatically by using a dimensionality reduction technique.

Lalam: But it’s not just simple compression; because they are using ADMM, it drives consensus in that compressed space, which ensures that even though the data is reduced, the collective intelligence of all participants is still aligned.

Tom: It sounds like they've found a way to shrink the information while keeping its structural integrity. So we have this low-dimensional vector sent to the center, but how does that guarantee privacy?

Jane: The paper makes it clear that even though they are using this projection, the randomness is key; because the central server doesn't have access to a shared secret matrix, it cannot reconstruct the original parameters.

Lu: That lack of access means an attacker can't easily pull sensitive information out of the shared vector without solving a highly complex problem, which is where their (epsilon, delta) guarantee comes in.

Meng: It’s much more robust than just relying on random noise because we are actually changing the structure of the all model updates themselves.

Lalam: This paper demonstrates that by combining these two concepts—privacy and dimensionality reduction—we are moving toward a new standard for secure, decentralized learning environments.

The paper's summary: Tom: So, "FedRP: A Communication-Efficient Approach for Differentially Private Federated Learning Using Random Projection" solves the problem of sending massive gradients, but what does it actually *deliver* better than existing solutions?

Jane: It delivers a triple win, Tom. The paper shows that FedRP maintains high model accuracy—comparable to standard methods like FedAvg—while drastically cutting communication costs.

Lu: It' also offers a very strong (epsilon, delta) differential privacy guarantee that is theoretically quantified by the authors using specific values for epsilon and delta.

Meng: And importantly, it outperforms the existing "FedAvg+DP" approach in every test they ran, which is huge because DP often comes with a performance hit.

Lalam: The implications here are massive; we are seeing proof that we don't have to choose between strong privacy and high accuracy anymore.

Tom: It's not just about the theoretical gains, however, practical improvements matter too. Does the paper show it resists real-world attacks?

Jane: Yes, they demonstrate resilience against data reconstruction attacks because of how they structure the random projection mechanism itself.

Lu: The fact that it allows us to quantify epsilon is a big deal; we can finally measure exactly how private our AI is, rather than just saying "it's protected."

Meng: And the communication reduction isn' huge, especially with models like VGG16 where the original parameters are huge vectors.

Lalam: This paper suggests that we have found a viable path forward where security and performance aren’t mutually exclusive for global AI systems.

The paper's improvements: Tom: We’ve covered so much ground, but to wrap up, let's look at the overall message of "FedRP: A Communication-Efficient Approach for Differentially Private Federated Learning Using Random Projection." What is the final word on this paper?

Jane: The authors have successfully shown that we don't need to sacrifice performance or speed when trying to achieve robust differential privacy in federated learning.

Lu: I think the big picture here is that by merging random projection with ADMM, they've created a foundation for extremely scalable and secure AI systems globally.

Meng: From an engineering viewpoint, it provides a concrete blueprint for building decentralized networks that are both fast and secure.

Lalam: This paper gives us confidence in the future of global AI, proving that user privacy does not need to be an obstacle to collaborative progress.

Tom: It's definitely a massive step forward for the entire field. So, as we conclude our discussion on "FedRP: A Communication-Efficient Approach for Differentially Private Federated Learning Using Random Projection," thank you all for joining us!

Lu: It’s exciting to see the theoretical proof backing such practical applications.

Meng: I'm ready to implement this approach in a real-world deployment right now.

Lalam: I believe this work will help build a more trustworthy and efficient digital culture for everyone.

Conclusion: Tom: So, we’ve spent time diving into everything this paper is doing, and it's clear that FedRP: A Communication-Efficient Approach for Differentially Private Federated Learning Using Random Projection is doing something truly remarkable.

Jane: It’s really about making sure that strong protection for user privacy doesn't have to come at the expense of good model performance, which is a huge win for everyone involved.

Lu: I think the creative way they’ve managed to harmonize the ADMM framework with random projection opens up so many new possibilities for how distributed systems can evolve globally.

Meng: And from my perspective, seeing that massive reduction in communication costs makes it immediately viable for real-world deployment on large-scale IoT networks.

Lalam: The ability to quantify this level of privacy is what I find most meaningful, allowing us to build a more trustworthy and ethical digital culture moving forward.

Tom: It’s a testament to the fact that robust security can coexist with impressive speed, which is something we hadn't seen on this scale before.

Jane: It feels like we are finally seeing a solution that truly addresses the core challenges in decentralized AI systems.

Lu: I’m just thrilled by how far this has pushed the boundaries of what seems technically possible.

Meng: I'm looking forward to seeing how fast this translates into production-level efficiency gains.

Lalam: It makes me feel optimistic about the potential for a more equitable and private future.

Tom: Well, with all that said, we’re going to take a quick break, but then we’ll be back with another fascinating piece of research.

Mohammad Hasan Narimania, Mostafa Tavassolipoura

University of Tehran, Iran

cs.LG

Submitted: 2026-08-23

Updated: 2026-08-25

Code: https://github.com/mhnarimani/FedRP

Importance score: 81/100

Key concepts

Random Projection
This technique maps high-dimensional model parameters into a much smaller, lower-dimensional space before they are sent to the central server. This dimensionality reduction dramatically reduces the volume of data that needs to be transmitted, making communication more efficient.
Differential Privacy (DP)
The paper uses this framework to ensure privacy. The randomness in the projection prevents the central server from reconstructing original parameters, meaning an attacker cannot easily pull sensitive information out of the shared vectors without solving a highly complex problem.
ADMM Optimization
Alternating Direction Method of Multipliers (ADMM) is used to drive consensus within the compressed space. This ensures that even though the data is reduced in size, the collective intelligence of all participating nodes remains aligned and consistent.
(epsilon, delta) Guarantee
This provides a quantifiable measure of privacy. The authors can specify exact values for epsilon and delta, allowing researchers to precisely measure how private the AI system is rather than just stating that it is protected.

Terminology

Summary

Summary of FedRP: A Communication-Efficient Approach for Differentially Private Federated Learning Using Random Projection

Federated learning (FL) is a paradigm where the model is brought to the data, allowing collaborative training while maintaining user privacy by preventing raw data transfer. However, FL faces critical challenges, including high communication costs associated with transmitting large model updates, vulnerability to data reconstruction attacks from shared gradients, and issues arising from statistical heterogeneity in local datasets.

This paper introduces a novel algorithm called FedRP (Federated Random Projection), which addresses the dual challenges of privacy preservation and communication cost reduction by integrating random projection techniques with the Alternating Direction Method of Multipliers (ADMM) optimization framework.

The FedRP Methodology

The core idea of FedRP is to use random projection to reduce the dimensionality of model parameters before transmission. This process creates a one-way information bottleneck, making it computationally infeasible for an attacker or the server to reconstruct the original model parameters without access to a secret matrix.

  1. Framing the Problem: The FL objective is framed within the context of a consensus problem, where K clients aim to find a global model parameter vector that minimizes local objective functions f i subject to the constraint w i =.

  2. The ADMM Framework: The optimization is solved using the Augmented Lagrangian, which includes local updates, global updates, and dual variable updates.

  3. The FedRP Mechanism: In each training round t:

  • Clients first update their local model w i+1.

  • A new random projection matrix A t+1 is generated (using a shared seed unknown to the server).

  • Each client computes the compressed vector z t+1 = A w i.

  • These vectors are sent to the central server, which aggregates them as t+1 = 1 over K sum z i.

  • The process repeats until convergence, ensuring that the projected parameters A w i become approximately equal to the global vector.

Theoretical Guarantees and Privacy Analysis

FedRP provides a strong (epsilon, delta) -differential privacy guarantee. The theoretical analysis confirms that the algorithm is (epsilon, delta) -DP with a specific value for epsilon derived from the L2 sensitivity and the reduced dimensionality (m). The paper demonstrates that the lower the value of epsilon, the better the privacy of our algorithm is preserved.

The method's structure resists data reconstruction attacks because, instead of sending full model parameters, clients send only a dimension-reduced vector. Due to this random mapping, it is impossible for an attacker or the server to reconstruct... an error-free solution for estimating these values, thus effectively resisting common data reconstruction threats.

Experimental Results

The algorithm was tested using three datasets (MNIST, CIFAR-10, and CIFAR-100) across various model complexities (LeNet-5, ResNet-18, VGG16).

  • Model Performance: FedRP maintains high model accuracy and performs better than the conventional FedAvg+DP algorithm. The dimensionality of the random projection matrix (m) was found not to exert a significant impact on performance.

  • Privacy Preservation: Table 3 shows that the proposed algorithm consistently achieves a lower epsilon value compared to the FedAvg+DP (noise-based) approach, ensuring stronger privacy guarantees.

  • Communication Costs: The reduction in communication cost is substantial. For example, when using the VGG16 model (138M parameters), FedAvg sends 512 MB per round, whereas FedRP sends only 4 KB per round for a random projection dimension of m=5. This represents a significant reduction in bandwidth required.

In conclusion, FedRP achieves a trifecta of benefits: it maintains high model accuracy comparable to conventional algorithms like FedAvg, drastically reduces communication costs, and provides quantifiable privacy enhancements.

Improvements for AI systems

Based on the rigorous analysis of the FedRP framework, here are the specific improvements to AI systems and what these improved systems can achieve:

  • Mechanism: By utilizing random projection (mapping high-dimensional parameters w via matrix A into a lower dimension) before transmission, the system drastically reduces the volume of data transmitted per client per round.

  • Improvement: This overcomes the critical communication bottleneck inherent in traditional Federated Learning (FL). Systems can now support a far larger number of distributed clients (e.g., millions of IoT devices or mobile phones) without network saturation, making large-scale cross-device deployment practical.

  • Mechanism: Unlike conventional Differential Privacy (DP) methods that inject noise into parameters (which often destroys model utility), FedRP employs a structural information bottleneck—the random projection matrix A. This ensures the core privacy requirement is met: it is computationally infeasible for an attacker to reconstruct the original, high-dimensional model parameters from the compressed, low-dimensional vector z.

  • Improvement: The system achieves verifiable (epsilon, delta) -Differential Privacy guarantees. This provides a quantifiable level of privacy protection against advanced data reconstruction attacks (e.g., DLG or DLM), ensuring that user data is protected even when gradients are exposed.

  • Mechanism: The integration of the Alternating Direction Method of Multipliers (ADMM) allows the system to find a consensus on the projected parameters within the reduced space, while simultaneously minimizing local loss functions (f i(w)). This process is stable and effective even when applied to complex, high-parameter models (like VGG16 or ResNet-18).

  • Improvement: The system maintains model accuracy comparable to non-private baseline methods (FedAvg), successfully mitigating the significant performance degradation typically associated with noise-based privacy techniques.

The improved system, utilizing the FedRP architecture, can execute the following tasks with unprecedented efficiency and security:

  1. Deploy Ultra-Large Scale Collaborative Training: It enables collaborative model training across massive, heterogeneous networks of decentralized devices (e.g., millions of mobile phones or smart sensors) where bandwidth and computational constraints are severe.

  2. Perform Sensitive Data Analysis Safely: It facilitates the analysis of highly sensitive data (e.g., medical records, private user behavior) without ever compromising the underlying privacy, as the model updates are structurally protected against reconstruction attempts.

  3. Achieve Optimal Trade-offs: It provides a quantifiable solution to the classic AI dilemma: it achieves a superior balance between high predictive accuracy, minimal communication overhead, and strong, verifiable privacy guarantees.

Sources

Related papers