Federated Adversarial Training with Transformers
cs.LG, cs.CR, cs.CV
Submitted: 2022-06-05
Updated: 2026-08-27
License: http://creativecommons.org/licenses/by/4.0/
The gist: Federated learning (FL) has emerged to enable global model training over distributed clients' data while preserving its privacy.
Terminology
Abstract
Federated learning (FL) has emerged to enable global model training over distributed clients' data while preserving its privacy. However, the global trained model is vulnerable to the evasion attacks especially, the adversarial examples (AEs), carefully crafted samples to yield false classification. Adversarial training (AT) is found to be the most promising approach against evasion attacks and it is widely studied for convolutional neural network (CNN). Recently, vision transformers have been found to be effective in many computer vision tasks. To the best of the authors' knowledge, there is no work that studied the feasibility of AT in a FL process for vision transformers. This paper investigates such feasibility with different federated model aggregation methods and different vision transformer models with different tokenization and classification head techniques. In order to improve the robust accuracy of the models with the not independent and identically distributed (Non-IID), we propose an extension to FedAvg aggregation method, called FedWAvg. By measuring the similarities between the last layer of the global model and the last layer of the client updates, FedWAvg calculates the weights to aggregate the local models updates. The experiments show that FedWAvg improves the robust accuracy when compared with other state-of-the-art aggregation methods.
Sources
- Advances in adversarial attacks and defenses in computer vision: A survey
- Reveal of Vision Transformers Robustness against Adversarial Attacks
- Conditional Positional Encodings for Vision Transformers
- Adversarial Attacks on ML Defense Models Competition
- Video Super-Resolution Transformer
- Certifiably-Robust Federated Adversarial Learning via Randomized Smoothing
- Transformer in Transformer
- Gaussian Error Linear Units (GELUs)
- Federated Robustness Propagation: Sharing Robustness in Heterogeneous Federated Learning
- Federated Learning for Resource-Constrained IoT Devices: Panoramas and State-of-the-art
- Survey of Personalization Techniques for Federated Learning
- Network In Network
- Federated Learning Meets Natural Language Processing: A Survey
- Comprehensive Privacy Analysis of Deep Learning: Passive and Active White-box Inference Attacks against Centralized and Federated Learning
- Threats to Federated Learning: A Survey
- Adversarial Attacks are Reversible with Natural Supervision
- Towards Robust Vision Transformer
- Rethinking Architecture Design for Tackling Data Heterogeneity in Federated Learning
- Adversarial training in communication constrained federated learning
- Data Poisoning Attacks on Federated Machine Learning
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: Local-Consistency Optimisation for Bayesian Neural Networks
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks