WaSA: Wavelet Scattering Autoencoders for Unsupervised Offline Detection of Malicious Clients in Federated Learning

summary

Video file (mp4)

The gist

The gist The WaSA framework proposes a fully unsupervised, offline framework for malicious client detection in Federated Learning by training a denoising autoencoder on clean spectral prototypes to

In short

WaSA proposes a fully unsupervised method to detect malicious clients in Federated Learning offline. It trains a denoising autoencoder on clean data prototypes to learn the structure of benign client representations. Clients are then scored based on their deviation from this learned benign manifold, allowing for detection of various attacks without needing prior knowledge of the threats.

Key concepts

Denoising Autoencoder
This is a neural network trained to reconstruct clean data after it has been intentionally corrupted or 'denoised'. In WaSA, it learns the essential features of benign client data by trying to reconstruct these prototypes. The bottleneck forces the network to capture only the core, low-dimensional structure of normal client embeddings.
Spectral Prototypes
These are summaries of clean spectral data created using variance-weighted principal directions from a rank-reduced Singular Value Decomposition (SVD). These prototypes represent the most important features of benign client data. They serve as the 'ground truth' examples used to train the autoencoder to define what 'normal' looks like.
Wavelet Scattering Transform (WST)
The WST is a mathematical transformation applied to client data embeddings. It helps project the high-dimensional data into a representation where structural deviations caused by malicious clients become more easily measurable. The final client embedding is derived from the coefficients of this transform, capturing how the data scatters across different frequencies.

Terminology used across episodes

This episode discusses

The paper

WaSA: Wavelet Scattering Autoencoders for Unsupervised Offline Detection of Malicious Clients in Federated Learning · Read on arXiv

Alessandro Licciardi

Department of Mathematical Sciences, Politecnico di Torino · National Institute of Nuclear Physics (INFN), Sezione di Torino

Federated Learning (FL) enables collaborative model training without exposing raw data, yet remains vulnerable to malicious clients injecting poisoned datasets. Existing defenses are typically reactive (operating during training) or supervised (requiring prior knowledge of attack types) - assumptions that rarely hold in open-ended IoT deployments. We propose WaSA, a fully unsupervised, offline framework for malicious client detection. Before training, each client transmits a single, privacy-preserving spectral summary of its data to the server. A denoising autoencoder, trained solely on clean auxiliary data, scores clients based on their deviation from the learned benign manifold, identifying corruptions regardless of the attack type. WaSA requires no attack labels, generalizes to unseen anomalies, and introduces a Boltzmann sampling strategy for client selection that smoothly interpolates between hard exclusion and standard aggregation. Experiments on MNIST, CIFAR-10, and CIFAR-100 under diverse attack scenarios (up to 90 % malicious clients) demonstrate consistent improvements over supervised baselines and robust aggregation methods, with negligible communication overhead. Code is available at https://github.com/alelicciardi99/wavelet autoencoders malicious.

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: I'm Tom.

Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.

Tom: Today's paper: "WaSA: Wavelet Scattering Autoencoders for Unsupervised Offline Detection of Malicious Clients in Federated Learning".

Jane: The gist The WaSA framework proposes a fully unsupervised,

Tom: First, who's behind it and why it matters.

Paper summary: Tom: To summarize the WaSA framework, it operates in two distinct phases, an offline phase and a subsequent online evaluation phase.

Jane: In that offline phase, the server approximates the compact manifold of benign client data by minimizing a denoising empirical risk objective over synthetic wavelet scattering transform embeddings derived from an auxiliary dataset.

Lu: Before federated training begins, clients perform a locally optimal rank-reduced projection of their data using a wavelet scattering operator.

Tom: The server then evaluates these representations by computing their reconstruction error, which formally quantifies their structural deviation from the expected benign distribution.

Jane: The core idea is that this setup allows for client detection regardless of the specific type of corruption because it learns what 'clean' looks like based only on clean data.

Meng: It claims this method requires no attack labels, generalizes to unseen anomalies, and introduces a Boltzmann sampling strategy for client selection that interpolates between hard exclusion and uniform sampling via a temperature parameter T.

Jane: That Boltzmann strategy is important because it lets you tune how strictly you filter clients based on their anomaly scores.

Tom: The evaluation shows that embeddings mapping near the learned benign manifold yield an anomaly score E k less than or equal to one, while anomalous embeddings yield E k greater than one.

Lu: And experimental validation across MNIST, CIFAR-ten and CIFAR-one hundred under various attack scenarios demonstrates consistent improvements over supervised baselines.

Jane: Specifically, they show the WaSA-WST backbone yields a clear spectral separation between benign clients and all four corruption types, including structurally subtle attacks like Gaussian blur.

Tom: The results are pretty strong; they achieve ninety-eight point eight percent F1 and one hundred percent recall on CIFAR-ten under the sixty percent corruption scenario when using WaSA-Boltz with T equals zero point five.

Meng: That's a high recall number, but I gotta ask, what about deploying this at scale?

Jane: That's where the next part of the paper gets interesting; it shows that if you use Boltzmann sampling, it recovers near-oracle sampling behavior even when the clean client pool is small.

Conclusion: Tom: So we've looked at the WaSA paper, which is titled "WaSA: Wavelet Scattering Autoencoders for Unsupervised Offline Detection of Malicious Clients in Federated Learning," and the authors are Alessandro Licciardi and his team.

Jane: The main implication here is that we can move away from defenses that depend on knowing exactly what kind of poison attack you're dealing with.

Tom: WaSA shifts the focus to learning the geometry of benign data—the manifold—and scoring clients based on their distance from that learned structure.

Lu: It’s about using unsupervised learning to create a universal filter for client participation in federated learning.

Jane: The Boltzmann sampling strategy is key because it gives you control over the trade-off between being super strict and letting more potentially clean clients participate.

Tom: This framework allows the defense to generalize to attacks that the authors haven't even seen during training, which is a significant step forward in making federated learning more robust in open settings.

More episodes

← Home