Defense via Consensus: Mitigating Subliminal Learning through Multi-Teacher Distillation
summary
The gist
The gist: Consensus decoding suppresses source-specific misbehavior while preserving shared desirable behavior across multiple datasets by aggregating next-token distributions from separately
In short
The system uses consensus decoding to defend against source-specific misbehavior in multi-teacher distillation settings. It aggregates next-token distributions from multiple fine-tuned reference models to suppress costs unique to individual sources while preserving shared, desirable behaviors across the ensemble.
Key concepts
- Consensus Decoding
- A method during generation where the final output token is chosen based on a consensus among multiple source models. This process aims to filter out errors or malicious outputs specific to one source by requiring agreement from several sources, thereby preserving the general, shared correct behavior.
- Token-wise Minimum Decoder
- One of two proposed decoding strategies where each individual token is capped at the lowest probability assigned by any of the source models. This mechanism aggressively suppresses tokens that are highly likely to be specific errors or poisoned outputs from a single source.
- Base-Relative Variant
- A second consensus decoder that focuses on changes relative to a common base model. It keeps the smallest change when all sources agree on a direction but reverts to the base probability if sources disagree, helping maintain fidelity to the original, desired behavior.
- Quorum Aggregation
- An extension of consensus that replaces strict unanimity with a support threshold. A token is accepted if it receives support from more than a predefined number of sources, ensuring robustness even when some sources provide conflicting or weak signals.
Terminology used across episodes
This episode discusses
- Defense via Consensus: Mitigating Subliminal Learning through Multi-Teacher Distillation · Paper Radio
- Subliminal Effects in Your Data: A General Mechanism via Log-Linearity
- Semantic Smoothing for Language Models via Distribution Estimation and Embeddings
- Weird Generalization and Inductive Backdoors: New Ways to Corrupt LLMs
- Steering Out-of-Distribution Generalization with Concept Ablation Fine-Tuning
- Towards Poisoning Robustness Certification for Natural Language Generation
- CharED: Character-wise Ensemble Decoding for Large Language Models
- M-Ped: Multi-Prompt Ensemble Decoding for Large Language Models
- Consensus Sampling for Safer Generative AI
- Tulu 3: Pushing Frontiers in Open Language Model Post-Training
- The Consensus Trap: Rescuing Multi-Agent LLMs from Adversarial Majorities via Token-Level Collaboration
- Assessing Domain-Level Susceptibility to Emergent Misalignment from Narrow Finetuning
- Learning to Decode Collaboratively with Multiple Language Models
- Poisoning Attacks on LLMs Require a Near-constant Number of Poison Samples
- Semantic Consensus Decoding: Backdoor Defense for Verilog Code Generation
- PECAN: A Deterministic Certified Defense Against Backdoor Attacks
The paper
Defense via Consensus: Mitigating Subliminal Learning through Multi-Teacher Distillation · Read on arXiv
Adhyyan Narang, Artin Tajdini, Claire Zhang, Jamie Morgenstern
University of Washington
Fine-tuning can introduce unwanted behaviors through deliberate poisoning or unexpected generalization from seemingly benign data. Because these effects may be difficult to anticipate or detect, we seek robustness in the fine-tuning process itself. Our premise is that datasets independently collected for the same task would share the intended capability while differing in their unwanted effects. This creates an opportunity to learn from agreement across sources while reducing dependence on any individual source. We introduce Multi-Source Consensus Distillation (MSCD): we fine-tune one model on each dataset, use agreement among these models to generate replacement training responses, and train a fresh student on the resulting data. We develop minimum and base-relative rules for combining the models' predictions, with extensions that accommodate useful behaviors learned by only some models or expressed in different words. Experiments on controlled poisoning, subliminal learning, and emergent misalignment show suppression of unwanted behavior while retaining shared benefits. In a practical language-understanding task, MSCD retains intent-classification capability while reducing unsafe medical advice introduced during fine-tuning. Comparing distilled students with direct consensus decoding shows that distillation closely preserves the measured desirable and undesirable behavior rates while replacing multiple model evaluations per generated token with a single student evaluation.
Transcript
Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.
Tom: We're looking at the paper: "Defense via Consensus".
Jane: The gist: Consensus decoding suppresses source-specific misbehavior while preserving shared desirable behavior across multiple datasets by aggregating next-token distributions from separately fine-tuned reference models.
Tom: First, who's behind it and why it matters.
Title and authors: Tom: So we’re starting with the title and authors of this paper. It’s "Defense via Consensus: Mitigating Subliminal Learning through Multi-Teacher Distillation" by Adhyyan Narang, Artin Tajdini, and Claire Zhang >
Jane: The authors are researchers at a University of Washington and they’re tackling something called subliminal learning attacks, which is when poisoned data tries to teach the model things without leaving an obvious signature in the input data itself >
Lu: The core idea they are proposing is using inference-time consensus for mitigating hidden behaviors from LLM fine-tuning >
Meng: So they are not trying to fix the training process directly, but rather they fine-tune a separate reference model on each source’s dataset and aggregate their next-token distributions at decoding time >
Lalam: The paper is exploring whether collecting multiple datasets from different sources for the same task can help us prevent vulnerability to malicious actors >
The paper's summary: Tom: So what they are summarizing is that standard defenses like data filtering or mixing in harmless data don't eliminate these hidden preferences, and even KL regularization toward a collection of reference models averages their distributions so behavior carried by one source is diluted but never vetoed >
Jane: They show that the structural asymmetry between the desired and poisoned behavior is captured by assumption A2, which means the desired behavior is shared across more sources than any single cost >
Lu: The paper argues that a shared cost would require aligned attack objectives and coordination overhead, making it less likely in practice >
Meng: They show that fine-tuning one reference model per source and keeping only what their next-token distributions agree on removes the source-specific behavior >
Lalam: Basically, the defense rests on the fact that no single source can lift a token’s probability on its own, so the aggregate stays safe unless a majority of sources teaches the same misbehavior >
The paper's improvements: Tom: They introduce two specific consensus decoders for aggregation during decoding, which are called a token-wise minimum and a base-relative variant >
Jane: The token-wise minimum caps each token at the lowest probability any source assigns, while the base-relative variant reverts to the base probability on any token where sources move in opposing directions >
Lu: These decoders are compared against union training, which trains one adapter on all pooled source data, and LoRA weight merging, which averages the source adapter updates >
Meng: The paper validates this across three settings: explicit prefix poisoning, subliminal learning, and emergent misalignment >
Lalam: In the explicit prefix poisoning setting, they found that both union training and weight averaging retain both the shared joke behavior and the private prefixes >
Conclusion: Tom: So to wrap up on "Defense via Consensus: Mitigating Subliminal Learning through Multi-Teacher Distillation," they’re showing that consensus decoding suppresses source-specific misbehavior while preserving shared desirable behavior >
Jane: They confirm that even union training and weight averaging can retain the unwanted behavior, but their consensus decoders are more effective at separating the two at decoding time >
Lu: The token-wise minimum generally retains more of the shared behavior, and base-relative minimum suppresses changes that don't agree across sources more strongly >
Meng: They also extended this idea to handle partial support with quorum aggregation, where a behavior is retained if it’s supported by strictly more sources than any single cost >
Lalam: And they even introduced semantic smoothing to handle surface-form disagreement by embedding continuations and comparing them using cosine similarity to generate pseudo-support before token-level quorum aggregation >
Tom: So the conclusion is that consensus measures support, not correctness, and that when useful behavior repeats across sources but unwanted behavior remains source-specific, consensus can separate those two at decoding time >
More episodes
- 2607.25182-EKLAV: Elicited Key CoT Learning via Answer only superVision
- 2608.05587-Q-Reflector: Learning GUI Action Reflection with Generated Conditions
- 2608.17895-BEAR-Bench: A Bilingual Enterprise and Academic Reasoning Benchmark for Multimodal Models
- 2608.23215-BenthicDINO: Physics-Informed Self-Distillation for View-Invariant Side-Scan Sonar Representations
- 2607.26710-PowerAtlas: Feasibility-Aware Language Models for Electricity-Computing Co-Scheduling
- 2608.14408-Online Inference in Distributional Temporal-Difference Learning
- 2608.20338-ConceptGuard: Benchmarking Context-Sensitive Unlearning in Large Language Models
- 2607.26232-BG-REAL: A Public Real-Data Anchored Benchmark for Background Manipulation Detection and Localization
- 2608.20870-RDANet: Relative Degradation Aware Network for Infrared Small Target Detection
- 2607.23258-CAPT: A Multi-task Continuous Autoregressive Transformer enabling Cross-dataset and Cross-species Transfer for Calcium Population Dynamics