Decoy Direction Optimization: A Post-Hoc Defense Against LLM Abliteration
cs.LG, cs.CL, cs.CR
Submitted: 2026-09-14
Updated: 2026-09-14
Code: https://github.com/aashiqmuhamed/defending-against-abliteration
License: http://creativecommons.org/licenses/by/4.0/
The gist: Safety guardrails in open-weight language models can be readily bypassed using Refusal Feature Ablation (RFA), a technique that identifies and projects out a linear refusal direction from the
Terminology
Abstract
Safety guardrails in open-weight language models can be readily bypassed using Refusal Feature Ablation (RFA), a technique that identifies and projects out a linear refusal direction from the residual stream, often achieving a high attack success rate (ASR) while preserving model capability. Defending against these attacks typically requires computationally expensive safety finetuning for every new checkpoint. We introduce Decoy Direction Optimization (DDO), a fast, post-hoc weight-editing defense that requires no base-model finetuning. Our approach is based on a simple mechanistic insight: ablation attacks rely on contrastive estimators to find the refusal direction. Rather than trying to hide the true refusal circuitry, DDO actively injects a high-magnitude, nonlinear decoy signal into the network's MLP neurons. When an attacker attempts to locate the refusal direction, the decoy corrupts their estimator, tricking them into ablating a harmless orthogonal feature while the actual safety mechanism remains intact. We prove a spectral bound formalizing this effect and evaluate DDO across six model families, achieving <10% ASR under standard RFA. On Llama-3-8B-Instruct, DDO remains comparable to trained defenses under adaptive multi-phase attacks (65% vs. 58% worst-case ASR) and reduces Heretic weight-level attack ASR from 88.7% to 18%, all at 30 to 450 times lower optimization cost per configuration than the trained baselines.
Sources
- Constitutional AI: Harmlessness from AI Feedback
- Direct Preference Optimization: Your Language Model is Secretly a Reward Model
- HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal
- Representation Engineering: A Top-Down Approach to AI Transparency
- Defending Against Unforeseen Failure Modes with Latent Adversarial Training
- Robust LLM safeguarding via refusal feature adversarial training
- The Llama 3 Herd of Models
- The Geometry of Refusal in Large Language Models: Concept Cones and Representational Independence
- There Is More to Refusal in Large Language Models than a Single Direction
- The Hidden Dimensions of LLM Alignment: A Multi-Dimensional Analysis of Orthogonal Safety Directions
- LLMs Encode Harmfulness and Refusal Separately
- Universal and Transferable Adversarial Attacks on Aligned Language Models
- Yi: Open Foundation Models by 01.AI
- ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Tools
- Gemma 2: Improving Open Language Models at a Practical Size
- Qwen3 Technical Report
- Mistral 7B
- Llama 2: Open Foundation and Fine-Tuned Chat Models
- Surgical, Cheap, and Flexible: Mitigating False Refusal in Language Models via Single Vector Ablation
- Measuring Massive Multitask Language Understanding
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: Local-Consistency Optimisation for Bayesian Neural Networks
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks