What Drives Representation Steering? A Mechanistic Case Study on Steering Refusal
cs.LG, cs.AI, cs.CL
Submitted: 2026-04-09
Updated: 2026-09-01
Comments: EMNLP 2026 Main Conference. Updated from previous preprint to contain experiments on Qwen 3 8B, revised explanation of attribution patching method, and additional results in appendix
License: http://creativecommons.org/licenses/by/4.0/
The gist: Applying steering vectors to large language models (LLMs) is an efficient and effective model alignment technique, but we lack an interpretable explanation for how it works--specifically, what
Terminology
Abstract
Applying steering vectors to large language models (LLMs) is an efficient and effective model alignment technique, but we lack an interpretable explanation for how it works--specifically, what internal mechanisms steering vectors affect and how this results in different model outputs. To investigate the causal mechanisms underlying the effectiveness of steering vectors, we conduct a comprehensive case study on refusal. We propose a multi-token activation patching framework and discover that different steering methodologies leverage functionally interchangeable circuits when applied at the same layer. These circuits reveal that steering vectors primarily interact with the attention mechanism through the OV circuit while largely ignoring the QK circuit. Freezing all attention scores during steering drops performance by only 8.83% across three model families. A mathematical decomposition of the steered OV circuit further reveals semantically interpretable concepts, even in cases where the steering vector itself does not. Leveraging the activation patching results, we show that steering vectors can be sparsified by up to 85-96% while retaining most performance, and that different steering methodologies agree on a subset of important dimensions.
Sources
- Catastrophic Jailbreak of Open-source LLMs via Exploiting Generation
- Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Models
- Persona Vectors: Monitoring and Controlling Character Traits in Language Models
- Direct and Indirect Effects
- The Llama 3 Herd of Models
- Gemma 2: Improving Open Language Models at a Practical Size
- Steering Language Models With Activation Engineering
- Understanding Reasoning in Thinking Language Models via Steering Vectors
- A StrongREJECT for Empty Jailbreaks
- HyperSteer: Activation Steering at Scale with Hypernetworks
- Representation Engineering: A Top-Down Approach to AI Transparency
- Universal and Transferable Adversarial Attacks on Aligned Language Models
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: Local-Consistency Optimisation for Bayesian Neural Networks
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks