Rewired or Gated? How Instruction Tuning Shapes Knowledge-Conflict Circuits in LLMs
cs.LG, cs.CL
Submitted: 2026-09-22
Updated: 2026-09-22
Comments: Accepted at BlackboxNLP 2026, Co-located with EMNLP 2026
Code: https://github.com/IvLabs/Rewired-or-Gated
License: http://creativecommons.org/licenses/by/4.0/
The gist: In language models, the choice between believing the prompt and believing the weights is made by a handful of identifiable attention heads.
Terminology
Abstract
In language models, the choice between believing the prompt and believing the weights is made by a handful of identifiable attention heads. Instruction tuning changes how models behave under conflict, but whether it rewires the underlying circuit or merely gates/reweights already present components, remains unknown. We provide the first mechanistic base-vs-instruct comparison of conflict-resolution circuits, across three families (Llama-3.2-3B, Qwen-2.5-3B, Gemma-3-4B). Five independent methods, node and edge attribution, superposition role analysis, causal ablation, and path patching, converge on gating, with the same heads, in the same late-layers, are found to be reweighted rather than replaced with a high node overlap (0.60-0.82). Behaviorally, tuning shifts models toward parametric memory, making instruct models reject a terse counterfactual context far more than base ones, the opposite of a naive user-following expectation. Yet this added skepticism is a factor of framing since it disappears when the same false claim is delivered as a coherent, evidential passage. The robustness that instruction tuning buys against terse injection is therefore real but narrow. More broadly, we believe that because the conflict circuit is preserved rather than rebuilt, interpretability and control tools calibrated on base models should transfer directly to their deployed instruct siblings.
Sources
- Studying Large Language Model Behaviors Under Context-Memory Conflicts With Real Documents
- Mechanistic Interpretability as Statistical Estimation: A Variance Analysis
- Gemma 3 Technical Report
- The Llama 3 Herd of Models
- Qwen2.5 Technical Report
- EAP-GP: Mitigating Saturation Effect in Gradient-based Automated Circuit Identification
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: Local-Consistency Optimisation for Bayesian Neural Networks
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks