Where Does Robustness Live? Neuron-Guided Adaptation for Retrieval-Augmented Language Models

arXiv:2604.02194 · cs.CL, cs.AI · Submitted 2026-04-02 · Read on arXiv

cs.CL, cs.AI

Submitted: 2026-04-02

Updated: 2026-08-31

Terminology

• Instruction Tuning — 40× in this paper · explained in 2 episodes
• Robustness — 38× in this paper · explained in 5 episodes
• Benchmark — 16× in this paper · explained in 2 episodes
• RAG — 16× in this paper · explained in 3 episodes
• Fine-Tuning — 14× in this paper · explained in 5 episodes
• Accuracy — 12× in this paper · explained in 1 episode
• Adaptation — 10× in this paper · explained in 1 episode
• Sparsity — 7× in this paper · explained in 4 episodes
• Distillation — 5× in this paper · explained in 2 episodes
• Retrieval-Augmented Language Models — 5× in this paper
• Alignment — 4× in this paper · explained in 1 episode
• Hallucination — 4× in this paper · explained in 3 episodes
• Ablation — 3× in this paper · explained in 1 episode
• Decoupling — 3× in this paper · explained in 2 episodes
• Integrated Gradients — 3× in this paper
• Large Language Models (LLMs) — 3× in this paper · explained in 6 episodes
• Parameter Efficiency — 3× in this paper · explained in 3 episodes
• Transformer — 3× in this paper · explained in 2 episodes
• Ablation study — 2× in this paper · explained in 1 episode
• Benchmarking — 2× in this paper · explained in 3 episodes
• Chain-of-Thought (CoT) — 2× in this paper · explained in 4 episodes
• Generalization — 2× in this paper · explained in 13 episodes
• Natural Questions — 2× in this paper
• Neuron-guided Robust Instruction Tuning — 2× in this paper
• Activation Sparsity — 1× in this paper · explained in 1 episode
• AdamW Optimizer — 1× in this paper · explained in 1 episode
• Associative Memory — 1× in this paper · explained in 2 episodes
• Computational overhead — 1× in this paper · explained in 2 episodes
• Cooperation — 1× in this paper · explained in 2 episodes
• Core — 1× in this paper · explained in 2 episodes
• End of Text — 1× in this paper
• Few-Shot Learning — 1× in this paper · explained in 4 episodes
• Multi-hop Reasoning — 1× in this paper · explained in 3 episodes
• Retrieval Augmented Generation — 1× in this paper
• Retrieval Augmented Generation (RAG) — 1× in this paper · explained in 4 episodes
• Retrieval-Augmented Generation (RAG) — 1× in this paper · explained in 16 episodes
• Trace — 1× in this paper · explained in 2 episodes
• Verification — 1× in this paper · explained in 2 episodes

Sources

• Mistral 7B
• The Llama 3 Herd of Models
• Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Steps
• SPLADE-v3: New baselines for SPLADE
• The Lazy Neuron Phenomenon: On Emergence of Activation Sparsity in Transformers
• RAT: Retrieval Augmented Thoughts Elicit Context-Aware Reasoning in Long-Horizon Generation
• InstructRAG: Instructing Retrieval-Augmented Generation via Self-Synthesized Rationales
• RECOMP: Improving Retrieval-Augmented LMs with Compression and Selective Augmentation

Related papers

• Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
• Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
• Subliminal Steering: Stronger Encoding of Hidden Signals
• MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
• The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
• Untangling the Mechanisms of Misleading Context in Medical Question Answering