Beyond Dense States: Sparse Transcoders as Causally Testable Operators for LLM Latent Reasoning
cs.AI, cs.LG
Submitted: 2026-02-02
Updated: 2026-08-30
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Terminology
Sources
- Training Large Language Models to Reason in a Continuous Latent Space
- Compressed Chain of Thought: Efficient Reasoning Through Dense Representations
- DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
- Implicit Chain of Thought Reasoning via Knowledge Distillation
- From Explicit CoT to Implicit CoT: Learning to Internalize CoT Step by Step
- OpenAI o1 System Card
- Towards System 2 Reasoning in LLMs: Learning How to Think With Meta Chain-of-Thought
- A Survey on Latent Reasoning
- Transcoders Beat Sparse Autoencoders for Interpretability
- Solving General Arithmetic Word Problems
- CODI: Compressing Chain-of-Thought into Continuous Space via Self-Distillation
- Think Silently, Think Fast: Dynamic Latent Compression of LLM Reasoning Chains
- The Llama 3 Herd of Models
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems
- VSAL: A Vision Solver with Adaptive Layouts for Graph Property Detection