A Universal Vibe? Finding and Controlling Language-Agnostic Informal Register with SAEs

arXiv:2603.26236 · cs.CL · Submitted 2026-03-27 · Read on arXiv

cs.CL

Submitted: 2026-03-27

Updated: 2026-09-01

Terminology

• Steering — 60× in this paper · explained in 2 episodes
• Core — 17× in this paper · explained in 2 episodes
• Perplexity — 10× in this paper · explained in 3 episodes
• Activation Steering — 9× in this paper · explained in 4 episodes
• Abstraction — 7× in this paper · explained in 1 episode
• Feature Extraction — 7× in this paper · explained in 2 episodes
• Interpretability — 7× in this paper · explained in 15 episodes
• Cosine Similarity — 6× in this paper · explained in 3 episodes
• Zero-Shot Transfer — 5× in this paper · explained in 4 episodes
• Sparse Autoencoders (SAEs) — 4× in this paper · explained in 3 episodes
• Ablation — 3× in this paper · explained in 1 episode
• Alignment — 3× in this paper · explained in 1 episode
• Benchmark — 3× in this paper · explained in 2 episodes
• Latent Space — 3× in this paper · explained in 7 episodes
• Sparsity — 3× in this paper · explained in 4 episodes
• Transformer — 3× in this paper · explained in 2 episodes
• Adam — 2× in this paper · explained in 2 episodes
• Clustering — 2× in this paper · explained in 2 episodes
• Consistency — 2× in this paper · explained in 3 episodes
• Generalization — 2× in this paper · explained in 13 episodes
• Mechanistic interpretability — 2× in this paper · explained in 2 episodes
• Pragmatics — 2× in this paper · explained in 2 episodes
• Accuracy — 1× in this paper · explained in 1 episode
• Activation Addition — 1× in this paper · explained in 1 episode
• Activation Patching — 1× in this paper · explained in 2 episodes
• Eleventh International Conference on Learning Representations — 1× in this paper
• Generative AI — 1× in this paper · explained in 2 episodes
• LLM judge — 1× in this paper · explained in 2 episodes
• LLM-as-a-Judge — 1× in this paper · explained in 3 episodes
• Large Language Models (LLMs) — 1× in this paper · explained in 6 episodes
• Latent Representation — 1× in this paper · explained in 2 episodes
• Representation Learning — 1× in this paper · explained in 2 episodes
• Robustness — 1× in this paper · explained in 5 episodes
• Zero-Shot Generalization — 1× in this paper · explained in 5 episodes

Sources

• SAEs Are Good for Steering -- If You Select the Right Features
• Sparse Autoencoders Find Highly Interpretable Features in Language Models
• Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2
• Steering Language Model Refusal with Sparse Autoencoders
• GPT-4 Technical Report
• Automated Annotation with Generative AI Requires Validation
• Qwen2.5 Technical Report
• BLOOM: A 176B-Parameter Open-Access Multilingual Language Model

Related papers

• Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
• Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
• Subliminal Steering: Stronger Encoding of Hidden Signals
• MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
• The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
• Untangling the Mechanisms of Misleading Context in Medical Question Answering