First-Token Broadcasters: Mechanistic Origins of Language Identity and Distributed Robustness in Transformers

arXiv:2606.22361 · cs.CL, cs.AI · Submitted 2026-08-19 · Read on arXiv

Arjun Pillai, Christian Hoang, Anjelo Jann Laroza

cs.CL, cs.AI

Submitted: 2026-08-19

Updated: 2026-08-20

Terminology

Ablation — 24× in this paper · explained in 1 episode
Accuracy — 16× in this paper · explained in 1 episode
Instruction Tuning — 8× in this paper · explained in 2 episodes
Interpretability — 7× in this paper · explained in 15 episodes
Transformer — 7× in this paper · explained in 2 episodes
Mechanistic interpretability — 6× in this paper · explained in 2 episodes
Redundancy — 6× in this paper · explained in 2 episodes
Robustness — 4× in this paper · explained in 5 episodes
Attention Entropy — 2× in this paper · explained in 2 episodes
Clustering — 2× in this paper · explained in 2 episodes
Language Identity Head Ablation — 2× in this paper
Steering — 2× in this paper · explained in 2 episodes
Benchmark — 1× in this paper · explained in 2 episodes
Consistency — 1× in this paper · explained in 3 episodes
Generalizability — 1× in this paper · explained in 6 episodes
Generalization — 1× in this paper · explained in 13 episodes
Grouped-Query Attention — 1× in this paper
Linear Probing — 1× in this paper · explained in 2 episodes
Representation Engineering — 1× in this paper · explained in 2 episodes
Representation Learning — 1× in this paper · explained in 2 episodes

Sources

Related papers