Late Attention Layers Alone Can Copy Entity Tokens, but Not Without Attending to Their Context
cs.CL
Submitted: 2026-09-28
Updated: 2026-09-28
Code: https://github.com/mitkox/Thinking-with-Visual-Primitives
Terminology
Sources
- Conditional Memory via Scalable Lookup: A New Axis of Sparsity for Large Language Models
- How do Language Models Bind Entities in Context?
- Do I Know This Entity? Knowledge Awareness and Hallucinations in Language Models
- The Dual-Route Model of Induction
- Thinking to Recall: How Reasoning Unlocks Parametric Knowledge in LLMs
- Mixing Mechanisms: How Language Models Retrieve Bound Entities In-Context
- Locating and Editing Factual Associations in GPT
- Circuit Component Reuse Across Tasks in Transformer Language Models
- LLM Circuit Analyses Are Consistent Across Training and Scale
- Attention Is All You Need
- Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small
- Qwen3 Technical Report
- Towards Best Practices of Activation Patching in Language Models: Metrics and Methods
Related papers
- Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
- Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
- Subliminal Steering: Stronger Encoding of Hidden Signals
- MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
- The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
- Untangling the Mechanisms of Misleading Context in Medical Question Answering