Nameless Tokenization: A Lossless Tokenizer-Level Defense Against Control-Token Forgery in Open-Weight LLMs
cs.CL
Submitted: 2026-09-15
Updated: 2026-09-15
Comments: preprint
License: http://creativecommons.org/licenses/by/4.0/
The gist: Open-weight language models publish the strings their chat templates use to mark turns, roles and tool results, which the tokenizer maps back to the reserved identifiers the model obeys.
Terminology
Abstract
Open-weight language models publish the strings their chat templates use to mark turns, roles and tool results, which the tokenizer maps back to the reserved identifiers the model obeys. Anyone who controls text in a prompt can therefore write a turn boundary indistinguishable from one the serving stack wrote. We audit 256 deployed chat tokenizers. All are forgeable, and the flag usually recommended as a fix leaves 56.6% forgeable because it misses the tool and reasoning markers agent systems rely on. We propose nameless tokenization, which leaves the control entries with a reserved identifier and no surface string, so the content encoder cannot emit one and message content reaches the model unaltered. Across five tokenizer families it reproduces the standard token stream exactly on attack-free data and lifts accuracy on a probe of delimiter-bearing text from 8.5% to 59.9%, where sanitizers lose it. Separating a delimiter's appearance from its identifier shows the identifier matters little against a bare task instruction, but carries most of a forged tool result and most of any forged turn once the system message tells the model to treat user content as data.
Sources
- Defending Against Indirect Prompt Injection Attacks With Spotlighting
- Formalizing and Benchmarking Prompt Injection Attacks and Defenses
- Ignore Previous Prompt: Attack Techniques For Language Models
- StruQ: Defending Against Prompt Injection with Structured Queries
- The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructions
- Instructional Segment Embedding: Improving LLM Safety with Instruction Hierarchy
- Can LLMs Separate Instructions From Data? And What Do We Even Mean By That?
- ASIDE: Architectural Separation of Instructions and Data in Language Models
Related papers
- Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
- Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
- Subliminal Steering: Stronger Encoding of Hidden Signals
- MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
- The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
- Untangling the Mechanisms of Misleading Context in Medical Question Answering