The Dialect Tax: Dialectal Biases Persist throughout the Language Modeling Pipeline

arXiv:2608.24952 · cs.CL, cs.AI, cs.LG · Submitted 2026-08-24 · Read on arXiv

cs.CL, cs.AI, cs.LG

Submitted: 2026-08-24

Updated: 2026-08-24

Code: https://github.com/socialnlp/dialecttax

Terminology

• Tokenization — 63× in this paper · explained in 2 episodes
• Accuracy — 16× in this paper · explained in 1 episode
• Cosine Similarity — 15× in this paper · explained in 3 episodes
• Benchmark — 13× in this paper · explained in 2 episodes
• Alignment — 4× in this paper · explained in 1 episode
• Perplexity — 4× in this paper · explained in 3 episodes
• Transformer — 3× in this paper · explained in 2 episodes
• Adam — 2× in this paper · explained in 2 episodes
• Embeddings — 2× in this paper · explained in 2 episodes
• Representation Learning — 2× in this paper · explained in 2 episodes
• Semantic Similarity — 2× in this paper · explained in 2 episodes
• Trace — 2× in this paper · explained in 2 episodes
• Vector Representations — 2× in this paper · explained in 2 episodes
• Ablation — 1× in this paper · explained in 1 episode
• African American Vernacular English — 1× in this paper
• Calibration — 1× in this paper · explained in 7 episodes
• Foundation Model — 1× in this paper · explained in 3 episodes
• Foundation Models — 1× in this paper · explained in 7 episodes
• Instruction Tuning — 1× in this paper · explained in 2 episodes
• Low-Resource Languages — 1× in this paper · explained in 4 episodes
• Pragmatics — 1× in this paper · explained in 2 episodes
• Regularization — 1× in this paper · explained in 3 episodes
• Robustness — 1× in this paper · explained in 5 episodes
• Standard American English — 1× in this paper
• Transfer Learning — 1× in this paper · explained in 10 episodes
• Verifier — 1× in this paper · explained in 2 episodes

Sources

• Do All Languages Cost the Same? Tokenization in the Era of Commercial Language Models
• Why do language models perform worse for morphologically complex languages?
• A Bit of a Problem: Measurement Disparities in Dataset Sizes Across Languages
• Program Synthesis with Large Language Models
• Large Language Models Reflect the Ideology of their Creators
• FOLIO: Natural Language Reasoning with First-Order Logic
• Evaluating Large Language Models Trained on Code
• Training Verifiers to Solve Math Word Problems
• BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
• Natural Language Processing for Dialects of a Language: A Survey
• The Llama 3 Herd of Models
• Scale Dependent Data Duplication
• Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidates
• SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing
• Matryoshka Representation Learning
• Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
• Graph-enhanced Large Language Models in Asynchronous Plan Reasoning
• CoQA: A Conversational Question Answering Challenge
• Tokenization Is More Than Compression
• Neural Machine Translation of Rare Words with Subword Units

Related papers

• Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
• Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
• Subliminal Steering: Stronger Encoding of Hidden Signals
• MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
• The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
• Untangling the Mechanisms of Misleading Context in Medical Question Answering