What Is The Political Content in LLMs' Pre- and Post-Training Data?
cs.CL, cs.AI, cs.CY
Submitted: 2025-09-26
Updated: 2026-09-21
Comments: 9 pages, under review
License: http://creativecommons.org/licenses/by/4.0/
The gist: Large language models (LLMs) reflect politically-slanted opinions in their generated text.
Terminology
Abstract
Large language models (LLMs) reflect politically-slanted opinions in their generated text. Even though it is widely assumed that model behavior stem from training data, there has been no study quantifying the extent to which political content is part of the training data. To bridge this gap, we aim to directly estimate (1) the proportion of politically engaged texts in training data, (2) respective data imbalance, (3) cross-dataset similarity, and (4) correlations between data composition and model behaviour. We analyze the political content of pre- and post-training datasets of open-source LLMs, combining large-scale sampling, political-leaning classification, and stance detection. We find that all LLM training datasets are systematically skewed towards left-leaning content, with pre-training containing more politically engaged than post-training corpora. We further observe a strong correlation between political stances in training data and model behavior, which is present already in most base models and persists across post-training stages. These findings highlight the role of data composition in correlating with model behavior and motivate the need for greater data transparency as a means to understand and monitor model behavior.
Sources
- BERTopic: Neural topic modeling with a class-based TF-IDF procedure
- Apertus: Democratizing Open and Compliant LLMs for Global Language Environments
- Challenges and Applications of Large Language Models
- Tracing the Representation Geometry of Language Models from Pretraining to Post-training
- Conversational AI increases political knowledge as effectively as self-directed internet search
- 2 OLMo 2 Furious
- IssueBench: Millions of Realistic Prompts for Measuring Issue Bias in LLM Writing Assistance
Related papers
- Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
- Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
- Subliminal Steering: Stronger Encoding of Hidden Signals
- MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
- The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
- Untangling the Mechanisms of Misleading Context in Medical Question Answering