Alignment Forecasting: Predicting Misalignment From Training Data
cs.CL, cs.AI
Submitted: 2026-09-19
Updated: 2026-09-30
Code: https://github.com/YuehHanChen/alignment_forecasting
Terminology
Sources
- Sabotage Evaluations for Frontier Models
- Anatomy of Post-Training: Using Interpretability to Characterize Data and Shape the Learning Signal
- Weird Generalization and Inductive Backdoors: New Ways to Corrupt LLMs
- Emergent Misalignment: Narrow finetuning can produce broadly misaligned LLMs
- Looking Inward: Language Models Can Learn About Themselves by Introspection
- Subliminal Learning: Language models transmit behavioral traits via hidden signals in data
- Enhancing Chat Language Models by Scaling High-quality Instructional Conversations
- Approaching Human-Level Forecasting with Language Models
- Autodata: An agentic data scientist to create high quality synthetic data
- Tulu 3: Pushing Frontiers in Open Language Model Post-Training
- Chunky Post-Training: Data Driven Failures of Generalization
- Stress Testing Deliberative Alignment for Anti-Scheming Training
- Introspection Adapters: Training LLMs to Report Their Learned Behaviors
- Alignment Pretraining: AI Discourse Causes Self-Fulfilling (Mis)alignment
- From Data to Behavior: Predicting Unintended Model Behaviors Before Training
- Self-Consistency Improves Chain of Thought Reasoning in Language Models
- Predicting Empirical AI Research Outcomes with Language Models
- Predicting LLM Safety Before Release by Simulating Deployment
Related papers
- Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
- Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
- Subliminal Steering: Stronger Encoding of Hidden Signals
- MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
- The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
- Untangling the Mechanisms of Misleading Context in Medical Question Answering