REER-PT: Reverse-Engineered Reasoning for Perplexity-Guided Pre-training Data Augmentation
cs.CL
Submitted: 2026-08-31
Updated: 2026-08-31
Terminology
Sources
- Scaling Laws for Neural Language Models
- Training Compute-Optimal Large Language Models
- Deduplicating Training Data Makes Language Models Better
- Textbooks Are All You Need
- STaR: Bootstrapping Reasoning With Reasoning
- Quiet-STaR: Language Models Can Teach Themselves to Think Before Speaking
- Thinking Augmented Pre-training
- Rephrasing the Web: A Recipe for Compute and Data-Efficient Language Modeling
- Mining Hidden Thoughts from Texts: Evaluating Continual Pretraining with Synthetic Data for LLM Reasoning
- Reasoning to Learn from Latent Thoughts
- Reinforcement Pre-Training
- RLP: Reinforcement as a Pretraining Objective
- Reinforcement Learning on Pre-Training Data
- Reverse-Engineered Reasoning for Open-Ended Generation
- Rho-1: Not All Tokens Are What You Need
- Irreducible Curriculum for Language Model Pretraining
- Recycling the Web: A Method to Enhance Pre-training Data Quality and Quantity for Language Models
- Learning Facts at Scale with Active Reading
- Rewriting Pre-Training Data Boosts LLM Performance in Math and Code
- FineInstructions: Scaling Synthetic Instructions to Pre-Training Scale
Related papers
- Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
- Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
- Subliminal Steering: Stronger Encoding of Hidden Signals
- MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
- The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
- Untangling the Mechanisms of Misleading Context in Medical Question Answering