Masking Frequent Tokens Sharpens Direct Preference Optimization
cs.CL, cs.LG
Submitted: 2026-09-26
Updated: 2026-09-26
Terminology
Sources
- Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback
- SparsePO: Controlling Preference Alignment of LLMs via Sparse Token Masks
- UltraFeedback: Boosting Language Models with Scaled AI Feedback
- Length-Controlled AlpacaEval: A Simple Way to Debias Automatic Evaluators
- Representation Degeneration Problem in Training Natural Language Generation Models
- AlphaPO: Reward Shape Matters for LLM Alignment
- Normalized Rewards for Preference Optimization
- Smaug: Fixing Failure Modes of Preference Optimisation with DPO-Positive
- ConfPO: Exploiting Policy Model Confidence for Critical Token Selection in Preference Optimization
- Token-level Direct Preference Optimization
- SLiC-HF: Sequence Likelihood Calibration with Human Feedback
- TGDPO: Harnessing Token-Level Reward Guidance for Enhancing Direct Preference Optimization
Related papers
- Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
- Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
- Subliminal Steering: Stronger Encoding of Hidden Signals
- MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
- The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
- Untangling the Mechanisms of Misleading Context in Medical Question Answering