Learning Where It Matters: Geometric Anchoring for Robust Preference Alignment
cs.LG
Submitted: 2026-02-04
Updated: 2026-09-25
Code: https://github.com/tatsu-lab/alpaca_eval
Terminology
Sources
- Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback
- Provably Robust DPO: Aligning Language Models with Noisy Feedback
- KTO: Model Alignment as Prospect Theoretic Optimization
- Sharpness-Aware Minimization for Efficiently Improving Generalization
- ORPO: Monolithic Preference Optimization without Reference Model
- Disentangling Length from Quality in Direct Preference Optimization
- Proximal Policy Optimization Algorithms
- Gemma 2: Improving Open Language Models at a Practical Size
- Towards Robust Alignment of Language Models: Distributionally Robustifying Direct Preference Optimization
- Contrastive Preference Optimization: Pushing the Boundaries of LLM Performance in Machine Translation
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: Local-Consistency Optimisation for Bayesian Neural Networks
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks