Reward Score Matching: Unifying Reward-based Fine-tuning for Flow and Diffusion Models
cs.LG, cs.AI, cs.CV
Submitted: 2026-04-19
Updated: 2026-08-26
Comments: 50 pages, 15 figures. Best Paper Award at SPIGM workshop, ICML 2026
Code: https://github.com/jaylee2000/rsm
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Terminology
Sources
- Rethinking the Design Space of Reinforcement Learning for Diffusion Models: On the Importance of Likelihood Estimation Beyond Loss Design
- Scaling Rectified Flow Transformers for High-Resolution Image Synthesis
- MixGRPO: Unlocking Flow-based GRPO Efficiency with Mixed ODE-SDE
- A unified view of entropy-regularized Markov decision processes
- Equivalence Between Policy Gradients and Soft Q-Learning
- Understanding Reinforcement Learning-Based Fine-Tuning of Diffusion Models: A Tutorial and Review
- Fine-Tuning of Continuous-Time Diffusion Models as Entropy-Regularized Control
- Inference-Time Alignment in Diffusion Models with Reward-Guided Generation: Tutorial and Review
- GRPO-Guard: Mitigating Implicit Over-Optimization in Flow Matching via Regulated Clipping
- Human Preference Score v2: A Solid Benchmark for Evaluating Human Preferences of Text-to-Image Synthesis
- Advantage Weighted Matching: Aligning RL with Pretraining in Diffusion Models
- DanceGRPO: Unleashing GRPO on Visual Generation
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: Local-Consistency Optimisation for Bayesian Neural Networks
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks