Finetuning with Sampling: SFT Learns Better Than You Think

arXiv:2610.02140 · cs.LG, cs.AI, cs.CL · Submitted 2026-10-01 · Read on arXiv

cs.LG, cs.AI, cs.CL

Submitted: 2026-10-01

Updated: 2026-10-01

Terminology

• Accuracy — 22× in this paper · explained in 1 episode
• Trace — 22× in this paper · explained in 2 episodes
• Generalization — 17× in this paper · explained in 13 episodes
• Distillation — 15× in this paper · explained in 2 episodes
• Self-Distillation — 11× in this paper · explained in 7 episodes
• Benchmark — 7× in this paper · explained in 2 episodes
• Catastrophic Forgetting — 5× in this paper · explained in 5 episodes
• GRPO — 5× in this paper · explained in 6 episodes
• Reinforcement Learning — 5× in this paper · explained in 4 episodes
• Markov Chain Monte Carlo — 4× in this paper
• Fine-Tuning — 3× in this paper · explained in 5 episodes
• On-Policy Distillation — 2× in this paper · explained in 3 episodes
• Alignment — 1× in this paper · explained in 1 episode
• Bayesian Inference — 1× in this paper · explained in 5 episodes
• Consistency — 1× in this paper · explained in 3 episodes
• Continual Learning — 1× in this paper · explained in 2 episodes
• Generative Model — 1× in this paper · explained in 2 episodes
• Generative Models — 1× in this paper · explained in 2 episodes
• Inverse Problem — 1× in this paper · explained in 2 episodes
• Online Reinforcement Learning — 1× in this paper · explained in 2 episodes
• Reinforcement Learning (RL) — 1× in this paper · explained in 25 episodes
• Self-consistency — 1× in this paper · explained in 2 episodes
• Steering — 1× in this paper · explained in 2 episodes
• Verifier — 1× in this paper · explained in 2 episodes

Sources

• Self-Consistency via Marginal Sharpening
• Power-SMC: Low-Latency Sequence-Level Power Sampling for Training-Free LLM Reasoning
• Retaining by Doing: The Role of On-Policy Data in Mitigating Forgetting
• HuatuoGPT-o1, Towards Medical Complex Reasoning with LLMs
• SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-training
• Training Verifiers to Solve Math Word Problems
• SciKnowEval: Evaluating Multi-level Scientific Knowledge of Large Language Models
• When does a predictor know its own loss?
• DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
• Measuring Massive Multitask Language Understanding
• Measuring Mathematical Problem Solving With the MATH Dataset
• Open-Reasoner-Zero: An Open Source Approach to Scaling Up Reinforcement Learning on the Base Model
• Scalable Power Sampling: Unlocking Efficient, Training-Free Reasoning for LLMs via Distribution Sharpening
• Reasoning with Sampling: Your Base Model is Smarter Than You Think
• Unrolled denoising networks provably learn optimal Bayesian inference
• ReGuidance: A Simple Diffusion Wrapper for Boosting Sample Quality on Hard Inverse Problems
• Sequential Monte Carlo Steering of Large Language Models using Probabilistic Programs
• Blink of an eye: a simple theory for feature localization in generative models
• Competition-Level Code Generation with AlphaCode
• UFT: Unifying Supervised and Reinforcement Fine-Tuning

Related papers

• Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
• AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
• Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
• Convergence issues in Relational Concept Analysis based on AOC-posets
• Beliefs Beyond Posteriors: Local-Consistency Optimisation for Bayesian Neural Networks
• Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks