Entropy Regularization: A Free Correction to Cross-Entropy for Verified Demonstrations
cs.LG, stat.ML
Submitted: 2026-09-24
Updated: 2026-09-24
Code: https://github.com/huggingface/peft
Terminology
Sources
- Program Synthesis with Large Language Models
- Rethinking Fine-Tuning when Scaling Test-Time Compute: Limiting Confidence Improves Mathematical Reasoning
- Evaluating Large Language Models Trained on Code
- Inference-Aware Fine-Tuning for Best-of-N Sampling in Large Language Models
- Scaling Instruction-Finetuned Language Models
- Training Verifiers to Solve Math Word Problems
- FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning
- DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
- Maximum Entropy RL (Provably) Solves Some Robust RL Problems
- Learning to Answer from Correct Demonstrations
- Tulu 3: Pushing Frontiers in Open Language Model Post-Training
- Compute Aligned Training: Optimizing for Test Time Inference
- Llama 2: Open Foundation and Fine-Tuned Chat Models
- Qwen2 Technical Report
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: Local-Consistency Optimisation for Bayesian Neural Networks
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks