Reinforcing Multi-Turn Reasoning in LLM Agents via Turn-Level Reward Design
Quan Wei, Siliang Zeng, Chenliang Li, Zhongruo Wang, William Brown, Oana Frunza, Wei Deng, Anderson Schneider, Yuriy Nevmyvaka, Yang Katie Zhao, Alfredo Garcia, Mingyi Hong
cs.LG
Submitted: 2026-08-21
Updated: 2026-08-24
Code: https://github.com/willccbb/verifiers
Terminology
Sources
- Survey: Multi-Armed Bandits Meet Large Language Models
- Reinforcement Learning for Long-Horizon Interactive LLM Agents
- ReSearch: Learning to Reason with Search for LLMs via Reinforcement Learning
- Stop Summation: Min-Form Credit Assignment Is All Process Reward Model Needs for Reasoning
- Process Reward Models for LLM Agents: Practical Framework and Directions
- Process Reinforcement through Implicit Rewards
- ReTool: Reinforcement Learning for Strategic Tool Use in LLMs
- Group-in-Group Policy Optimization for LLM Agent Training
- ToRA: A Tool-Integrated Reasoning Agent for Mathematical Problem Solving
- DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
- Segment Policy Optimization: Effective Segment-Level Credit Assignment in RL for Large Language Models
- Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Steps
- OpenAI o1 System Card
- An Empirical Study on Reinforcement Learning for Reasoning-Search Interleaved LLM Agents
- Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning
- ToRL: Scaling Tool-Integrated RL
- Generalist Reward Models: Found Inside Large Language Models
- Let's reward step by step: Step-Level reward model as the Navigators for Reasoning
- When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memories
- A Survey of Temporal Credit Assignment in Deep Reinforcement Learning
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: Local-Consistency Optimisation for Bayesian Neural Networks
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks