Reinforcement Learning with Verifiable Rewards for Small Search Agents

arXiv:2609.28765 · cs.AI, cs.IR · Submitted 2026-09-23 · Read on arXiv

cs.AI, cs.IR

Submitted: 2026-09-23

Updated: 2026-09-23

Code: https://github.com/NVIDIA-NeMo/RL

Terminology

• GRPO — 27× in this paper · explained in 6 episodes
• Benchmark — 21× in this paper · explained in 2 episodes
• Reinforcement Learning — 14× in this paper · explained in 4 episodes
• Accuracy — 12× in this paper · explained in 1 episode
• Ablation — 9× in this paper · explained in 1 episode
• Distillation — 8× in this paper · explained in 2 episodes
• Reward Shaping — 8× in this paper · explained in 2 episodes
• Verifiable Rewards — 7× in this paper · explained in 2 episodes
• Trace — 6× in this paper · explained in 2 episodes
• International Conference on Learning Representations — 5× in this paper
• Reinforcement Learning with Verifiable Rewards — 5× in this paper
• RAG — 4× in this paper · explained in 3 episodes
• Empirical Methods in Natural Language Processing — 3× in this paper
• Information Gain — 3× in this paper · explained in 2 episodes
• Robustness — 3× in this paper · explained in 5 episodes
• Conference on Language Modeling — 2× in this paper
• Embeddings — 2× in this paper · explained in 2 episodes
• Gradient Flow — 2× in this paper · explained in 2 episodes
• Group Relative Policy Optimization (GRPO) — 2× in this paper · explained in 5 episodes
• Meeting of the Association for Computational Linguistics — 2× in this paper
• Reasoning Traces — 2× in this paper · explained in 3 episodes
• Agentic AI — 1× in this paper · explained in 1 episode
• Agentic Retrieval — 1× in this paper · explained in 1 episode
• Agentic System — 1× in this paper · explained in 2 episodes
• Anti-correlation — 1× in this paper · explained in 3 episodes
• Archetype — 1× in this paper · explained in 1 episode
• B with Group Relative Policy Optimization — 1× in this paper
• Chain-of-Thought Prompting — 1× in this paper · explained in 2 episodes
• Core — 1× in this paper · explained in 2 episodes
• Fine-Tuning — 1× in this paper · explained in 5 episodes
• Grounding — 1× in this paper · explained in 3 episodes
• Large Language Models (LLMs) — 1× in this paper · explained in 6 episodes
• Reinforcement Learning (RL) — 1× in this paper · explained in 25 episodes
• Reward Function — 1× in this paper · explained in 2 episodes
• Transactions of the Association for Computational Linguistics — 1× in this paper

Sources

• Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection
• Small Language Models are the Future of Agentic AI
• ReSearch: Learning to Reason with Search for LLMs via Reinforcement Learning
• DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
• Concise Reasoning via Reinforcement Learning
• JustRL: Scaling a 1.5B LLM with a Simple RL Recipe
• Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Steps
• Active Retrieval Augmented Generation
• Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning
• FlashRAG: A Modular Toolkit for Efficient Retrieval-Augmented Generation Research
• TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension
• Can Compact Language Models Search Like Agents? Distillation-Guided Policy Optimization for Preserving Agentic RAG Capabilities
• Tulu 3: Pushing Frontiers in Open Language Model Post-Training
• Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
• RoRecomp: Enhancing Reasoning Efficiency via Rollout Response Recomposition in Reinforcement Learning
• IG-Search: Step-Level Information Gain Rewards for Search-Augmented Reasoning
• Understanding R1-Zero-Like Training: A Critical Perspective
• When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memories
• Training language models to follow instructions with human feedback
• Measuring and Narrowing the Compositionality Gap in Language Models

Related papers

• MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
• Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
• The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
• MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
• Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems
• VSAL: A Vision Solver with Adaptive Layouts for Graph Property Detection