AdaSearch: Balancing Parametric Knowledge and Search in Large Language Models via Reinforcement Learning

arXiv:2512.16883 · cs.CL · Submitted 2025-12-18 · Read on arXiv

cs.CL

Submitted: 2025-12-18

Updated: 2026-09-01

Comments: Accepted to EMNLP 2026. Code and artifacts will be uploaded to https://github.com/MiuLab/AdaSearch

Code: https://github.com/MiuLab/AdaSearch

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Terminology

• RAG — 19× in this paper · explained in 3 episodes
• Benchmark — 12× in this paper · explained in 2 episodes
• Reinforcement Learning — 12× in this paper · explained in 4 episodes
• GRPO — 8× in this paper · explained in 6 episodes
• Hallucination — 8× in this paper · explained in 3 episodes
• Adaptivity — 6× in this paper · explained in 1 episode
• Reward Function — 5× in this paper · explained in 2 episodes
• Accuracy — 4× in this paper · explained in 1 episode
• Adam — 4× in this paper · explained in 2 episodes
• Credit assignment — 4× in this paper · explained in 3 episodes
• Fine-Tuning — 3× in this paper · explained in 5 episodes
• Interpretability — 3× in this paper · explained in 15 episodes
• Large Language Models (LLMs) — 3× in this paper · explained in 6 episodes
• Reinforcement Learning (RL) — 3× in this paper · explained in 25 episodes
• Reward Shaping — 3× in this paper · explained in 2 episodes
• Ablation — 2× in this paper · explained in 1 episode
• F1 Score — 2× in this paper · explained in 3 episodes
• Foundation Model — 2× in this paper · explained in 3 episodes
• Generalizability — 2× in this paper · explained in 6 episodes
• Retrieval-Augmented Generation (RAG) — 2× in this paper · explained in 16 episodes
• Two-stage training — 2× in this paper · explained in 2 episodes
• Verification — 2× in this paper · explained in 2 episodes
• Ablation Studies — 1× in this paper · explained in 1 episode
• Chain-of-Thought (CoT) — 1× in this paper · explained in 4 episodes
• Chain-of-Thought (CoT) Reasoning — 1× in this paper · explained in 2 episodes
• Chain-of-Thought Prompting — 1× in this paper · explained in 2 episodes
• Consistency — 1× in this paper · explained in 3 episodes
• Core — 1× in this paper · explained in 2 episodes
• Deep Reinforcement Learning — 1× in this paper · explained in 2 episodes
• Distribution Shift — 1× in this paper · explained in 5 episodes
• Embeddings — 1× in this paper · explained in 2 episodes
• Few-Shot Learning — 1× in this paper · explained in 4 episodes
• Foundation Models — 1× in this paper · explained in 7 episodes
• Online Reinforcement Learning — 1× in this paper · explained in 2 episodes
• Query Complexity — 1× in this paper · explained in 2 episodes
• Robustness — 1× in this paper · explained in 5 episodes
• Supervised Fine-Tuning (SFT) — 1× in this paper · explained in 8 episodes

Sources

• Retaining by Doing: The Role of On-Policy Data in Mitigating Forgetting
• ReSearch: Learning to Reason with Search for LLMs via Reinforcement Learning
• SSRL: Self-Search Reinforcement Learning
• The Llama 3 Herd of Models
• DeepRAG: Thinking to Retrieve Step by Step for Large Language Models
• DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
• Measuring Massive Multitask Language Understanding
• Reinforced Internal-External Knowledge Synergistic Reasoning for Efficient Adaptive Search Agent
• OpenAI o1 System Card
• Why Language Models Hallucinate
• Internet-augmented language models through few-shot prompting for open-domain question answering
• ToRL: Scaling Tool-Integrated RL
• WebGPT: Browser-assisted question-answering with human feedback
• A Survey of Temporal Credit Assignment in Deep Reinforcement Learning
• SMART: Self-Aware Agent for Tool Overuse Mitigation
• DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
• RL's Razor: Why Online Reinforcement Learning Forgets Less
• Retrieval Augmentation Reduces Hallucination in Conversation
• Gemini: A Family of Highly Capable Multimodal Models
• Llama 2: Open Foundation and Fine-Tuned Chat Models

Related papers

• Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
• Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
• Subliminal Steering: Stronger Encoding of Hidden Signals
• MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
• The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
• Untangling the Mechanisms of Misleading Context in Medical Question Answering