Training Documents Reranker with Search Rubrics for Deep Research Agent

arXiv:2608.03527 · cs.IR, cs.AI, cs.CL · Submitted 2026-08-04 · Read on arXiv

Wenhan Liu, Yu Lu, Qiaolin Xia, Hui Xu, Tong Zhao, Jian Xi, Yutao Zhu, Haijin Liang, Haibo Shi, Hao Wang, Zhicheng Dou

cs.IR, cs.AI, cs.CL

Submitted: 2026-08-04

Comments: 28 pages

Code: https://github.com/openai/simple-evals

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Terminology

• RAG — 51× in this paper · explained in 3 episodes
• Benchmark — 32× in this paper · explained in 2 episodes
• Consistency — 17× in this paper · explained in 3 episodes
• Reinforcement Learning — 12× in this paper · explained in 4 episodes
• LLM judge — 7× in this paper · explained in 2 episodes
• Two-stage training — 6× in this paper · explained in 2 episodes
• Fine-Tuning — 5× in this paper · explained in 5 episodes
• Aggregation — 4× in this paper · explained in 1 episode
• Redundancy — 4× in this paper · explained in 2 episodes
• Ablation — 3× in this paper · explained in 1 episode
• GRPO — 3× in this paper · explained in 6 episodes
• Ablation study — 2× in this paper · explained in 1 episode
• Agentic Search — 2× in this paper · explained in 1 episode
• Benchmarking — 2× in this paper · explained in 4 episodes
• Core — 2× in this paper · explained in 2 episodes
• Embeddings — 2× in this paper · explained in 2 episodes
• Generalization — 2× in this paper · explained in 13 episodes
• Natural Questions — 2× in this paper
• Reinforcement Learning (RL) — 2× in this paper · explained in 25 episodes
• Supervised Fine-Tuning (SFT) — 2× in this paper · explained in 8 episodes
• Ablation Studies — 1× in this paper · explained in 1 episode
• Aperture — 1× in this paper · explained in 1 episode
• Evaluation Dimensions — 1× in this paper · explained in 3 episodes
• Two-Stage Training Strategy — 1× in this paper · explained in 3 episodes

Sources

• HealthBench: Evaluating Large Language Models Towards Improved Human Health
• OpenScholar: Synthesizing Scientific Literature with Retrieval-augmented LMs
• DeepResearch Bench: A Comprehensive Benchmark for Deep Research Agents
• Rank4Gen: RAG-Preference-Aligned Document Set Selection and Ranking
• SumRank: Aligning Summarization Models for Long-Document Listwise Reranking
• Retrieval-Augmented Generation for Large Language Models: A Survey
• Rubrics as Rewards: Reinforcement Learning Beyond Verifiable Domains
• AdvancedIF: Rubric-Based Benchmarking and Reinforcement Learning for Advancing LLM Instruction Following
• Reinforcement Learning with Rubric Anchors
• Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning
• Tongyi DeepResearch Technical Report
• Search Arena: Analyzing Search-Augmented LLMs
• RankVicuna: Zero-Shot Listwise Document Reranking with Open-Source Large Language Models
• RankZephyr: Effective and Robust Zero-Shot Listwise Reranking is a Breeze!
• ReasonRank: Empowering Passage Ranking with Strong Reasoning Ability
• Agentic-R: Learning to Retrieve for Agentic Search
• Sliding Windows Are Not the End: Exploring Full Ranking with Long-Context Large Language Models
• DR Tulu: Reinforcement Learning with Evolving Rubrics for Deep Research
• DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
• Rethinking Rubric Generation for Improving LLM Judge and Reward Modeling for Open-ended Tasks

Related papers

• The Price of Isolation: Estimating the Ecosystem Cost of Symmetric Two-Sided A/B Testing
• SCAR: Semantic Continuity-Aware Retrieval for Efficient Context Expansion in RAG
• MixLoRA-DSI: Dynamically Expandable Mixture-of-LoRA Experts for Rehearsal-Free Generative Retrieval over Dynamic Corpora
• RRCM: Ranking-Driven Retrieval over Collaborative and Meta Memories for LLM Recommendation
• Right Family, Wrong Skill: Evaluating Risk Exposure in Agent Skill Retrieval
• UltRAG: a Universal Simple Scalable Recipe for Knowledge Graph RAG