tau-Rec: A Verifiable Benchmark for Agentic Recommender Systems

arXiv:2606.10156 · cs.IR, cs.AI, cs.CL · Submitted 2026-06-08 · Read on arXiv

cs.IR, cs.AI, cs.CL

Submitted: 2026-06-08

Updated: 2026-07-24

Code: https://github.com/nbharaths/tau-rec

Terminology

• Benchmark — 22× in this paper · explained in 2 episodes
• Verifiable Rewards — 4× in this paper · explained in 2 episodes
• Benchmarking — 3× in this paper · explained in 4 episodes
• LLM-as-a-Judge — 3× in this paper · explained in 3 episodes
• Thinking mode — 3× in this paper · explained in 2 episodes
• Consistency — 2× in this paper · explained in 3 episodes
• European Conference on Information Retrieval — 2× in this paper
• Failure Modes — 2× in this paper · explained in 3 episodes
• Partially Observable Markov Decision Process — 2× in this paper
• Thirteenth International Conference on Learning Representations — 2× in this paper
• Verification — 2× in this paper · explained in 2 episodes
• Action Space — 1× in this paper · explained in 1 episode
• Adversarial Defense — 1× in this paper · explained in 1 episode
• Agentic System — 1× in this paper · explained in 2 episodes
• Downstream Tasks — 1× in this paper · explained in 2 episodes
• Knowledge Graph — 1× in this paper · explained in 7 episodes
• Knowledge graphs — 1× in this paper · explained in 2 episodes
• Meeting of the Association for Computational Linguistics — 1× in this paper
• Multi-agent Collaboration — 1× in this paper · explained in 2 episodes
• Partially observable Markov decision process (POMDP) — 1× in this paper · explained in 2 episodes
• Trace — 1× in this paper · explained in 2 episodes

Sources

• $\tau^2$-Bench: Evaluating Conversational Agents in a Dual-Control Environment
• Toward Safe and Human-Aligned Game Conversational Recommendation via Multi-Agent Decomposition
• Interactive Recommendation Agent with Active User Commands
• Benchmark Leakage Trap: Can We Trust LLM-based Recommendation?
• Instruction-Following Evaluation for Large Language Models

Related papers

• The Price of Isolation: Estimating the Ecosystem Cost of Symmetric Two-Sided A/B Testing
• SCAR: Semantic Continuity-Aware Retrieval for Efficient Context Expansion in RAG
• MixLoRA-DSI: Dynamically Expandable Mixture-of-LoRA Experts for Rehearsal-Free Generative Retrieval over Dynamic Corpora
• RRCM: Ranking-Driven Retrieval over Collaborative and Meta Memories for LLM Recommendation
• Right Family, Wrong Skill: Evaluating Risk Exposure in Agent Skill Retrieval
• UltRAG: a Universal Simple Scalable Recipe for Knowledge Graph RAG