Overview of the TREC 2025 Million Large Language Models track

arXiv:2609.31921 · cs.IR, cs.AI, cs.MA · Submitted 2026-09-25 · Read on arXiv

cs.IR, cs.AI, cs.MA

Submitted: 2026-09-25

Updated: 2026-09-25

Project page: https://trec-mllm.github.io

Terminology

• Clustering — 6× in this paper · explained in 2 episodes
• Embeddings — 4× in this paper · explained in 2 episodes
• RAG — 4× in this paper · explained in 3 episodes
• Agentic AI — 3× in this paper · explained in 1 episode
• Benchmark — 3× in this paper · explained in 2 episodes
• Generative AI — 3× in this paper · explained in 2 episodes
• LLM-as-a-Judge — 3× in this paper · explained in 3 episodes
• Accuracy — 2× in this paper · explained in 1 episode
• Benchmarking — 2× in this paper · explained in 3 episodes
• Feature Extraction — 2× in this paper · explained in 2 episodes
• Ablation — 1× in this paper · explained in 1 episode
• Aggregation — 1× in this paper · explained in 1 episode
• Calibration — 1× in this paper · explained in 7 episodes
• Generalization — 1× in this paper · explained in 13 episodes
• Grounding — 1× in this paper · explained in 3 episodes
• Prompt Engineering — 1× in this paper · explained in 4 episodes
• Retrieval-Augmented Generation — 1× in this paper
• Retrieval-Augmented Generation (RAG) — 1× in this paper · explained in 16 episodes
• Transformer — 1× in this paper · explained in 2 episodes

Sources

• Future of Information Retrieval Research in the Age of Generative AI
• Large Language Models vs. Search Engines: Evaluating User Preferences Across Varied Information Retrieval Scenarios
• Emergent Abilities of Large Language Models
• BloombergGPT: A Large Language Model for Finance
• Large Language Models for Information Retrieval: A Survey

Related papers

• The Price of Isolation: Estimating the Ecosystem Cost of Symmetric Two-Sided A/B Testing
• SCAR: Semantic Continuity-Aware Retrieval for Efficient Context Expansion in RAG
• MixLoRA-DSI: Dynamically Expandable Mixture-of-LoRA Experts for Rehearsal-Free Generative Retrieval over Dynamic Corpora
• RRCM: Ranking-Driven Retrieval over Collaborative and Meta Memories for LLM Recommendation
• Right Family, Wrong Skill: Evaluating Risk Exposure in Agent Skill Retrieval
• UltRAG: a Universal Simple Scalable Recipe for Knowledge Graph RAG