MemoryArena: Benchmarking Agent Memory in Interdependent Multi-Session Agentic Tasks

arXiv:2602.16313 · cs.CL · Submitted 2026-02-18 · Read on arXiv

cs.CL

Submitted: 2026-02-18

Updated: 2026-09-17

Comments: ICML 2026

Project page: https://memoryarena.github.io

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Terminology

• Trace — 52× in this paper · explained in 2 episodes
• Benchmark — 41× in this paper · explained in 2 episodes
• Agent Memory — 39× in this paper · explained in 1 episode
• Benchmarking — 37× in this paper · explained in 3 episodes
• Agentic Task — 35× in this paper · explained in 1 episode
• Agentic Tasks — 34× in this paper · explained in 1 episode
• RAG — 21× in this paper · explained in 3 episodes
• Regret — 6× in this paper · explained in 3 episodes
• Loss Function — 5× in this paper · explained in 2 episodes
• Abstraction — 4× in this paper · explained in 1 episode
• Accuracy — 3× in this paper · explained in 1 episode
• Consistency — 3× in this paper · explained in 3 episodes
• Reasoning Traces — 3× in this paper · explained in 3 episodes
• Sample Complexity — 3× in this paper · explained in 15 episodes
• VC Dimension — 3× in this paper · explained in 2 episodes
• Core — 2× in this paper · explained in 2 episodes
• Embeddings — 2× in this paper · explained in 2 episodes
• Retrieval-Augmented Generation (RAG) — 2× in this paper · explained in 16 episodes
• Task Progress Score — 2× in this paper
• Task Success Rate — 2× in this paper
• AI Agents — 1× in this paper · explained in 2 episodes
• Action Space — 1× in this paper · explained in 1 episode
• Agentic AI — 1× in this paper · explained in 1 episode
• Agentic AI systems — 1× in this paper · explained in 1 episode
• Agentic Search — 1× in this paper · explained in 1 episode
• Alignment — 1× in this paper · explained in 1 episode
• Clustering — 1× in this paper · explained in 2 episodes
• Convergence Guarantees — 1× in this paper · explained in 3 episodes
• Failure Modes — 1× in this paper · explained in 3 episodes
• Generalization — 1× in this paper · explained in 13 episodes
• Grounding — 1× in this paper · explained in 3 episodes
• In-Context Learning — 1× in this paper · explained in 4 episodes
• Large Language Model (LLM) — 1× in this paper · explained in 3 episodes
• Self-consistency — 1× in this paper · explained in 2 episodes
• Semantic Drift — 1× in this paper · explained in 3 episodes
• Semantic Similarity — 1× in this paper · explained in 2 episodes
• Verification — 1× in this paper · explained in 2 episodes

Sources

• MemoryBench: A Benchmark for Memory and Continual Learning in LLM Systems
• Beyond Task Completion: An Assessment Framework for Evaluating Agentic AI Systems
• SWE-bench: Can Language Models Resolve Real-World GitHub Issues?
• BrowseComp-Plus: A More Fair and Transparent Evaluation Benchmark of Deep-Research Agent
• AgencyBench: Benchmarking the Frontiers of Autonomous Agents in 1M-Token Real-World Contexts
• Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory
• EMemBench: Interactive Benchmarking of Episodic Memory for VLM Agents
• VeriWeb: Verifiable Long-Chain Web Benchmark for Agentic Information-Seeking
• MEMTRACK: Evaluating Long-Term Memory and State Tracking in Multi-Platform Dynamic Agent Environments
• From Local to Global: A Graph RAG Approach to Query-Focused Summarization
• ReasoningBank: Scaling Agent Self-Evolving with Reasoning Memory
• AgentLongBench: A Controllable Long Benchmark For Long-Contexts Agents via Environment Rollouts
• Mind2Web 2: Evaluating Agentic Search with Agent-as-a-Judge
• RULER: What's the Real Context Size of Your Long-Context Language Models?
• Memory in the Age of AI Agents
• Mem2ActBench: A Benchmark for Evaluating Long-Term Memory Utilization in Task-Oriented Autonomous Agents
• Evaluating Memory in LLM Agents via Incremental Multi-Turn Interactions
• BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agents
• Evo-Memory: Benchmarking LLM Agent Test-time Learning with Self-Evolving Memory

Related papers

• Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
• Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
• Subliminal Steering: Stronger Encoding of Hidden Signals
• MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
• The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
• Untangling the Mechanisms of Misleading Context in Medical Question Answering