MoM: Memory of Memory
cs.CL
Submitted: 2026-09-07
Updated: 2026-09-07
License: http://creativecommons.org/licenses/by/4.0/
The gist: For a long-horizon LLM agent, the memory question is not what was once recorded but what currently holds.
Terminology
Abstract
For a long-horizon LLM agent, the memory question is not what was once recorded but what currently holds. Most designs answer it only indirectly: every interaction is stored, and the present is reconstructed at query time by retrieving and reconciling records, so stale values re-enter and the same conflicts are re-litigated. Committing the current value at write time avoids this, but existing write-time (CRUD) memories overwrite, so a wrong update is unrecoverable and prior state is lost. We take the missing combination---commit on arrival while retaining what is displaced---and formalize it as Memory of Memory (MoM): memory tracks not only content but the provenance, status, and history of its own entries. We instantiate MoM as Provenant Memory (P-Mem), a typed provenance graph whose active frontier exposes one current value per resolved key while displaced values are retained as provenance; typed operations decide whether a new observation supports, supersedes, contests, rejects, revokes, or resolves an existing value. P-Mem's decisive gain is validity rather than accuracy: its turn-level read matches the strongest retrieval memory in accuracy at about 4 times fewer read tokens---a retrieval-granularity effect---while graph-guided turn pruning cuts the knowledge-update stale-answer rate (19.4% to 10.9%); on revision chains it stays at 100% where query-time reading collapses to 25%, and, because displaced values are retained rather than overwritten, it recovers committed errors a CRUD memory cannot (100% vs. 0%).
Sources
- Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory
- From RAG to Memory: Non-Parametric Continual Learning for Large Language Models
- MemoryArena: Benchmarking Agent Memory in Interdependent Multi-Session Agentic Tasks
- Evaluating Memory in LLM Agents via Incremental Multi-Turn Interactions
- Memory OS of AI Agent
- MemGPT: Towards LLMs as Operating Systems
- Zep: A Temporal Knowledge Graph Architecture for Agent Memory
- Hierarchical Memory for High-Efficiency Long-Term Reasoning in LLM Agents
- Preference-Aware Memory Update for Long-Term LLM Agents
- Evo-Memory: Benchmarking LLM Agent Test-time Learning with Self-Evolving Memory
- How Memory Management Impacts LLM Agents: An Empirical Study of Experience-Following Behavior
- A-MEM: Agentic Memory for LLM Agents
- Diagnosing Retrieval vs. Utilization Bottlenecks in LLM Agent Memory
- On the Structural Memory of LLM Agents
- MemSkill: Learning and Evolving Memory Skills for Self-Evolving Agents
- Learning to Remember: End-to-End Training of Memory Agents for Long-Context Reasoning
- Neuromem: A Granular Decomposition of the Streaming Lifecycle in External Memory for LLMs
- MemRL: Self-Evolving Agents via Runtime Reinforcement Learning on Episodic Memory
- A Survey on the Memory Mechanism of Large Language Model based Agents
Related papers
- Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
- Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
- Subliminal Steering: Stronger Encoding of Hidden Signals
- MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
- The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
- Untangling the Mechanisms of Misleading Context in Medical Question Answering