MemAudit: Auditing Long-Term Agent Memory via Hidden User-State Recovery

arXiv:2606.24595 · cs.CL · Submitted 2026-06-23 · Read on arXiv

cs.CL

Submitted: 2026-06-23

Updated: 2026-10-01

Code: https://github.com/sora1998/MemProbe

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Terminology

• Benchmark — 39× in this paper · explained in 2 episodes
• Abstraction — 16× in this paper · explained in 1 episode
• Personalization — 16× in this paper · explained in 2 episodes
• Trace — 15× in this paper · explained in 2 episodes
• Agent Memory — 6× in this paper · explained in 1 episode
• Benchmarking — 6× in this paper · explained in 4 episodes
• Commentary — 5× in this paper · explained in 2 episodes
• Calibration — 4× in this paper · explained in 7 episodes
• Alignment — 3× in this paper · explained in 1 episode
• Core — 3× in this paper · explained in 2 episodes
• In International Conference on Learning Representations — 3× in this paper
• Adaptation — 2× in this paper · explained in 1 episode
• Agentic Task — 2× in this paper · explained in 1 episode
• Agentic Tasks — 2× in this paper · explained in 1 episode
• Hallucination — 2× in this paper · explained in 4 episodes
• LLM-as-a-Judge — 2× in this paper · explained in 3 episodes
• AI Agents — 1× in this paper · explained in 2 episodes
• Abstention — 1× in this paper · explained in 1 episode
• Adam — 1× in this paper · explained in 2 episodes
• Aggregation — 1× in this paper · explained in 1 episode
• Failure Modes — 1× in this paper · explained in 3 episodes
• Fidelity — 1× in this paper · explained in 2 episodes
• Frustration — 1× in this paper · explained in 2 episodes
• LLM judge — 1× in this paper · explained in 2 episodes
• Meeting of the Association for Computational Linguistics — 1× in this paper
• Reinforcement Learning — 1× in this paper · explained in 4 episodes
• Robustness — 1× in this paper · explained in 5 episodes
• Transformer — 1× in this paper · explained in 2 episodes

Sources

• HaluMem: Evaluating Hallucinations in Memory Systems of Agents
• Bootstrapping a User-Centered Task-Oriented Dialogue System
• Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory
• MemoryArena: Benchmarking Agent Memory in Interdependent Multi-Session Agentic Tasks
• Evaluating Memory in LLM Agents via Incremental Multi-Turn Interactions
• Know Me, Respond to Me: Benchmarking LLMs for Dynamic User Profiling and Personalized Responses at Scale
• PersonaMem-v2: Towards Personalized Intelligence via Learning Implicit User Personas and Agentic Memory
• PERMA: Benchmarking Personalized Memory Agents via Event-Driven Preference and Realistic Task Environments
• Evaluating Very Long-Term Conversational Memory of LLM Agents
• MemGPT: Towards LLMs as Operating Systems
• Generative Agents: Interactive Simulacra of Human Behavior
• LaMP: When Large Language Models Meet Personalization
• Reflexion: Language Agents with Verbal Reinforcement Learning
• JudgeBench: A Benchmark for Evaluating LLM-based Judges
• From Recall to Forgetting: Benchmarking Long-Term Memory for Personalized Agents
• DeepPersona: A Generative Engine for Scaling Deep Synthetic Personas
• LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memory
• AlpsBench: An LLM Personalization Benchmark for Real-Dialogue Memorization and Preference Alignment
• A-MEM: Agentic Memory for LLM Agents
• PersonaLens: A Benchmark for Personalization Evaluation in Conversational AI Assistants

Related papers

• Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
• Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
• Subliminal Steering: Stronger Encoding of Hidden Signals
• MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
• The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
• Untangling the Mechanisms of Misleading Context in Medical Question Answering