LSTMem: Hierarchical Long Short-Term Online Memory for Large Language Models

arXiv:2609.33268 · cs.AI · Submitted 2026-09-27 · Read on arXiv

cs.AI

Submitted: 2026-09-27

Updated: 2026-09-27

Terminology

• Ablation — 13× in this paper · explained in 1 episode
• Transformer — 13× in this paper · explained in 2 episodes
• Accuracy — 9× in this paper · explained in 1 episode
• Benchmark — 9× in this paper · explained in 2 episodes
• KV cache — 7× in this paper · explained in 3 episodes
• Reconstruction Error — 6× in this paper · explained in 2 episodes
• Decode — 4× in this paper · explained in 2 episodes
• Associative Memory — 3× in this paper · explained in 2 episodes
• RAG — 3× in this paper · explained in 3 episodes
• Generative AI — 2× in this paper · explained in 2 episodes
• AI Agents — 1× in this paper · explained in 2 episodes
• Ablation Studies — 1× in this paper · explained in 1 episode
• Abstention — 1× in this paper · explained in 1 episode
• Adaptation — 1× in this paper · explained in 1 episode
• Benchmarking — 1× in this paper · explained in 3 episodes
• Distillation — 1× in this paper · explained in 2 episodes
• Frozen model — 1× in this paper · explained in 3 episodes
• Large Language Models (LLMs) — 1× in this paper · explained in 6 episodes
• Regularization — 1× in this paper · explained in 3 episodes
• Self-Evolving Agents — 1× in this paper · explained in 2 episodes
• Transformer Architecture — 1× in this paper · explained in 2 episodes

Sources

• Understanding LoRA as Knowledge Memory: An Empirical Analysis
• ATLAS: Learning to Optimally Memorize the Context at Test Time
• LoRA Learns Less and Forgets Less
• Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory
• Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality
• Rethinking Memory in LLM based Agents: Representations, Operations, and Emerging Topics
• Context Length Alone Hurts LLM Performance Despite Perfect Retrieval
• In-context Autoencoder for Context Compression in a Large Language Model
• Mamba: Linear-Time Sequence Modeling with Selective State Spaces
• RULER: What's the Real Context Size of Your Long-Context Language Models?
• LoRA: Low-Rank Adaptation of Large Language Models
• TransMem: Transforming Hidden States into Memory for Large Language Models
• delta-mem: Efficient Online Memory for Large Language Models
• Decoupled Weight Decay Regularization
• Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention
• MemGPT: Towards LLMs as Operating Systems
• Eagle and Finch: RWKV with Matrix-Valued States and Dynamic Recurrence
• HGRN2: Gated Linear RNNs with State Expansion
• GPQA: A Graduate-Level Google-Proof Q&A Benchmark
• Learning to (Learn at Test Time): RNNs with Expressive Hidden States

Related papers

• MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
• Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
• The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
• MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
• Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems
• VSAL: A Vision Solver with Adaptive Layouts for Graph Property Detection