APEX-EM: Non-Parametric Online Learning for Autonomous Agents via Structured Procedural-Episodic Experience Replay

arXiv:2603.29093 · cs.CL, cs.AI, cs.IR · Submitted 2026-03-31 · Read on arXiv

cs.CL, cs.AI, cs.IR

Submitted: 2026-03-31

Updated: 2026-08-31

Terminology

• APEX — 60× in this paper · explained in 1 episode
• Ablation — 30× in this paper · explained in 1 episode
• Benchmark — 29× in this paper · explained in 2 episodes
• RAG — 26× in this paper · explained in 3 episodes
• Trace — 17× in this paper · explained in 2 episodes
• Verification — 17× in this paper · explained in 2 episodes
• Accuracy — 14× in this paper · explained in 1 episode
• Aggregation — 12× in this paper · explained in 1 episode
• Semantic Similarity — 10× in this paper · explained in 2 episodes
• Verifier — 8× in this paper · explained in 2 episodes
• Embeddings — 6× in this paper · explained in 2 episodes
• Knowledge Graph — 6× in this paper · explained in 7 episodes
• Reinforcement Learning — 6× in this paper · explained in 4 episodes
• Ontology — 5× in this paper · explained in 2 episodes
• Procedural Knowledge — 5× in this paper · explained in 2 episodes
• Ablation Studies — 4× in this paper · explained in 1 episode
• Cumulative Success Rate — 4× in this paper
• Procedural Knowledge Graph — 4× in this paper
• Task Understanding — 4× in this paper
• Agent Memory — 3× in this paper · explained in 1 episode
• Distribution Shift — 3× in this paper · explained in 5 episodes
• Generalization — 3× in this paper · explained in 13 episodes
• In-Context Learning — 3× in this paper · explained in 4 episodes
• Ablation study — 2× in this paper · explained in 1 episode
• Adaptation — 2× in this paper · explained in 1 episode
• Benchmarking — 2× in this paper · explained in 3 episodes
• Core — 2× in this paper · explained in 2 episodes
• Distillation — 2× in this paper · explained in 2 episodes
• Error Decomposition — 2× in this paper · explained in 3 episodes
• Last Epoch Success Rate — 2× in this paper
• Meta-learning — 2× in this paper · explained in 2 episodes
• Operating System — 2× in this paper
• Provenance — 2× in this paper · explained in 3 episodes
• Self-Evolving Agents — 2× in this paper · explained in 2 episodes
• AI Agents — 1× in this paper · explained in 2 episodes
• Accountability — 1× in this paper · explained in 2 episodes
• Action Space — 1× in this paper · explained in 1 episode
• Adam — 1× in this paper · explained in 2 episodes
• Catastrophic Forgetting — 1× in this paper · explained in 5 episodes
• Chain-of-Thought Prompting — 1× in this paper · explained in 2 episodes
• Consistency — 1× in this paper · explained in 3 episodes
• Convergence Guarantees — 1× in this paper · explained in 3 episodes
• Cosine Similarity — 1× in this paper · explained in 3 episodes
• Fidelity — 1× in this paper · explained in 2 episodes
• Generative AI — 1× in this paper · explained in 2 episodes
• Graph Construction — 1× in this paper · explained in 3 episodes
• Knowledge graphs — 1× in this paper · explained in 2 episodes
• LLM judge — 1× in this paper · explained in 2 episodes
• Large Language Model (LLM) — 1× in this paper · explained in 3 episodes
• Model-Agnostic — 1× in this paper · explained in 2 episodes
• Multi-hop Reasoning — 1× in this paper · explained in 3 episodes
• Regret — 1× in this paper · explained in 3 episodes
• Reinforcement Learning (RL) — 1× in this paper · explained in 24 episodes
• Self-consistency — 1× in this paper · explained in 2 episodes
• Steering — 1× in this paper · explained in 2 episodes

Sources

• Learning to Continually Learn via Meta-learning Agentic Memory Designs
• Hindsight Experience Replay
• Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory
• Uncertainty quantification for stationary and time-dependent PDEs subject to Gevrey regular random domain deformations
• MemSkill: Learning and Evolving Memory Skills for Self-Evolving Agents
• AWAC: Accelerating Online Reinforcement Learning with Offline Datasets
• MemGPT: Towards LLMs as Operating Systems
• Humanity's Last Exam
• Zep: A Temporal Knowledge Graph Architecture for Agent Memory
• SkillRL: Evolving Agents via Recursive Skill-Augmented Reinforcement Learning
• MIRIX: Multi-Agent Memory System for LLM-Based Agents
• A-MEM: Agentic Memory for LLM Agents
• Diagnosing and Addressing Pitfalls in KG-RAG Datasets: Toward More Reliable Benchmarking
• The doubly librating Plutinos
• Unbiased surveys of dust-enshrouded galaxies using ALMA

Related papers

• Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
• Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
• Subliminal Steering: Stronger Encoding of Hidden Signals
• MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
• The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
• Untangling the Mechanisms of Misleading Context in Medical Question Answering