Memento No More: Coaching AI Agents to Master Multiple Tasks via Hints Internalization
cs.LG
Submitted: 2025-02-03
Updated: 2026-09-20
Journal ref: Transactions on Machine Learning Research (TMLR) (07/2026)
Code: https://github.com/minttusofia/memento-no-more
License: http://creativecommons.org/licenses/by/4.0/
The gist: As the general capabilities of artificial intelligence (AI) agents continue to evolve, their ability to learn to master multiple complex tasks through experience remains a key challenge.
Terminology
Abstract
As the general capabilities of artificial intelligence (AI) agents continue to evolve, their ability to learn to master multiple complex tasks through experience remains a key challenge. Current LLM agents, particularly those based on proprietary language models, typically rely on prompts to incorporate knowledge about the target tasks. This approach does not allow the agent to internalize this information and instead relies on ever-expanding prompts to sustain its functionality in diverse scenarios. This resembles a system of notes used by a person affected by anterograde amnesia, the inability to form new memories. In this paper, we propose a novel method to train AI agents to incorporate knowledge and skills for multiple tasks without the need for either cumbersome note systems or prior high-quality demonstration data. Our approach employs an iterative process where the agent collects new experiences, receives corrective feedback from humans in the form of hints, and integrates this feedback into its weights via a context distillation training procedure. We demonstrate the efficacy of our approach by implementing it in a Llama-3-based agent that, after only a few rounds of feedback, outperforms advanced models GPT-4o and DeepSeek-V3 in tasksets requiring correct sequencing of information retrieval, tool use, and question answering.
Sources
- Constitutional AI: Harmlessness from AI Feedback
- Evaluating Large Language Models Trained on Code
- Training Verifiers to Solve Math Word Problems
- The Llama 3 Herd of Models
- Distilling the Knowledge in a Neural Network
- LoRA: Low-Rank Adaptation of Large Language Models
- Large Language Models Cannot Self-Correct Reasoning Yet
- Efficient Knowledge Injection in LLMs via Self-Distillation
- DeepSeek-V3 Technical Report
- Decoupled Weight Decay Regularization
- DynaSaur: Large Language Agents Beyond Predefined Actions
- Show Your Work: Scratchpads for Intermediate Computation with Language Models
- GPT-4 Technical Report
- GPT-4o System Card
- Learning by Distilling Context
- Executable Code Actions Elicit Better LLM Agents
- OfficeBench: Benchmarking Language Agents across Multiple Applications for Office Automation
- ReWOO: Decoupling Reasoning from Observations for Efficient Augmented Language Models
- An Examination on the Effectiveness of Divide-and-Conquer Prompting in Large Language Models
- Language Agent Tree Search Unifies Reasoning Acting and Planning in Language Models
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: Local-Consistency Optimisation for Bayesian Neural Networks
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks