Mitigating the Stability-Plasticity Dilemma in Adaptive Train Scheduling with Curriculum-Driven Continual DQN Expansion
cs.LG, cs.NE
Submitted: 2024-08-19
Updated: 2026-09-14
Comments: 21 Pages, 6 Tables, 8 Figures, CoLLAs
Journal ref: P13 2025 CoLLAs
License: http://creativecommons.org/licenses/by/4.0/
The gist: A continual learning agent builds on previous experiences to develop increasingly complex behaviors by adapting to non-stationary and dynamic environments while preserving previously acquired
Terminology
Abstract
A continual learning agent builds on previous experiences to develop increasingly complex behaviors by adapting to non-stationary and dynamic environments while preserving previously acquired knowledge. However, scaling these systems presents significant challenges, particularly in balancing the preservation of previous policies with the adaptation of new ones to current environments. This balance, known as the stability-plasticity dilemma, is especially pronounced in complex multi-agent domains such as the train scheduling problem, where environmental and agent behaviors are constantly changing, and the search space is vast. In this work, we propose addressing these challenges in the train scheduling problem using curriculum learning. We design a curriculum with adjacent skills that build on each other to improve generalization performance. Introducing a curriculum with distinct tasks introduces non-stationarity, which we address by proposing a new algorithm: Continual Deep Q-Network (DQN) Expansion (CDE). Our approach dynamically generates and adjusts Q-function subspaces to handle environmental changes and task requirements. CDE mitigates catastrophic forgetting through EWC while ensuring high plasticity using adaptive rational activation functions. Experimental results demonstrate significant improvements in learning efficiency and adaptability compared to RL baselines and other adapted methods for continual learning, highlighting the potential of our method in managing the stability-plasticity dilemma in the adaptive train scheduling setting.
Sources
- Learning to Continually Learn
- Adaptive Rational Activations to Boost Deep Reinforcement Learning
- Continual Backprop: Stochastic Gradient Descent with Persistent Randomness
- Maintaining Plasticity in Deep Continual Learning
- Building a Subspace of Policies for Scalable Continual Learning
- A Neural Dirichlet Process Mixture Model for Task-Free Continual Learning
- Linear Mode Connectivity in Multitask and Continual Learning
- Flatland-RL : Multi-Agent Reinforcement Learning on Trains
- Continual and Multi-Task Architecture Search
- Policy Distillation
- Progressive Neural Networks
- Proximal Policy Optimization Algorithms
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: Local-Consistency Optimisation for Bayesian Neural Networks
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks