Fast-Convergent Meta-RL via Gradient-Clustered BS Sampling for Edge Caching

arXiv:2609.16370 · cs.NI, cs.LG, cs.SY, eess.SY · Submitted 2026-09-14 · Read on arXiv

cs.NI, cs.LG, cs.SY, eess.SY

Submitted: 2026-09-14

Updated: 2026-09-14

License: http://creativecommons.org/licenses/by-nc-sa/4.0/

The gist: Wireless edge caching networks typically consist of many independent Base Stations (BSs), each facing its own request rate and content popularity profile.

Terminology

Abstract

Wireless edge caching networks typically consist of many independent Base Stations (BSs), each facing its own request rate and content popularity profile. Training a Reinforcement Learning (RL) caching agent from scratch at every BS forces each agent to relearn, through slow trial and error, a decision problem that is structurally identical across the network. Meta-reinforcement learning removes this redundancy by learning a shared initialization that adapts to any BS in a few local updates; however, meta-training itself becomes the bottleneck at scale: the meta-gradient must be estimated from a small subset of BSs at each meta-iteration, and sampling this subset uniformly at random yields a high-variance estimate, an issue existing meta-RL caching frameworks leave unaddressed. This paper proposes a meta-reinforcement learning framework for caching across independent, non-overlapping BSs that directly targets this bottleneck. Each BS runs a local Proximal Policy Optimization (PPO) agent, formulated as a Semi-Markov Decision Process (SMDP) over content popularity, size, lifetime, and importance, while a shared meta-policy is learned via a Model-Agnostic Meta-Learning (MAML)-style loop. To scale meta-training and accelerate convergence, we introduce gradient-based clustering, which groups BSs by local gradient similarity and draws from every cluster, in proportion to its size, at each meta-iteration. We prove, via an Analysis of Variance (ANOVA)-style decomposition of gradient variance, that this strategy yields a strictly lower-variance meta-gradient estimator than uniform random sampling under BS heterogeneity.

Sources

Related papers