Learning to Manage Memory for Domain Incremental Learning via Adaptive Replay

Real-world data often evolves across domains, challenging models to adapt to new distributions while retaining previously acquired knowledge. Replay-based methods for Domain Incremental Learning (DIL) address this challenge by interleaving current data with samples from previous domains. However, as the model and domains evolve, some stored memories may become stale, redundant, or less useful for replay. The challenge is therefore not simply to preserve memory, but to keep it useful. Inspired by neuroscience studies that view forgetting as an active and functional process of memory regulation, we propose Reinforcement learning-based Adaptive Memory (RAM), a framework that treats memory as an adaptive resource rather than a static collection of raw samples. We further present a Mutual-Information Sequential Adaptation Bound that relates cross-domain error bounds to the mutual information between semantic-related representations and domain identity. This analysis motivates separating semantic- and domain-related representations, whose drift provides structured signals for assessing memory throughout training. Guided by drift-based priors and performance feedback across all seen domains, RAM learns a lightweight policy to actively retain, repair, or remove memory entries under a fixed budget. Experiments on four DIL benchmarks demonstrate that RAM consistently improves accuracy and reduces forgetting across different backbones, outperforming the evaluated baselines and highlighting the value of active memory regulation for continual learning.