- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes RAM, a replay-based Domain Incremental Learning (DIL) framework that treats memory management as a reinforcement learning problem. Instead of maintaining a static replay buffer, RAM stores semantic and domain representations for each memory sample and learns a policy that decides whether to retain, repair, or remove samples. The policy is guided by representation drift and validation feedback. The paper also derives a mutual-information-based adaptation bound to motivate disentangling semantic and domain representations. Experimental results on DomainNet, Camelyon17, iWildCam, and Rotated-MNIST show improvements over replay baselines such as ER, DER++, X-DER, and AdaER.
Critical strengths
- The motivation is clear and intuitive. The central observation that stored replay samples can become stale under evolving domains is reasonable and underexplored relative to sample selection literature.
- Strong empirical results. The reported gains over ER, DER++, X-DER, AdaER and several prompt-based approaches are non-trivial across multiple datasets and architectures.
- The proposed method could orchestrate multiple integration into one coherent system.
Critical weaknesses
- Novelty. The repair action sounds novel at first glance. However, inspecting Eq. (13), repair merely updates stored features using an EMA: z^{old} = (1−\beta)z^{old} + \beta z^{new}. Overall, the memory management systems uses standard operations for the flow of experience replay.
- The reinforcement learning controller's role is actually questionable in the pipeline as the proposed heuristic rule already partitions the drift plane. Thus, the policy may simply learn the handcrafted rule.
- The paper presents a "Mutual-Information Sequential Adaptation Bound". This theoretical proof mostly plays a motivational role for the proposed algorithm rather than providing new guarantees.
- The theoretical arguments built upon the Wasserstein framework depend on an approximately constant covariance assumption throughout memory updates, limiting the validity of the conclusions in realistic settings. Thus, the empirical gains could not be well-justified.
- The ablations remove components individually, but there is no clean decomposition demonstrating each algorithm related to reinforcement learning, disentanglement, and repair operation in memory.
Questions for authors
- Regarding the assumption that covariance remains approximately stable during memory updates. How sensitive is the method to violations of this assumption, and do the theoretical conclusions still hold under realistic feature distributions?
- Since replay is performed using the original raw samples, while the "repair" operation only updates stored feature representations via EMA, what is the precise mechanism through which repair improves replay quality?
- Have the authors analyzed cases where the learned policy disagrees with the drift-based heuristic? If so, are these disagreements responsible for the observed gains over heuristic policies?