- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper studies sample-level displacement of latent representations between epochs ("representation flux") in replay-based continual learning. It reports that flux correlates with, and tends to precede, forgetting events. It then proposes FlowLess-R, an L2 penalty between the current features of buffer samples and features stored when those samples entered memory. FlowLess-R is added to ER, DER++ and ER-ACE and evalutated Split MNIST/FashionMNIST/CIFAR-10/TinyImageNet.
Critical strengths
The addressed problem is relevant and the method is simple.
Critical weaknesses
-
That forgetting stems from drift of old representations is the premise of all feature-distillation methods in CL, and it has been measured before (Davari et al., CVPR 2022; Caccia et al., ICLR 2022). The flux analysis restates this premise at sample level and remains correlational.
-
L2 feature matching against past features is standard: Jung et al. (AAAI 2018), Michieli & Zanuttigh (ICCVW 2019), LUCIR (CVPR 2019), PODNet (ECCV 2020), BFP (CVPR 2023), EFC (ICLR 2024). Using targets stored at buffer insertion is what DER (NeurIPS 2020) already does with logits. Thus, the novelty of the method appears to be very limited.
-
Replay samples are selected uniformly at random, with no comparison to or combination with coreset selection methods such as GSS, Rainbow Memory or OCS. This is a natural experiment given the sample-level analysis.
Questions for authors
N/A
3 comments
Official Comment By Reviewer hXB2
Thanks for the detailed response and the additional experiments.
My main concern remains. The rebuttal confirms that the regularizer is not novel, so the contribution rests on the flux analysis. However, the method does not use flux: it penalizes accumulated drift from anchors stored at insertion, which your new Table 2 shows to be less informative. One question: why not regularize flux directly?
On soundness, my rating reflects this gap between analysis and method and the evaluation issue that changed the main results during the rebuttal. I think the paper needs a major revision and a new round of review, so I keep my score.
Reponse to Official Comment By Reviewer hXB2
We thank the reviewer for the prompt response and clarification.
We agree that FlowLess-R constrains accumulated displacement rather than instantaneous flux. This distinction is intentional: direct flux regularization discourages consecutive representation changes but permits gradual accumulated drift, whereas fixed anchors preserve historical representations while allowing local adaptation, thereby providing a more persistent constraint against cumulative representation drift.
The two displacement measures are mathematically related through the triangle inequality, although controlling accumulated drift does not necessarily control instantaneous flux.
Importantly, we also investigate task-wise and epoch-wise anchor refreshing in the revised manuscript (Section 5, Table 5), which more closely approximates regularization of shorter-term representation changes.
Table 5: Anchor-refresh ablation (final average accuracy, %; ER + FlowLess-R).
| Anchor policy | SplitMNIST | SplitFashionMNIST | SplitCIFAR10 | SplitTinyImageNet |
|---|---|---|---|---|
| Fixed | 79.56 | 70.79 | 40.16 | 18.21 |
| Task-wise | 79.52 | 68.69 | 39.65 | 17.18 |
| Epoch-wise | 73.88 | 66.26 | 40.23 | 19.18 |
These results suggest that neither fixed nor frequently refreshed anchors universally dominate. Nevertheless, epoch-wise refreshing is not equivalent to directly minimizing instantaneous flux.
Our primary contribution remains the sample-level geometric and temporal characterization of forgetting. In particular, Table 2 of the revised manuscript demonstrates that flux provides complementary information beyond historical drift.
Table 2: Representation flux versus historical drift (concurrent forgetting-detection ROC-AUC).
| Features | SplitMNIST | SplitFashionMNIST | SplitCIFAR10 | SplitTinyImageNet |
|---|---|---|---|---|
| Historical L2 Drift | 0.643 | 0.617 | 0.525 | 0.576 |
| Historical Cosine Drift | 0.632 | 0.552 | 0.519 | 0.594 |
| Normalized Historical L2 | 0.653 | 0.654 | 0.526 | 0.597 |
| Historical Drift (Combined) | 0.742 | 0.715 | 0.536 | 0.621 |
| Representation Flux | 0.731 | 0.702 | 0.766 | 0.673 |
| Historical Drift + Flux | 0.776 | 0.764 | 0.766 | 0.721 |
Adding flux to combined historical drift improves ROC-AUC by 0.033, 0.049, 0.230, and 0.100, respectively, across the four benchmarks. These findings support the distinction between instantaneous flux and accumulated drift, demonstrating that flux captures complementary information about forgetting.
FlowLess-R serves as an illustrative stabilization intervention motivated by this geometric analysis, rather than a direct optimization of flux.
Response to Reviewer hXB2
We thank the reviewer for the constructive feedback and relevant references. We address the concerns below.
1. Representation flux and prior drift analyses
We acknowledge that representation drift has been studied previously (Davari et al., 2022; Caccia et al., 2022). Our contribution is not the discovery of drift, but its sample-level temporal characterization and relationship to forgetting. We examine whether elevated representation flux precedes forgetting events and whether it provides information beyond historical displacement.
The revised manuscript compares representation flux against historical L2, cosine, and normalized drift (Section 4, Table 2, p. 6). Adding flux to combined historical drift improves concurrent forgetting-detection ROC-AUC by 0.033 on SplitMNIST, 0.049 on SplitFashionMNIST, 0.230 on SplitCIFAR10, and 0.100 on SplitTinyImageNet. Paired 95% confidence intervals for these improvements exclude zero across all four benchmarks (Appendix B.1, Table 6, p. 19). These results demonstrate that flux provides complementary discriminative information beyond historical drift.
The distinction between instantaneous representation flux and accumulated historical drift is further formalized in Appendix A.6 (Proposition 3, pp. 17–18), showing that historical drift does not uniquely determine instantaneous flux.
2. FlowLess-R and feature distillation
We agree that L2 feature matching is well established in continual learning (Jung et al., 2018; Michieli & Zanuttigh, 2019; LUCIR, 2019; PODNet, 2020; BFP, 2023; EFC, 2024), as is storing historical targets, exemplified by DER (2020). We do not claim novelty for either technique.
Our primary contribution is the geometric and temporal analysis of catastrophic forgetting. FlowLess-R serves as a simple, controlled intervention illustrating how this analysis can motivate representation stabilization, rather than as a fundamentally new regularization method.
To strengthen the empirical evaluation, we added comparisons with frozen-teacher distillation, FER-style feature-and-logit preservation, and an adapted PODNet implementation under shared class-incremental protocols. Replay tensor storage is matched for the first three benchmarks; frozen-teacher parameters are excluded from this budget.
Final average accuracy (%, mean ± SD):
Source: Appendix D.5, Table 10, p. 29, with implementation details on pp. 28–29.
Fixed-anchor FlowLess-R improves over ER by 6.02, 6.54, 1.40, and 1.88 percentage points, respectively. It achieves the highest accuracy on SplitMNIST, while FER-style leads on SplitFashionMNIST, adapted PODNet on SplitCIFAR10, and epoch-refresh FlowLess-R on SplitTinyImageNet. Thus, no method universally dominates.
These comparisons support the practical value of FlowLess-R while reinforcing our central claim: the paper's primary novelty lies in characterizing the geometry and temporal dynamics of forgetting, with representation regularization serving as an illustrative intervention motivated by the analysis.
3. Replay selection strategies
Replay selection and representation stabilization address orthogonal aspects of continual learning: GSS, Rainbow Memory, and OCS determine which samples are retained, whereas FlowLess-R constrains how their representations evolve during subsequent training. We use uniform random replay to isolate the effect of representation stabilization independently of selection policies. FlowLess-R is compatible in principle with alternative replay-selection strategies; evaluating their combinations is beyond the scope of this study.
4. Soundness assessment
We respectfully request clarification of the soundness rating of 1. The stated concerns primarily address novelty and experimental scope. Could the reviewer identify any specific mathematical, methodological, or experimental validity issue that motivates this rating? We would welcome the opportunity to address it directly.
We thank the reviewer again for their consideration.