- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3(+1)
Read review
Paper summary
This paper proposes SRCL, a continual learning method that decouples short-term adaptation from long-term consolidation. The approach incorporates a LoRA module—acting as working memory—onto a frozen backbone model, alongside a long-term memory component that expands incrementally. Gradient consistency is used to determine the timing of consolidation, while projection operations—which steer new information away from previously consolidated input subspaces—dictate the directions in which this information is stored. Experimental results demonstrate that this method yields significant improvements in both sequential learning accuracy and knowledge retention.
Critical strengths
- SRCL separates when to consolidate knowledge from where to store it. Table 3 supports the importance of both components: with six consolidation events and the same rank budget, the full method achieves 68.4% math accuracy, compared with 60.3% for periodic consolidation and 61.7% without projection. These controls provide evidence that the gains reflect consolidation timing and storage geometry, beyond simply allocating additional memory.
- Across the tested baselines, SRCL consistently improves over prioritized replay on both math and code and performs comparably to InfLoRA. Achieving this performance without boundary annotations or stored training examples supports the practical value of the proposed consolidation mechanism within the evaluated settings.
Critical weaknesses
- Unclear treatment of working memory in the main evaluations. Section 3.1 states that “At evaluation the WM is cleared,” whereas Section 4.5 states that “Every method is evaluated with its full state at the end of each domain—for SRCL, Θ + Φ + ϕ.” These descriptions appear inconsistent and leave uncertainty about the memory state used to obtain the main CRP and BWT results. Full-state evaluation is a reasonable measure of overall continual-learning performance.
- Limited evidence for sustained learning under a fixed budget. The selected math configuration uses six of the 32 available consolidation events. Although the paper studies budget exhaustion in ablations and contamination experiments, it remains unclear how the recommended configuration balances new-rule acquisition and old-rule retention on longer, clean streams approaching capacity. The interference bound does not establish continued learning, since the LTM stops accepting updates once the budget is exhausted.
Questions for authors
- Could the authors reconcile Section 3.1 (“At evaluation the WM is cleared”) with Section 4.5 (“Every method is evaluated with its full state at the end of each domain—for SRCL, Θ + Φ + ϕ”)? Was WM cleared or retained when computing the main CRP and BWT results, particularly those in Table 1? If WM was retained, could the authors provide paired evaluations of Θ + Φ + ϕ and Θ + Φ at the same checkpoints, including per-domain accuracy and the corresponding BWT? The comparison should leave the training trajectory unchanged and remove WM only for the additional evaluation, so that it isolates the contribution of consolidated memory.
- Under the fixed memory budget and recommended configuration, how does performance evolve on longer, clean streams that introduce additional rules or domains? Could the authors characterize this progression by reporting cumulative consolidation events and rank usage, accuracy on newly encountered domains, and retention of those domains after subsequent learning? Of particular interest is when acquisition or retention begins to deteriorate as the budget is approached and eventually exhausted. This would help establish how much continuing novelty the method can accommodate and the practical range over which it maintains both acquisition and retention.
- The experiments do not include CLS-ER or SYNERgy, which also use multiple memories and learn without task boundaries. Although these methods require storing past examples, comparison with at least one would help assess whether SRCL’s advantage over PER extends to replay methods with a more similar memory design. Could the authors compare SRCL with CLS-ER or SYNERgy, or explain why these methods are unsuitable for this setting? Reporting accuracy alongside memory and computing costs would help assess the trade-offs fairly.