- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes Structural Memory Consolidation for replay-based online class-incremental learning. At task boundaries, it selects anchors from the replay buffer and stores row-normalized similarity matrices between anchors from adjacent historical tasks. Subsequent updates reconstruct those relations and distil toward the stored matrices alongside the base training objective. Experiments use randomly initialized ResNet-18 models on CIFAR-10, CIFAR-100, ImageNet-100, and Tiny-ImageNet. The paper reports the highest final accuracy in 11 of 12 evaluated settings when either anchor-selection variant is allowed, and uses representation-drift measurements to examine the proposed mechanism.
Critical strengths
1 The proposed memory representation is compact and addresses a reasonable question. Storing relation matrices can preserve a previous encoder's geometry without retaining that encoder or full historical embeddings (Section 3.2). The adjacent-task construction limits snapshot storage to order scalars. Evaluating structural drift using class-prototype similarities rather than exactly the anchor-level training loss is also a useful diagnostic choice (Section 4.5). These are worthwhile aspects of the design, although their benefit needs a consistent evaluation and complete resource accounting.
Critical weaknesses
- The ablation and implementation description do not isolate the claimed contribution. Table 3 reports an increase from 12.35% to 19.64% when structural snapshots are retained but their consolidation objective is removed. Under the stated procedure, merely caching unused targets should not alter training; any remaining difference needs to be specified. Conversely, consolidation without stored structural targets is not defined.
Questions for authors
- Please provide the exact forgetting formula and the per-task accuracy matrices behind the ER and DVC Tiny-ImageNet entries. Were Tables 2 and 4 produced with the same runs and evaluation protocol?
- How does a matched relational-distillation baseline perform, and what is the complete extra storage and computation cost? In particular, how are cached anchors retained if reservoir sampling would otherwise evict them?