- Rating
- 3
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper argues that a cumulative merged task vector loses some directions that matter to earlier tasks. CUSP-Merge therefore maintains a fixed-size memory of historical directions, with shared directions for repeated cross-task support and private directions for task-specific information. The memory also sets direction-level protection strengths when a new model is merged. Experiments measure final accuracy, forgetting, directional coverage, and storage across CLIP backbones and task orders.
Critical strengths
- The paper separates loss caused by a fixed rank from loss caused by reconstructing history from the merged vector. The equal-rank coverage analysis supports this distinction.
- The shared–private design has task-level evidence: on ViT-B/32 with 20 tasks, it raises worst-task accuracy from 35.43% for Single Memory to 43.52%.
- The growing-memory comparison makes the fixed-budget trade-off measurable across sequence lengths.
Critical weaknesses
- The average gain over NUFILT is modest in several 8- and 14-task settings. The paper does not report paired significance tests for this strongest baseline.
- CUSP-Merge has more negative BWT because it reaches higher accuracy when tasks first arrive.
- The full performance-versus-storage analysis focuses on Growing Memory. It does not give the same accounting for NUFILT and the other strongest continual-merging baselines.
- Directional coverage is measured on task vectors. Higher coverage is consistent with better retention, but it is not a direct measure of which directions preserve task accuracy.
- The experiments use related CLIP vision encoders and classification tasks. The memory rule’s behavior with substantially different model structures remains untested.
Questions for authors
Please refer to weakness.