- Rating
- 2
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
In Class-Incremental Learning (CIL) using Vision-Language Models like CLIP, existing methods rely on a fixed global ratio to combine visual prototypes and text classifiers. The authors show this is suboptimal because the best fusion ratio varies across inputs and classes.To fix this, the paper introduces GMF-CLIP, a framework featuring: Gating Network (Dynamically adjusts modality contributions for each input and candidate class, supported by retention and augmentation-consistency losses.) and Visual Prototype Refresh (Automatically recalibrates old-class prototypes based on shifts in the gating ratios). Extensive experiments show that GMF-CLIP achieves state-of-the-art performance across four standard datasets.
Critical strengths
The paper introduces GMF-CLIP, The major contributions are 1) a Gated Modality Fusion framework for class-incremental learning that replaces suboptimal fixed global ratios with a lightweight gating network to dynamically adapt visual prototype and text classifier contributions per input and class and 2) Visual Prototype Refresh (VPR) mechanism to effectively prevent catastrophic forgetting. The proposed method presents state-of-the-art performance across four standard benchmarks with lower auxiliary memory and computational overhead than task-expanding architectures.
Critical weaknesses
Major
The contributions are minor and lean more toward engineering, although the method shows better performance than prior works. The proposed method essentially uses a CLIP-based CIL framework, and its primary contribution is learning per-class modality fusion weights, which can be seen as an optimization of existing systems rather than a fundamental change.
The discrepancy between visual and textual embedding distributions was already well-studied in the computer vision community in the late 2010s or early 2020s for image classification. Consequently, the paper's motivation lacks novelty. And, the reviewer is unsure if this motivation is relevant for CIL. The major goal for CIL is maintain the learning experience while accepting new experience. Reviewer is unsure the connection between the discrepancy and CIL.
The differences from prior work are not stated sufficiently.
Minor
Some figures are too small to read the text inside them.
Questions for authors
It would be better to explain what is the meaning of the equations, instead of presenting equation and relying on readers to understand the equations.