Method Section
Comment 1 (Figure 1 and Eqs. (1), (4), (5): inconsistent notation and incomplete definitions).
Figure 1 writes the total loss as "L = L_ortho + L_drift + L_reg + L_cls," but Eq. (5) includes a weight λ on L_drift (λ = 1000 in Table 8). The two formulas should match. Also, Eq. (1), "Z_cv = Cost(F(I))," does not show the text embeddings used to compute category similarities, while Eq. (4) does not explain how each expert output is pooled or projected to match the CLIP text embedding. Please make these key definitions more explicit. In addition, Figure 1 uses "Rol" instead of "RoI".
Comment 2 (Sec. 3.1 vs. Sec. 3.4: identification and treatment of old-object proposals).
The paper says "proposals matching previous categories are excluded from the classification loss rather than treated as negatives" (Sec. 3.4). However, Sec. 3.1 says that only current classes have annotations. Without old-class labels or pseudo-labels, how can these old-object proposals be identified? Please clarify whether the method merely skips old-class classification terms or also ignores particular proposals. Please also clarify whether unlabeled old-class objects are still treated as background during RPN training, and how the proposed spatial attention guidance addresses this issue.
Comment 3 (Sec. 3.2: what exactly is claimed to be stable).
The authors claim that the cost volume provides a "stable per-category spatial signal that does not degrade as new tasks arrive" (Sec. 3.2). Yet they also state that "all other modules (cost-volume pathway, FPN, RPN, current-task experts) are trainable" (Sec. 3). Freezing CLIP preserves the raw image–text similarities, but it does not ensure that the refined response maps produced by trainable aggregation and decoding remain unchanged. The orthogonality loss reduces overlap between classes, not changes within the same class across tasks. Please clarify which spatial signal is claimed to be stable and show how old-class responses change after learning new tasks.
Comment 4 (Sec. 3.4: EWC implementation details are missing).
Regarding EWC, the paper says, "The diagonal Fisher information is computed after each task from the total loss gradients" (Sec. 3.4). However, it does not explain exactly how Fisher importance is estimated and normalized, which earlier parameters are used as references, or how information is carried across tasks. Table 8 provides the weight λ = 1000, but not these update rules. As Table 11 shows large benefits from drift regularization, the complete loss equation and step-by-step update procedure are important for reproducing the method.
Experiments Section
Comment 5 (Table 3: cost-volume gating and per-category experts are conflated).
Table 3 explicitly states that "Cost-volume gating and per-category experts are ablated jointly." Therefore, the experiment shows that the combined design helps, but not whether the gains mainly come from the cost-volume gate or from having separate experts whose old weights are frozen. The additional comparison of convolutional and MLP experts does not answer this question. Please report the two components separately, ideally with similar model capacity and the same EWC setting.
Comment 6 (Table 6: all-class AP does not match the class-wise averaging rule).
In Table 6 (COCO 70+10, FG-CLIP ViT-L/14), the reported entries are "40.6" for current-class AP, "21.2" for previous-class AP, and "21.7" for all-class AP. Under the standard class-wise averaging rule, the all-class value should be (10 × 40.6 + 70 × 21.2) / 80 = 23.625, approximately 23.6, rather than 21.7. Please check the numbers or explain whether a different evaluation rule was used.
Comment 7 (Sec. 4.4, Tables 7 and 14: better initialization is conflated with better retention).
The paper states that CL-CLIP "improves old-class and all-class AP over the corresponding baseline" (Sec. 4.4). However, its initial detector is already stronger. On VOC 10+10, the FineCLIP baseline has 80.2 AP after Task 1 (Table 7), while CL-CLIP has 86.2 (Table 14, seed 0). After Task 2, old-class AP is 55.8 versus 64.8. Thus, the old-class AP drops are 24.4 versus 21.4 points: a 3.0-point difference in forgetting-related decline, rather than the full 9.0-point final gap. On COCO Task 4, old-class AP improves only from 17.4 to 18.6. Please report forgetting measures that separate better initial accuracy from better retention.
Appendix Section
Comment 8 (Appendix F: gradient path of frozen old-class experts).
Appendix F says that "each frozen old-class expert still contributes forward activations (and gradients through the shared features it feeds)." However, Sec. 3.4 defines the classification loss only for current-class scores, and no loss using the old experts' outputs is clearly described. A frozen expert can pass gradients to its inputs if its output is used by a loss, but simply running its forward pass does not create such gradients. Which loss, if any, directly uses old-expert outputs? Please clarify the actual gradient path.
Comment 9 (Appendix F and Table 13: cost scaling with present vs. learned categories).
Appendix F says that "the cost per image scales with the categories actually present rather than the full vocabulary." The same section also says that the expert pipeline operates over "all seen categories." If every learned expert runs on every image, computation should depend on the number of learned categories, not on how many object classes appear in that image. No mechanism for skipping irrelevant experts is described, and Table 13 shows FLOPs increasing from 289.48G to 594.69G as the vocabulary grows from 19 to 80 classes. Please explain which interpretation is correct.
Minor Presentation and Formatting Issues
Comment 10.
Figure 4 labels the first visualization "Original CLIP," while the text and caption refer to "Original FineCLIP".
The bibliography uses inconsistent venue naming conventions (e.g., abbreviated versus full ECCV titles) and DOI formats (plain identifiers versus full URLs), and some entries lack complete publication details.
The Conclusion contains "confirm substantially improvements," which should be "confirm substantial improvements".
"Human participations" in the Ethics Statement should be "human participants".