- Rating
- 3(+1)
- Confidence
- 3(−1)
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper investigates the variation of region cues across feature depths and their preferences across different tasks and regions. To address this, the authors propose ReGDec, which decomposes region features into CLS-aligned components and orthogonal residuals, and adaptively fuses them across layers. Extensive experiments on CLIP-family backbones and dense prediction benchmarks demonstrate the effectiveness and transferability of the proposed method.
Critical strengths
-
This paper includes a dedicated motivation section that clearly illustrates the motivation behind the proposed method through experimental results.
-
Both quantitative and qualitative analyses thoroughly demonstrate the simplicity and efficiency of the proposed method.
Critical weaknesses
-
The state-of-the-art performance of the proposed method is questionable. The methods used for comparison in the experiments are relatively outdated, with the most recent baselines published in 2023. Why are methods published in 2025 and 2026 not included in the comparisons? In particular, I noticed that the paper cites recent methods such as DeCLIP (2025), but does not include them in the experimental comparisons.
-
The proposed method involves numerous hyperparameters, but the underlying mechanisms governing their joint effects are not sufficiently investigated. For instance, slight changes in these hyperparameters may lead to substantial fluctuations in model performance.
-
The related work section is not appropriately presented. The core discussion of related work spans fewer than 24 lines. In particular, Section 5.2, "Open-Vocabulary Dense Perception," lacks detailed explanations of how previous methods work, making it difficult to distinguish the proposed method from existing approaches. Furthermore, why is the related work section placed after the experiments rather than immediately following the introduction?
Questions for authors
Please refer to 'Critical Weaknesses'.
5 comments
Thank you for the authors' detailed response. However, I would like to see actual revisions to the manuscript rather than simply receiving explanations in the response. Until substantive revisions are made and presented, I will maintain my current score.
Response to the Reviewer’s Suggestion for Manuscript Revisions
Thank you for your valuable feedback. Following your suggestions, we have revised the manuscript, with the revised content highlighted in blue. Specifically, we have expanded the related-work discussion in the Introduction (lines 43–53 and 73–75), Section 5.2 (lines 512–528), and Appendix A (lines 866–928); added the DeCLIP comparison to Table 4(a) (lines 344–345); and included joint hyperparameter sensitivity analyses in Appendix B.2 (lines 1155–1224).
Response to Reviewer vBWW
We thank the reviewer for the careful review and constructive feedback. To address the concerns raised, we provide our responses below.
1. Comparison with SOTA
Thank you for pointing out that we cited DeCLIP (CVPR 2025) but did not include it in our detection comparison. Our original experiments focused on measuring the gain from adding ReGDec to the same detector, using EVA-CLIP and CLIPSelf to cover both standard and dense-adapted backbones. We evaluated the OV-COCO detector checkpoint released with the TPAMI 2026 extension of DeCLIP, kept the detector weights fixed and trained only the ReGDec selector, then used it to fuse features from multiple DeCLIP layers for each proposal in the detector’s VLM scoring branch. Adding ReGDec increased novel AP50 from 49.7 to 49.9 (+0.2 AP). We will include this paired comparison in the main paper.
Our paper examines how combining features across layers for each region can improve recognition with frozen CLIP-family encoders, and whether these gains carry over to open-vocabulary detection and segmentation. Our detection experiments therefore compare the same detector with and without ReGDec. CLIPSelf is particularly relevant because, like DeCLIP, it uses COCO training images to adapt pretrained CLIP visual encoders for dense perception in the OV-COCO setting. With the backbone and detector weights held fixed, adding ReGDec to the released F-ViT detector with CLIPSelf ViT-L/14 improved novel AP50 from 44.3 to 44.8 (+0.5). The gain on EVA-CLIP ViT-L/14 was larger, from 24.2 to 33.4 (+9.2). One plausible explanation is that CLIPSelf’s region-level self-distillation and DeCLIP's region-level distillation from DINOv2 already make their layer features more discriminative for region recognition, leaving less room for improvement from cross-layer fusion.
2. Joint sensitivity to hyperparameters
An exhaustive evaluation of all hyperparameter combinations would require more time than the rebuttal period allowed. We therefore conducted two important joint sensitivity analyses. The candidate window size ∣W∣ and sparsity K jointly determine how many layers are available for fusion and how many are retained. The teacher temperature τu and utility-loss weight λsel determine how strongly the teacher favors high-utility layers and how closely the selector is encouraged to follow that preference. We evaluated all nine combinations in each grid. Both tables report mean COCO Top-1 accuracy (%) across boxes, thing masks, and stuff masks.
Table R1. Candidate window × residual sparsity.
Table R2. Utility-loss weight × teacher temperature.
Increasing K from 4 to 5 helps more with nine candidate layers than with six and gives no gain with twelve, showing that the effect of residual sparsity depends on the candidate window. In the loss grid, increasing τu from 1 to 5 changes the mean by +0.45, +0.56 and +0.09 at the three respective loss weights. The range of mean Top-1 accuracy is 1.66 percentage points in Table R1 and 0.94 in Table R2. For stuff masks alone, the corresponding ranges are 2.61 and 2.44 percentage points. In the revised main paper, we will expand the joint sensitivity analysis to cover additional hyperparameter combinations.
3. Positioning relative to prior work
We introduced the closest methods in the Introduction and placed the full Related Work section after the experiments so readers could follow the layer-wise observation directly into the motivation and the method. We agree that readers currently reach the main comparison with prior work too late. We will bring the distinction from related methods into the Introduction, expand Section 5.2 with concrete examples from detection and segmentation, and use the appendix for the fuller survey.
For detection, F-VLM uses a frozen VLM to score proposed regions and combines those scores with the detector’s predictions. For segmentation, CAT-Seg aggregates pixel–text matching costs. ResCLIP is particularly relevant because it uses intermediate layers to refine patch attention. ReGDec pools the CLS-aligned components and orthogonal residuals within each supplied region and learns separate cross-layer weights for the two components. We will make this distinction explicit in the main paper.