- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper proposes NC-CLIP, a vision-language alignment method built on SmartCLIP. It introduces symmetric shared routing to select a common feature support for image and text representations, fixed ETF anchors with Sinkhorn-balanced assignments to regularize the representation geometry, and shared-only cross-modal reconstruction to preserve useful information. The method is evaluated on image-text retrieval, zero-shot classification, and group-robustness benchmarks. The authors report improvements over SmartCLIP on retrieval and worst-group accuracy, along with several component ablations.
Critical strengths
S1. The paper addresses a relevant problem in vision-language alignment, and the reported experimental results are encouraging. In particular, NC-CLIP improves retrieval on several benchmarks and achieves noticeable worst-group accuracy gains on Waterbirds and CelebA. The authors also provide several ablations to examine the proposed components.
Critical weaknesses
W1. Poor presentation and inconsistencies in the method description. I found the paper unnecessarily difficult to follow, particularly the figures. Figure 1 introduces common and unique masks, but the unique mask is not explained in the subsequent method. Figure 2 instead uses fine and coarse masks without clarifying their relationship to Figure 1. The key averaging operation for shared routing is also not clearly illustrated. More importantly, Figure 2 shows reconstruction toward full image/text representations, which appears inconsistent with the shared-only reconstruction described in Eq. (11). There are also several apparent typos in the figures. These issues go beyond minor presentation problems and make it difficult to understand the actual method.
W2. The contribution of the individual components to robustness remains unclear. The paper reports noticeable worst-group accuracy improvements, but the targeted ablations mainly examine the proposed components using retrieval or geometric metrics. Shared routing and independent routing achieve nearly identical retrieval performance, while exact ETF and random fixed anchors also perform similarly on retrieval. Although the authors acknowledge these results, it remains unclear how much each component contributes to the robustness improvements. More direct worst-group accuracy ablations would help support the proposed design.
Questions for authors
Q1. Could the authors clarify the relationship between the common/unique masks in Figure 1 and the fine/coarse masks in Figure 2? Also, why does Figure 2 show full-representation reconstruction when Eq. (11) describes shared-only reconstruction?
Q2. Could the authors clarify how shared masks are computed during retrieval? In particular, are the masks generated separately for each query-candidate pair, as in pair-dependent scoring? A brief description of the inference procedure would be helpful.
Q3. Could the authors provide worst-group accuracy ablations for the main components, especially shared versus independent routing and exact ETF versus random fixed anchors? This would help clarify which design choices contribute most to the robustness improvements.