acceptodds
Withdrawn from ICLR 2027

GLSC: Global-Local Semantic-Structural Collaboration for Semi-Supervised Remote Sensing Segmentation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Vision-language models and self-supervised vision foundation models provide complementary priors for semi-supervised remote sensing segmentation: CLIP offers category-level semantic knowledge, whereas DINOv3 preserves fine-grained spatial structures. However, effectively coordinating these heterogeneous priors at both category and spatial levels remains challenging. We propose GLSC, a global–local semantic–structural collaboration framework that strengthens the interaction between CLIP and DINOv3 through two complementary modules. First, Shared-Query Response Fusion (SQRF) uses CLIP text prototypes and learnable DINOv3 queries to project the two visual streams into category-conditioned response spaces and fuses their responses while preserving an explicit category dimension. Second, Dual-Context Response Refinement (DCRR) refines the fused responses by modeling intra-category spatial dependencies under visual-feature guidance and inter-category relationships under query guidance. The refined responses are subsequently aggregated and reintegrated into the original visual representations through residual enhancement. During training, an asymmetric auxiliary guidance loss encourages DINOv3 to learn from CLIP, while gradients from this auxiliary pathway are stopped at the CLIP branch. Experiments on Potsdam, LoveDA, and WHDLD under four labeling ratios show that GLSC outperforms the reported Co2S results in all 12 dataset–ratio settings, achieving an average absolute improvement of 1.14 mIoU points and a maximum gain of 2.11 points on LoveDA with 1/8 labeled data.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Semi-Supervised Semantic Segmentation, Remote Sensing, Vision Foundation Models, Vision-Language Models, Cross-Model Collaboration, Response Refinement

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_mYezOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes GLSC for semi-supervised remote sensing segmentation, combining CLIP and DINOv3 through shared query response fusion, spatial and category response refinement, and asymmetric auxiliary guidance from CLIP to DINOv3. Inference uses only the native CLIP branch. Experiments on three datasets across twelve labeling settings report numerical improvements over published Co2S results.

Critical strengths
  1. The paper addresses a practically relevant problem by exploring complementary foundation model priors for remote sensing segmentation with limited pixel annotations. The reported configuration uses only the native CLIP branch during inference, offering potential practical value without additional computation at test time.
  2. The paper evaluates the method on three datasets under twelve combinations of datasets and labeling ratios, reporting numerical improvements over the published Co2S results in all settings.
Critical weaknesses
  1. he methodological contribution beyond closely related work is not sufficiently established. Co2S already combines CLIP and DINOv3 for the same task using text prototypes, learnable category queries, and confidence based collaboration. DCRR also shares the main conceptual structure of CAT-Seg, including spatial aggregation followed by category aggregation with visual and semantic guidance. The paper introduces specific differences, but does not adequately demonstrate their significance beyond adapting and combining these existing mechanisms.
  2. The connection between the auxiliary pathway and the evaluated CLIP branch remains unclear. Section 3 and Appendix D state that CLIP features entering the collaborative pathway are detached, collaborative supervision is applied only to DINOv3, and reverse guidance has zero weight. The stated native branch objectives do not explicitly include supervision from DINOv3 to CLIP. The current description therefore does not establish the computational or supervisory path through which SQRF and DCRR affect the native CLIP predictions used for evaluation.
  3. The main comparison does not sufficiently isolate the contribution of GLSC. Co2S results are taken from the published paper rather than reproduced under matched training conditions. Although the reported improvements span twelve settings, differences in implementation and training randomness remain possible explanations for part of the gains. A matched Co2S baseline would provide stronger evidence for the effectiveness of the proposed changes.
Questions for authors
  1. What evidence demonstrates that shared query fusion and DCRR provide benefits beyond simpler response fusion and CAT-Seg style spatial and category aggregation within the same training framework?
  2. Given the detached CLIP features, supervision applied only to DINOv3, and zero reverse guidance weight, through what computational or supervisory dependency do SQRF and DCRR affect the native CLIP branch used for evaluation? Is any such dependency omitted from the stated objectives?
  3. Can the paper provide a Co2S baseline reproduced under the same data splits, implementation, and training settings as GLSC, together with repeated runs to assess whether the reported improvements persist after accounting for training variability?
Reviewer_sSzbOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes GLSC, a global–local semantic–structural collaboration framework for semi-supervised remote sensing segmentation. It attempts to coordinate heterogeneous semantic and structural priors from CLIP and DINOv3 via Shared-Query Response Fusion (SQRF) and Dual-Context Response Refinement (DCRR).

Critical strengths

The topic of semi-supervised remote sensing segmentation is relevant, and leveraging foundation models is a reasonable direction.

Critical weaknesses
  1. The bibliography is surprisingly sparse for a top-tier conference submission, containing only around 20 references. This severely limits the comprehension of the proposed method. Notably, the information of the first and last references is incomplete.
  2. The paper fails to discuss and compare recent, highly relevant advancements in semi-supervised segmentation leveraging Vision Foundation Models (e.g., SemiVL, UniMatchV2).
  3. The reported improvements are marginal, with some component gains as low as 0.06 mIoU in Table 3. More concerningly, the authors admit in Section 4.3, These are single-run, joint-training outcomes rather than variance-controlled component effects. The paper fails to provide any statistical significance testing, making it impossible to verify whether each component actually contributes to the performance or if the gains are merely due to random initialization.
  4. The paper emphasizes that At evaluation, the DINOv3 branch and collaborative pathway are removed, and predictions are produced solely by the native CLIP branch. This reveals a major contradiction: if the DINOv3 structural priors and the proposed modules are effective, why are they entirely discarded at inference?

SemiVL: Hoyer L, Tan D J, Naeem M F, et al. SemiVL: semi-supervised semantic segmentation with vision-language guidance[C]//European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2024: 257-275.

UniMatchV2: Yang L, Zhao Z, Zhao H. Unimatch v2: Pushing the limit of semi-supervised semantic segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2025, 47(4): 3031-3048.

Questions for authors

See weaknesses

Reviewer_2auwOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposes GLSC, a training-time framework that combines CLIP and DINOv3 through shared category responses, spatial–category refinement, and asymmetric guidance for semi-supervised remote-sensing segmentation. The motivation is relevant, but the current evidence does not sufficiently establish the contribution and necessity of the proposed components

Critical strengths
  1. Combining CLIP’s semantic priors with DINOv3’s spatial representations addresses a relevant challenge in remote-sensing segmentation. Category-conditioned responses offer a plausible interface for coordinating these heterogeneous representations.

  2. The experiments cover three datasets and four labeling ratios per dataset. The reported configuration retains only the native CLIP branch at inference, avoiding additional test-time computation.

Critical weaknesses
  1. CLIP features are detached in the collaborative pathway, CLIP-side auxiliary supervision is disabled, and reverse DINO-to-CLIP guidance has zero weight. Equation (11) does not explicitly identify a remaining pathway through which the proposed auxiliaries affect the evaluated CLIP branch. The authors should specify any shared parameters, cross-branch pseudo-labels, or other dependencies and verify this pathway experimentally.

  2. Category-conditioned similarity volumes, sequential spatial and class aggregation, and visual/text guidance closely resemble CAT-Seg. PartCATSeg further introduces DINO-based structural guidance. The paper should precisely distinguish its shared-query fusion and refinement design from these approaches and demonstrate the value of those differences through controlled comparisons.

  3. The competing results are taken from published reports rather than matched reruns. The reported improvements may therefore also reflect implementation, initialization, or training differences. At minimum, Co2S, ACG-only, and full GLSC should be evaluated under identical settings with multiple random seeds.

  4. In Table 3, adding VSR to SQRF reduces mIoU from 61.62 to 61.58, while adding SCR yields only 61.68. The full combination reaches 62.03, but these single-run results cannot establish reliable complementarity. Repeated experiments and comparisons with simpler fusion, standard distillation, and alternative refinement orders are needed.

  5. Qualitative response maps and overall mIoU do not directly establish improved boundaries, small-object preservation, or reduced category confusion. Figure 7 also shows that the LoveDA gain is concentrated in Barren, while five of seven classes decline. Boundary metrics, size-specific results, and class-confusion or pseudo-label-quality analyses would better substantiate the proposed mechanisms.

Questions for authors

Please see the weaknesses.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.