acceptodds
Withdrawn from ICLR 2027

Not All Evidence Is Equal: Asymmetric Referential Evidence Modeling for Referring Remote Sensing Image Segmentation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Referring Remote Sensing Image Segmentation (RRSIS) aims to identify and segment the target specified by a natural-language expression in a remote-sensing image. A central challenge is to distinguish the intended target from multiple visually and semantically plausible candidates. Existing methods primarily enrich discriminative information, but often overlook how evidence with different functions should influence referent identification. We argue that different forms of referential evidence should be assigned authority according to their functional roles. Based on this principle, we propose Asymmetric Referential Evidence Modeling (\arem), which forms role-specific referential evidence, constructs positive-anchored dense grounding with bounded role-aware correction, and calibrates prompt decoding through discriminative training and controlled refinement. Extensive experiments on RefSegRS and RRSIS-D demonstrate the effectiveness and efficiency of \arem, while cross-architecture and cross-domain evaluations further support its applicability beyond the primary setting.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Referring remote sensing image segmentation, vision--language grounding, promptable segmentation, multimodal learning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_64xJOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes Asymmetric Referential Evidence Modeling (AREM) for Referring Remote Sensing Image Segmentation (RRSIS). To address referential ambiguity in dense remote sensing scenes, the authors introduce three components: Role-Asymmetric Evidence Formation (RAEF) to separate referent-supporting and exclusion-oriented cues, Positive-Anchored Evidence Grounding (PAEG) for bounded role-aware correction, and Evidence-Calibrated Prompt Decoding (ECPD) for discriminative training and refinement.

Critical strengths

The motivation to address referential ambiguity in remote sensing imagery, where multiple visually similar candidates often exist, is well-identified and practically relevant.

Critical weaknesses
  1. The motivation and framework diagrams in Figure 1 and 2 appear overly abstract and resemble AI-generated conceptual illustrations rather than clear scientific visualizations.
  2. The main contributions appear to be an incremental stacking of modules onto the existing RS2-SAM2 framework. The conceptual framing of asymmetric evidence authority and bounded correction is largely an over-packaging of standard deep learning practices.
  3. As listed in Table 4, the cross-dataset transfer experiment on RefCOCO is unconvincing. The authors only compare Baseline vs. Baseline+AREM using the same underlying architecture. They fail to compare the AREM-enhanced model against actual SOTAs specifically designed for natural referring image segmentation. The results do not prove generalization in the broader referring segmentation domain.
  4. Obviously, the computational burden of AREM is heavier than most existing methods, rendering it impractical for real-world remote sensing applications, especially for high-resolution scenarios.
Questions for authors

See weaknesses.

Reviewer_S4yeOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

AREM proposes a referringremote sensing image segmentation pipeline with role-conditioned queries, positive-anchored dense prompting, and candidate-based refinement. Experiments cover RefSegRS and RRSIS-D, with additional evaluations across architectures and domains.

Critical strengths
  • Distinguishing the referred object from nearby similar instances is a relevant problem, and the RefSegRS results show improvements over the reported baseline.
  • The supplement provides code and substantial component, efficiency, and error analyses, making the proposed system easier to examine.
Critical weaknesses
  • The central novelty claim is not convincing. IP-SAM already uses positive and negative prompts, asymmetric negative gating, and a residual anchored to the positive prompt, yet it is not discussed. Although the tasks differ, AREM needs to establish a clearer conceptual advance over these existing mechanisms.
  • The interpretation of the exclusion tokens is insufficiently justified. In Eqs. (3)–(5), negating cosine responses and supervising complementary masks still encourages foreground alignment of the underlying tokens, with a different pooling rule. This does not by itself establish distinct exclusion semantics.
  • Figure 3(a)’s selectivity result is partly built into the controls. Shared queries use the same map for both roles, forcing their selectivities to have opposite signs; positive-only queries have a zero exclusion map. Consequently, this diagnostic provides limited independent evidence of learned role separation.
  • The candidate score in Eq. (17) introduces a mask-size preference because overlap is averaged over the whole image. With uninformative role maps, the default weighting favors smaller masks. An area-normalized control is needed before attributing the improvement specifically to exclusion-aware selection.
  • The counterfactual ablation in Table 11 is not cleanly controlled in the supplied implementation. The index builder substitutes whole captions and groups them by token differences without preserving the remaining semantic content. These comparisons therefore do not isolate attribute changes from relation changes.
  • Figure 1&2 has the appearance of an AI-generated schematic. The text is disproportionately small relative to the graphical blocks, the feature-flow arrows are insufficiently specified, and labels such as “Referent-Suppoting” contain errors. The figure needs a careful technical redraw.
Questions for authors
  • Does role-aware candidate weighting still help after controlling for its implicit preference for smaller masks?
Reviewer_QJnaOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper adds eight learnable query vectors, similar to DETR object queries, to text-image segmentation models such as RS2-SAM2. In these models, BEiT-3 encodes the image and the expression, a coarse 256x256 probability map is computed from the features, and SAM2 takes it as a dense prompt.

The proposed vectors read from the text and visual tokens. Four are trained to respond to the target and four to all non-target pixels. The first four are used to recompute the probability map, and the other four can only scale each of its values up or down by at most 35%.

Compared with the published RS2-SAM2 results, it achieves marginal improvement.

Critical strengths

The added module is lightweight, spatial (storage) cost is negligible and time cost is small, while on particular settings, the improvement is significant (RefSegRS).

The appendix provides an extensive set of ablations.

Critical weaknesses

(1) The added object is not justified. The paper adds to RS2-SAM2 eight learnable query vectors, similar to DETR object queries. The paper does not explain why such an additional object is needed. Its motivation is expressed through undefined self-coined terms, such as "functional authority", "semantic-depth reconciliation" and "dense referential state".

(2) The central claim is not supported by the paper's own experiments. The claim is that the gain comes from splitting the eight queries into four trained on the target and four trained on non-target pixels. However, without the split, i.e. with all eight queries trained on the target, mIoU already rises from 72.02 (the reproduced baseline, Table 3) to 75.62 (Table 12). The split itself adds only 1.03 mIoU, from a single run. Moreover, the second group is trained on all non-target pixels (Eq. 5). Most of these pixels are background, so the second group does not learn to reject other objects of the same category.

(3) The state-of-the-art claim does not hold. The paper cites RSRefSeg 2 (Chen et al., 2026) in Section 2.2 but does not include it in Tables 1 and 2. RSRefSeg 2 reports 77.39 mIoU on RefSegRS and 69.17 mIoU on RRSIS-D, both higher than AREM (76.65 and 67.23).

References

  • Chen, K., Liu, C., Chen, B., Zhang, J., Zou, Z., Shi, Z. (2026). "RSRefSeg 2: Decoupling referring remote sensing image segmentation with foundation models." IEEE TGRS. arXiv:2507.06231.
Questions for authors

(1) What information do the eight query vectors provide that the BEiT-3 text and visual features, from which RS2-SAM2 already computes the dense prompt, do not? An experiment that adds the same number of parameters without the queries would help answer this.

(2) What is the variance of the experiment of all eight queries trained on the target vs. the 4+4 split?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.