- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper targets reasoning-intensive referring expression comprehension (REC) in remote sensing imagery, where existing benchmarks can be solved by shallow category or position biases. The authors introduce RS-Ref, a distractor-controlled benchmark of 1,560 image-expression pairs with same-category distractors and multi-constraint descriptions, constructed through a filtering, LLM-assisted generation, and two-round human verification pipeline. They further propose GeoREC, a two-stage grounded visual reasoning framework that combines supervised fine-tuning on grounded chain-of-thought trajectories with GRPO-based refinement using outcome and trajectory-level rewards. Experiments evaluate 20 VLMs on RS-Ref and GeoREC on seven external remote sensing grounding benchmarks, reporting strong performance and diagnostic robustness under controlled perturbations.
Critical strengths
- RS-Ref explicitly targets shortcut-based grounding by enforcing same-category distractors and multi-constraint expressions, and the perturbation analyses provide useful diagnostic evidence that the benchmark depends more on complete referring expressions than existing datasets.
- The construction pipeline includes image filtering, LLM-assisted candidate generation, annotation rules, and two rounds of human verification, with explicit efforts to avoid overlap and data leakage. These details make the benchmark contribution more credible.
- The two-stage framework achieves the best performance on RS-Ref among the evaluated models and generalizes well to five of seven external grounding benchmarks. The ablation further supports the contributions of grounded trajectory SFT and trajectory-aware GRPO.
Critical weaknesses
- Many citations in the main text are not properly formatted or integrated into the sentences, which disrupts the reading flow. The authors should carefully check and correct the citation formatting throughout the paper.
- The proposed GeoREC framework largely follows a conventional two-stage training pipeline consisting of SFT and GRPO-based reinforcement learning. Although grounded reasoning trajectories and a trajectory-level reward are introduced, the methodological novelty beyond standard SFT + RL remains insufficiently justified.
- While the paper introduces RS-Ref and evaluates 20 representative VLMs, some benchmark construction details remain unclear. Moreover, the analysis of frontier models primarily focuses on quantitative comparisons, with limited discussion of why different models exhibit substantial performance differences and what these results reveal about their reasoning limitations.
Questions for authors
- The proposed framework appears to primarily combine SFT on grounded reasoning trajectories with GRPO-based reinforcement learning. Could the authors clarify what distinguishes GeoREC from a naive SFT + GRPO pipeline? In particular, it would be helpful to compare against a standard CoT-based SFT + GRPO baseline under comparable settings to better isolate the benefits of the proposed grounded reasoning formulation.
- Could the authors elaborate on the benchmark construction process, particularly the image filtering criteria, hard distractor selection, and distribution of reasoning constraints? Furthermore, Table 5 reveals substantial performance differences among frontier models. For example, InternVL3.5-8B significantly outperforms several larger general-purpose VLMs. Could the authors provide a more detailed analysis of these observations, including representative failure cases and the underlying reasoning or grounding limitations revealed by RS-Ref?