- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
ContourVLA reformulates generalized referring expression segmentation (GRES) as a closed-loop perception–action process, where editable contours act as explicit geometric states and are iteratively refined through predicted actions. The framework introduces Evolution-Aware Semantic Scheduling (EASS) to adaptively sample boundary evidence and route multilevel multimodal features according to the current contour state, together with a Geometric Action Decoder for contour updates. It further proposes DECT-GRPO, which uses dustbin-augmented entropic transport to provide instance-level credit assignment for jointly optimizing grounding and contour actions. Experiments on gRefCOCO, RefCOCO, RefCOCO+, and RefCOCOg show strong improvements over prior methods.
Critical strengths
-
Novel closed-loop formulation for GRES. The paper reformulates GRES from conventional single-pass mask prediction into a closed-loop perception–action process, where editable contours serve as explicit geometric states to guide iterative multimodal perception and boundary refinement. This provides a conceptually meaningful alternative to static vision-language segmentation pipelines.
-
Well-designed adaptive multimodal routing and instance-level policy optimization. The proposed EASS dynamically routes multilevel multimodal representations according to the evolving contour state, allowing the model to select semantic information that is more appropriate for different refinement stages. In addition, DECT-GRPO introduces dustbin-augmented entropic transport to establish soft prediction–target correspondences and provide instance-level credit assignment for both grounding and contour actions. These two designs make the perception and optimization processes more adaptive to instance-specific states and prediction quality.
Critical weaknesses
-
Training and inference efficiency. ContourVLA relies on a relatively heavy training pipeline, including large-scale grounding pre-training, multi-dataset supervised fine-tuning, DECT-GRPO post-training, and iterative contour refinement during inference. However, the paper does not provide a comprehensive efficiency comparison with representative GRES methods in terms of end-to-end inference latency, GPU memory consumption, or total training cost. Therefore, it remains unclear whether the reported performance gains justify the additional computational and training overhead.
-
Attribution of the performance gains. The main results combine several factors, including large-scale grounding pre-training, joint training on multiple RES/GRES datasets, the proposed contour-based architecture, and RL post-training. Since the methods in the main comparison are evaluated under their respective training configurations, it is difficult to fully disentangle how much of the improvement comes from the proposed closed-loop formulation itself versus the stronger training recipe and additional supervision. Although the controlled HiMTok experiment partially addresses this concern, it still evaluates the complete ContourVLA pipeline rather than isolating these factors under a strictly matched training and computation budget.
Questions for authors
-
Clarification on computational efficiency. Could the authors provide a more comprehensive efficiency comparison with representative GRES methods, including end-to-end inference latency, peak GPU memory consumption, and total training cost? The current framework involves large-scale grounding pre-training, multi-stage optimization, RL post-training, and iterative contour refinement, so understanding the computational overhead relative to the reported performance gains would materially affect the assessment of its practical value.
-
Clarification on the contribution of DECT-GRPO in single-target RES. A key motivation of DECT-GRPO is instance-level credit assignment among multiple predicted referents. However, conventional RefCOCO, RefCOCO+, and RefCOCOg primarily involve a single referred instance. Could the authors clarify how much the instance-level credit mechanism contributes in this single-target setting?
4 comments
Response (Part II)
3. Contribution of DECT-GRPO to Single-Target RES [Q2]
DECT-GRPO contributes to single-target RES, although each expression in this task corresponds to only one ground-truth target. In this task, the VLIM may predict multiple instances, and DECT-GRPO can still assign different instance-level credits based on the matching quality between each prediction and the target. To quantify the benefits of this mechanism, we follow the ablation setting in Table 4 of the revised manuscript and compare full DECT-GRPO with classical Shared-advantage GRPO across five test splits of three RES datasets. The results are presented in Table R11.
Table R11: Impact of Instance-Level Credit Modulation on Single-Target RES Performance.
| Method | RefCOCO testA mIoU | RefCOCO testA cIoU | RefCOCO testB mIoU | RefCOCO testB cIoU | RefCOCO+ testA mIoU | RefCOCO+ testA cIoU | RefCOCO+ testB mIoU | RefCOCO+ testB cIoU | RefCOCOg test mIoU | RefCOCOg test cIoU |
|---|---|---|---|---|---|---|---|---|---|---|
| Without instance-level credit modulation (Shared-advantage GRPO) | 82.3 | 81.8 | 80.0 | 78.7 | 79.5 | 79.2 | 72.6 | 68.3 | 75.3 | 76.3 |
| With instance-level credit modulation (Full DECT-GRPO) | 83.7 | 82.9 | 81.1 | 79.2 | 80.7 | 80.7 | 74.2 | 69.3 | 76.7 | 77.3 |
Compared with the ablated version, DECT-GRPO improves both mIoU and cIoU across all five test splits. These results demonstrate that distinguishing the quality of different predictions for the same expression can also improve single-target RES performance.
Gentle reminder before the discussion deadline
Dear Reviewer 4Qi7,
Thank you again for your time and thoughtful efforts in reviewing our paper. In response to your valuable comments, we have analyzed the attribution of the performance gains, demonstrated the computational efficiency of ContourVLA, and evaluated the contribution of DECT-GRPO in single-target RES.
As the discussion period is coming to a close, we would like to kindly ask whether you have any remaining questions or concerns. If so, please feel free to let us know.
Many thanks,
Authors
Gentle reminder before the discussion deadline
Dear Reviewer 4Qi7,
Thank you again for your time and thoughtful feedback. As the discussion deadline approaches, we would appreciate your feedback on our rebuttal and welcome any remaining questions. Thank you for your time and consideration.
Best regards,
The Authors
Response (Part I)
We thank the reviewer for recognizing the novelty of ContourVLA's closed-loop contour formulation and appreciating the designs of adaptive multimodal routing and instance-level policy optimization. We are also grateful for your comments on the attribution of the performance gains, computational efficiency, and contribution of DECT-GRPO in single-target RES.
1. Attribution of the Performance Gains [W2]
More rigorous ablation experiments validate the contributions of the proposed architecture, the grounding pretraining, joint SFT training on multiple RES/GRES datasets, and RL post-training. (a) Architecture: As shown in Rows 1 and 3 in Table R9, replacing the native HiMTok semantic segmentation architecture with the closed-loop contour architecture improves most metrics, even if large-scale grounding pretraining (i.e., directly using the HiMTok pretrained checkpoint) and RL post-training are disabled. Rows 2 and 4 further show that the gIoU advantages of contour evolution further increase across all three splits after applying GRPO. (b) Pretraining: Row 4 in Table R9 shows that ContourVLA still achieves good performance even when using the HiMTok pretrained checkpoint without large-scale grounding pretraining, while the comparison with the full ContourVLA (last row of Table 1 of the manuscript) demonstrates that the latter achieves higher performance. (c) RL post-training: The comparisons between Rows 3 and 4 of Table R9 and Rows 1 and 5 in Table 4 of the main paper further validate the contribution of RL post-training. (d) Joint SFT training: This stage supervises GAD and EASS to learn contour evolution. Directly applying GRPO without this stage would make GAD and EASS unable to learn contour evolution from reward feedback alone, resulting in poor segmentation performance.
Table R9: Contributions of contour-evolution architecture and RL optimization without Stage-I grounding pretraining.
2. Computational Efficiency [W1, Q1]
Efficiency comparison experiments with representative GRES methods demonstrate that ContourVLA achieves high segmentation accuracy with lower inference overhead, outperforming the three representative GRES baselines in both end-to-end inference latency and peak GPU memory consumption. The detailed results are presented in Table R10.
Table R10: Segmentation Performance and End-to-End Inference Overhead on gRefCOCO.
(a) Inference Efficiency. Under the same GPU, numerical precision, and test sample conditions, ContourVLA achieves 14.8% lower end-to-end latency than the fastest baseline (RAS-13B) and 54.1% lower peak GPU memory consumption than the baseline with the lowest memory usage (HiMTok-8B). Meanwhile, it achieves the highest gIoU across all three gRefCOCO splits.
(b) Training Cost and Performance Gains. ContourVLA-3B completes three stage training on 16 NVIDIA H20 GPUs, consuming a total of 1,580 GPU hours. Grounding pretraining accounts for 76% of the total cost (1200 GPU hours). SFT consumes 160 GPU hours. DECT-GRPO post-training consumes 220 GPU hours, improving gIoU and cIoU on gRefCOCO val by respectively 5.9 and 4.9 percentage points over SFT. Note that the contour evolution architecture can directly start downstream SFT from an existing HiMTok-8B checkpoint without requiring the 1,200 GPU hour grounding pretraining. This configuration achieves higher gIoU on gRefCOCO val than the native HiMTok segmentation architecture (see the "Attribution of the Performance Gains" section). For reference, retraining HiMTok-8B on the same type of H20 GPUs consumes 2,290 GPU hours, requiring 44.9% more than our ContourVLA-3B.
Please refer to the following response for the continuation of our detailed discussion.