- Rating
- 2
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes Native Complete-Set Grounding to address a key challenge in generalized referring expression comprehension (GREC): models may correctly predict the number of referred objects yet fail to identify the complete set of those objects. The proposed method achieves impressive experimental performance.
Critical strengths
- The proposed method demonstrates strong performance on the evaluated benchmarks.
- The paper provides thorough analyses that support the effectiveness of the proposed approach.
Critical weaknesses
-
The motivation for converting unordered set prediction into ordered prediction requires further clarification. Existing GREC methods typically use ground-truth bounding boxes to supervise training, enabling models to learn both the number and locations of the referred objects. Introducing an ordering constraint appears to impose an additional requirement: the model must identify the correct objects while also predicting them in a prescribed order. Why is this reformulation necessary, and how does it improve complete-set grounding? Controlled comparisons with unordered set prediction would help substantiate this design choice.
-
The framework in Figure 2 does not clearly convey the novelty of the proposed method or its core technical details. The figure should more explicitly illustrate the key components, their interactions, and the aspects that distinguish the proposed approach from existing methods.
-
Table 1 show the results on regerring expression segmentation, but the topic of this paper is referring expression comprehension. Although these two task share some similarity, the inner logic is different
Questions for authors
see above mentioned.
2 comments
Dear Reviewers,
This is a friendly reminder that the rebuttal discussion period is approaching its deadline.
Please take some time to review the authors' rebuttal and actively participate in the discussion. If you have any remaining questions or concerns, please raise them and share your feedback with the authors and other reviewers.
As the discussion period is about to close, we kindly ask you to complete your discussion and provide your updated assessment before the deadline.
Thank you for your time and effort.
Best regards,
Area Chair
Response to Reviewer Ph6V
We thank the reviewer for the thoughtful comments and constructive feedback.
W1. Why serialize an unordered target set, and how does it compare with unordered set prediction?
The target is an unordered set of bounding boxes, while evaluation uses one-to-one IoU matching and is invariant to the order of predicted boxes. Serialization serves as a training representation for autoregressive generation, while prediction correctness remains independent of box order. It enables complete-set grounding through the MLLM's native localization pathway, allowing each generated box to be conditioned on the image, the referring expression, and previously generated boxes.
We compared this formulation with a permutation-invariant, Hungarian-matched DETR-style set decoder under the same 12k-train/3k-development protocol. Even with extended training (6 epochs), the unordered set decoder reaches 85.79% count accuracy but only 56.59% complete-set Pr (vs. 50.24% at 3 epochs), whereas Native achieves 77.22% Pr and improves directional switch accuracy on count-preserving query pairs from 4.02% to 65.68%.
We further tested how training order and the explicit count field affect performance using the same 12k/3k controlled protocol:
Fixed-order supervision improves complete-set Pr in both formats, while free-form boxes outperform explicit Count+Boxes under either ordering. These results support our autoregressive grounding formulation and show that training order affects complete-set recovery, while evaluation remains order-invariant.
W2. What is the technical distinction that Figure 2 should convey?
The key distinction is how target selection and localization are performed. The separate set decoder uses learned object queries, cross-attention to multimodal LM features, Hungarian matching for box supervision, and query ranking with a predicted target count. Native grounding instead supervises the complete variable-cardinality set through Qwen's autoregressive coordinate-generation pathway; the generated boxes are then parsed into an unordered set.
Figure 2 depicts the submitted Count+Boxes interface: its count is generated in the same response as the coordinates, rather than by a separate count head. We will clarify these two prediction pathways and the set-parsing step in the figure and caption.The figure's link is https://anonymous.4open.science/r/figure-materials-5B33/
Figure 2: Overview of Native Complete-Set Grounding and the separate set-prediction baseline. (A) Both approaches share the same Qwen2.5-VL backbone, with frozen visual components and LM-side LoRA adaptation. (B) Native performs target selection and localization within a single pretrained autoregressive pathway, generating target count and box coordinates in one response. Training uses valid orderings of unordered target sets with response-token causal LM supervision, while inference deterministically parses the generated response into an unordered set. (C) The separate baseline instead predicts cardinality with a count head and localizes targets through a newly trained query-based decoder over MLLM representations, using count classification and Hungarian-matched box/objectness supervision.
W3. Does Table 1 evaluate referring expression segmentation or comprehension?
Table 1 evaluates bounding-box Generalized Referring Expression Comprehension (GREC), not segmentation. Predictions and ground truth are bounding boxes; complete-set Pr requires the correct target count and one-to-one box matching at IoU ≥ 0.5, while N-acc measures correct empty-set prediction. gRefCOCO/GREC · GRES