acceptodds
Under review as a conference paper at ICLR 2027

Q-VG: Towards Quantitative Reasoning for Weakly Supervised Visual Grounding

Abstract

Weakly supervised visual grounding (WSVG) aims to locate objects referred by natural language queries, without region-level human annotations. However, existing methods struggle to ensure the correct number of targets, a common requirement in compositional reasoning. In this paper, we propose an end-to-end network, Q-VG, for quantitative reasoning and grounding under weak supervision. Specifically, Q-VG incorporates a novel Referring-Object Counting (RECO) module to estimate the number of objects referred to by the input expression. It enhances quantitative awareness prior to grounding and determines the number of bounding boxes to regress. Besides, we also propose a novel weakly supervised objective, which encourages multiple positive objects to align with the given referring phrase. To further evaluate the ability of quantitative reasoning, we introduce QGround-Bench, a quantitative grounding benchmark. It comprises three subsets, Exact-QG, Gen-QG and Comp-QG, which progressively evaluate exact quantitative constraints, generalized quantitative expressions, and quantitative compositional reasoning. Experiments show that Q-VG achieves competitive performance on both the proposed and classical benchmarks. The source code and dataset will be available on GitHub after double-blind phase.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Weakly Supervised Visual Grounding, Quantitative Reasoning, Multi-Object Grounding, Vision-Language Grounding, Compositional Grounding

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_nKTFOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

Q-VG introduces count prediction into weakly supervised visual grounding. It learns phrase-conditioned object slots, predicts how many targets the expression refers to, and uses that estimate to select candidate boxes. Training combines contrastive alignment, reconstruction, and count constraints derived from language. The paper also introduces QGround-Bench to test exact, generalized, and compositional quantity expressions, and evaluates the method on existing grounding datasets.

Critical strengths
  1. Count constraints address a real limitation of weakly supervised grounding, and the loss comparisons support their contribution to the gains on existing datasets.

  2. Separating exact, vague, and composed quantity expressions makes the benchmark more useful than a single counting test.

Critical weaknesses
  1. In the compositional protocol, a full-query box is dropped from Set-F1 if it cannot be matched to predictions from separate entity queries. Count-Acc still counts that box but cannot distinguish a correct target set from an equally sized wrong one, so the reported scores do not reliably measure the complete answer to the composed query.
Questions for authors
  1. Please rescore the saved full-query outputs with one-to-one matching to all required targets, counting every unmatched prediction as a false positive. Does the compositional advantage remain without using separate entity-query predictions to filter or attribute boxes?

  2. When the predicted count is too small, true target anchors outside the selected set become contrastive negatives. What prevents this from reinforcing under-counting? A training trace relating count errors to anchor selection would be useful.

  3. Among correctly counted compositional queries, how many failures select the wrong instances rather than merely localizing the right instances poorly? Please separate relation-selection errors from box-overlap errors using existing outputs.

Reviewer_g36ROpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Q-VG addresses variable output cardinality in weakly supervised visual grounding. Its RECO module uses phrase-conditioned slot attention over DINOv3 features to predict a referent count, which determines how many YOLO anchors are selected. Training combines a multi-positive contrastive objective, feature reconstruction, and linguistic interval-count supervision. The paper also introduces QGround-Bench, constructed from Flickr30K test images with exact, generalized, and compositional quantitative expressions. Evaluation includes this benchmark, Flickr30K Entities, and gRefCOCO, with additional counting-head, decoding-count, and relation-intervention analyses.

Critical strengths
  1. Output cardinality is a meaningful limitation of conventional weakly supervised grounding. Integrating a count estimate into anchor selection is a practical approach to multi-target and no-target prediction, and the proposed benchmark attempts to distinguish explicit cardinality following from more general quantitative expressions.

  2. The appendix provides useful evidence beyond a standard component ablation. On Gen-Vague, Table 9 reports 58.6% Target-Count-Match for full RECO versus 48.7% for Text-only, supporting a real contribution from visual information. Table 10 also shows a benefit from count supervision under identical ground-truth-count decoding. These controls substantiate aspects of the proposed mechanism and should be recognized when interpreting the counting gains.

Critical weaknesses
  1. The evaluation does not yet establish complete-set grounding reliably. For Comp-QG, p.15, lines 785-793, excludes full-query boxes that cannot be matched to separate group-query predictions from group-wise Set-F1 scoring. Consequently, surplus false positives can disappear from the denominator. A full prediction containing all correct boxes plus an irrelevant unassigned box can still obtain group-wise Set-F1=1. Count-Acc can detect the cardinality error, but does not make this Set-F1 a complete-set metric. Separately, the Flickr30K definitions on p.14, lines 732-736, do not explain how multiple predicted and ground-truth boxes are reduced to phrase-level accuracy. Because Q-VG explicitly changes output cardinality, these evaluation choices directly affect the interpretation and comparability of its main results.

  2. The incremental evidence for visual quantitative reasoning remains difficult to separate from linguistic cues and the prediction pipeline. Exact-QG writes the desired cardinality into each query, so its 98.9% Count-Acc alone does not establish visual counting. On Gen-QG, Text-only, Global V-L, and full RECO achieve 67.3%, 68.2%, and 69.5%, respectively. The Gen-Vague and controlled-decoding results are encouraging, but uncertainty estimates and a simple count-replacement baseline within the same candidate/feature pipeline are missing. Moreover, the principal weakly supervised comparisons convert heatmaps into box sets, while Q-VG uses additional DINOv3 features and explicit detector anchors. The authors acknowledge the interface limitation and test NMS sensitivity, but these results still do not fully isolate the quantitative module's independent contribution.

Questions for authors
  1. Evaluation clarification: What is the exact Flickr30K scoring procedure for multiple predicted boxes, multiple ground-truth boxes, duplicates, and empty predictions? Please provide the evaluation code and clarify whether all methods use the same phrase-level reduction.

  2. Validation of the existing benchmark claim: Can you report union-level F1 while retaining every full-query prediction, including unassigned boxes? Does this change the Comp-QG ranking or the claimed advantage? This would materially affect my assessment of the benchmark evidence.

  3. Mechanism validation: Within the same candidate-box, feature, and decoding pipeline, can you compare RECO with a rule-based or simple count predictor and report repeated-run uncertainty on Gen-Vague alongside the existing Global V-L control? This would clarify the incremental value of visually grounded slot counting without requiring a new task or dataset.

2 comments

Thank you for the experiments. The response address my main experimental concerns. Though the contribution remains limited, I will increase my rating to 3.

AuthorsOpenReview ↗

Thank you.

Thank you for reading our response and for your positive assessment.

Reviewer_RZ5jOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Q-VG, a weakly supervised visual grounding method that predicts the number of referred objects and uses it to select multiple candidate boxes. It also introduces QGround-Bench for evaluating exact, generalized, and compositional quantitative grounding. Q-VG substantially outperforms prior weakly supervised methods on the proposed benchmark.

Critical strengths
  1. Target cardinality is an important but underexplored aspect of weakly supervised grounding. The paper addresses a concrete failure mode of methods that return a fixed or uncontrolled number of regions.
  2. The author provides detailed experiments and ablation studies.
  3. The separation between Set-F1 and Count-Acc is useful: the reported results clearly show that predicting the right number is not equivalent to localizing the right objects.
Critical weaknesses
  1. Much of the count performance may come from lexical cues rather than visually grounded counting. Exact-QG Count-Acc reaches 98.9%, while Set-F1 is only 54.6%. On Gen-QG, the text-only counter is also close to the full model. This suggests that dataset construction introduces strong expression-to-count priors.
  2. The “weakly supervised” characterization needs qualification. Although target-dataset boxes are not used for training, Q-VG uses explicit count constraints, no-target labels, a box-supervised pretrained YOLO detector, and an additional DINO encoder.
  3. Comparisons against heatmap-based WSVG methods are not architecture-controlled. Q-VG has a stronger proposal and counting machinery, while baseline outputs must be converted into sets using thresholding, connected components, and NMS.
Questions for authors
  1. The high Exact-QG Count-Acc and strong text-only results suggest that the model may exploit lexical count cues. Can the authors evaluate on counterfactual image-query pairs or splits with unseen quantifiers, cardinalities, and quantifier-category combinations?
  2. Q-VG uses both YOLO and DINOv3, while several baselines use weaker heatmap-based architectures and require post-processing. Can the authors provide an architecture-controlled baseline using the same visual encoders, anchor candidates, and count supervision but without RECO?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.