- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper studies the problem that if we ask the VLMs to output a more precise format of the visual grounding bounding boxes, i.e., polygons at different resolutions, how the VLMs will perform. To this end, the paper constructs a benchmark, named GroundBench, to answer the question. The benchmark consists of 1500 image–expression–referent triples, and the samples are from the RefCOCO benchmark. The paper indicates that even if we allow the models to output more points, the performance is not getting better. The authors claim that this study shows even though the VLMs can locate the objects with boxes quite accurately, this does not ensure reliable explicit geometry.
Critical strengths
The ability of visual grounding for VLMs is an essential and important task to study. It is good to explore other more fine-grained formats in additional to the bounding box formats for the visual grounding task.
Critical weaknesses
-
Concern about the major discovery and conclusion of the paper. For a general VLM that has never been trained to output the visual grounding format in a polygon with many points, it is expected that they cannot perform very well for it. As the authors have also realised, if they ask the VLM to output mask first and then convert the mask into polygon, the performance gets improved. Therefore, the major evaluation on the benchmark / protocol the authors proposed may not be a good reflection of the actual fine-grained spatial visual grounding capability of the VLMs.
-
The paper is hard to read. It is difficult to understand the paper given the expressions and descriptions of the current submission. It would be better if the authors can polish the writing and express the ideas more clearly.
Questions for authors
I would suggest the authors to address my concerns as in the weakness section during the rebuttal.