- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper points to a "fine-grained semantic grounding gap" in multimodal LLMs: a model can describe a part's attributes verbally, yet fails to tie them to the image evidence or to the final decision or action. To address this, the authors propose SVP-R, which casts evidence as Semantic Visual Primitives, binding together a part's location, its attribute, and the action affordance it supports. These primitives are organized into an evidence graph and validated through "graph binding" at the node, edge, and graph levels. The idea is instantiated twice: as Reinforced Evidence Binding (REB, a correctness-gated GRPO reward) for fine-grained recognition, and as Affordance Evidence Binding (AEB, a training loss) for fine-grained manipulation. The evaluation spans CUB/Dogs/Cars (both within-domain and held-out), a MetaFine localization split, five simulated manipulation tasks (MetaFine plus RoboTwin 2.0), and three real-robot tasks, together with intervention and ablation studies.
Critical strengths
- A shared interface across recognition and manipulation. Binding "where/what/how" into a single primitive and applying graph binding to both tasks is a coherent conceptual contribution. Showing this interface working in perception and control alike supports the paper's central motivation.
- Intervention-based analysis of grounding. The degradation observed under evidence removal, swapping, shuffling, and jitter, combined with the interface-layer analysis, provides evidence that predictions genuinely depend on the intermediate primitives rather than merely being accompanied by them.
Critical weaknesses
- The main baseline gains are not isolated from the training recipe. Table 6 supports a benefit from the binding terms within SVP-R, but it does not resolve the lack of supervised warm-up for GRIT and Visual-RFT in Table 1. A common supervised starting point is needed to establish how much of the larger comparative gain is specific to binding.
- Dependence on evidence and correctness of evidence are not fully separated. Corrupting a primitive can show that the model uses it, but does not by itself establish that its location and attribute are both correct. The automatically generated Dogs/Cars targets make independent semantic validation especially important; the CUB labels offer a direct setting in which to test this part of the claim.
Questions for authors
-
How much of the recognition gain comes from binding rather than supervised warm-up? Table 6 already shows benefits from the REB terms within the 3B recipe. To interpret the much larger Cars gains in Table 1, I would like a comparison with GRIT and Visual-RFT starting from the same supervised checkpoint and using matched RL data and update budgets.
-
Does graph binding improve the correctness of the evidence, not just dependence on it? Does the grounding hit rate in Table 3 require both a correct part location and a correct attribute, or only localization? Using CUB's official annotations, I would like this joint measure for full REB and the ablated rewards in Table 6 alongside classification accuracy. The result would directly test the verified-evidence claim without relying on agreement with automatically generated Dogs/Cars labels.
8 comments
Thank you for your time and consideration. We would appreciate your further feedback on our rebuttal.
Dear Reviewer,
We would appreciate it if you could kindly review our rebuttal when you have a chance!
We would also like to note that Reviewer Mpon provided positive feedback following our rebuttal and maintained their positive score of 4. We welcome any further questions or concerns and would be happy to address them.
Thank you for your time and consideration!
A sincere request for your feedback before the discussion closes
Dear Reviewer THZE,
We sincerely apologize for writing to you again. As the discussion closes very soon, we would be truly grateful if you could spare a few minutes for our responses. We have completed both experiments exactly as you suggested.
-
Supervised warm-up (Q1). We re-trained GRIT and Visual-RFT from the same supervised checkpoint as SVP-R, with matched RL data and update budgets. SVP-R still leads on both datasets, 91.29 against 89.70 on CUB and 91.42 against 89.03 on Cars.
-
Correctness of the evidence (Q2). Using the official CUB annotations, the joint correctness of location and attribute rises from 47.3 under the node-only reward to 69.2 under the full reward, while accuracy rises from 80.71 to 84.87. The result is now in Tab. 8 and Sec. 4.3 of the revised paper.
If these results address your concerns, we would deeply appreciate it if you could kindly consider raising your score. If anything remains unclear, we will respond immediately. Thank you very much for your time.
Best regards, The Authors
Thank you for your constructive comments, and the revised version has been uploaded.
Thank you again for your constructive comments. We have uploaded the revised paper, with all changes marked in blue.
The joint correctness of the emitted evidence, together with the reward that contains only localization and semantic information, has been added to Tab. 8 and Sec. 4.3. As two tables have been added, Tab. 3 and Tab. 6 in our responses are now Tab. 5 and Tab. 8.
As the discussion period is ending soon, we would greatly appreciate it if you could kindly take a look at our responses, and we sincerely hope they have addressed your concerns. We are happy to provide any further clarification.