- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
This paper studies fine-grained visual question answering, where an answer is generated from an image-question pair. QRouter first uses Qwen2.5-VL and SAM2 to perform question-conditioned grounding for each pair, then reorganizes approximately 729 dense visual tokens into 32 region tokens, 128 routed context tokens, and 1 background token, for a total of 161 visual tokens, before passing them to a Mamba-based VLM. The paper reports accuracy improvements on multiple VQA benchmarks.
Critical strengths
- The problem is important, and organizing visual evidence conditioned on the question is intuitive.
- Accuracy improves on multiple benchmarks, and the question-shuffled control also indicates that grounding uses question semantics.
Critical weaknesses
The latency numbers in Table 4 are not measured for the two systems under the same end-to-end setup. For every image-question pair, QRouter must first run Qwen2.5-VL and SAM2 to produce question-conditioned grounding; however, the main comparison in Table 4 reports only the VQA latency after grounding, while Cobra-3.5B has no corresponding extra grounding stage. Appendix Table 12 reports 0.4386 s/sample for QRouter-B's grounding stage and 0.5248 s/sample for its complete online latency, whereas the reported latency for Cobra-3.5B is 0.0857 s/sample. Thus, in the complete online inference setting, QRouter-B is approximately 6.1 times slower, and grounding accounts for about 84% of its total latency. Moreover, compressing the visual prefix from 729 tokens to 161 tokens does not yield a clear practical speedup: even excluding grounding, QRouter-B takes 0.0862 s/sample, essentially the same as Cobra's 0.0857 s/sample. The current results therefore support an accuracy-compute trade-off in which substantial extra computation is exchanged for higher accuracy, rather than the paper's claim of an efficiency advantage. Since this issue directly affects the paper's central system claim, I do not think it can be resolved by a simple clarification.
Questions for authors
Please clarify the complete measurement scope of the latencies in Table 4: does QRouter-B's 0.0862 s/sample fully exclude the Qwen2.5-VL and SAM2 grounding stages, while Cobra-3.5B's 0.0857 s/sample covers its complete online inference process? If the measurement protocols are indeed different, do the authors agree that the efficiency claim should be limited to the post-grounding VQA stage rather than interpreted as an end-to-end system efficiency advantage?
As the discussion deadline approaches, we would appreciate your feedback on our rebuttal and any remaining questions.