- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper presents QGVR, a two-stage framework designed to improve visual sampling efficiency in MLLMs under a fixed visual token budget. The first stage, VJP-WARP, learns question-conditioned image warps through answer-guided candidate verification and density distillation. The second stage, WARPRANK, selects between the original image and multiple warped candidates based on learned preferences derived from offline answer scores, avoiding unnecessary warping when the original image is more suitable. The framework operates without modifying the underlying MLLM and aims to preserve global image context while emphasizing question-relevant regions. Experiments on five VQA benchmarks with InternVL3.5-8B and LLaVA-1.5-7B show improvements over the evaluated baselines. The authors also provide ablation studies, attention analyses, and efficiency measurements to examine the effectiveness and overhead of the proposed design.
Critical strengths
(1) The paper considers the problem of improving visual sampling efficiency under a fixed token budget. By adjusting the sampling density while retaining the full image context, the proposed method provides an alternative approach to fine-grained visual understanding without increasing the number of visual tokens.
(2) QGVR uses answer likelihoods from the frozen MLLM to evaluate candidate warps, providing task-related feedback for learning visual sampling patterns instead of relying solely on attention maps.
(3) By retaining the original image as a candidate and introducing a learned ranking and fallback mechanism, the method accounts for cases where image warping may not improve performance.
Critical weaknesses
(1) In Stage I, candidate warps are generated through local perturbations of the current proposal, and the verified candidates are then used as distillation targets. However, if the initial proposal leads to a poor sampling pattern, this procedure may reinforce suboptimal solutions rather than identify better alternatives. In addition, the update is skipped when no valid edit is found and the base warp is not significantly harmful. I would appreciate clarification on how the method handles these cases and whether the local search strategy is sufficient to avoid poor solutions.
(2) The selection gate threshold is set to for InternVL and for LLaVA, while the minimum ranking margin is zero for both models. Under these settings, the final decision appears to depend primarily on the ranking scores. This raises the question of whether the additional benefit gate is necessary and how much it contributes to the final performance.
(3) The experimental evaluation leaves some questions about generalization. QGVR requires training, but the comparisons include only one training-based baseline. In addition, the training datasets largely overlap with the evaluation benchmarks, with GQA being the only benchmark excluded from training. This makes it difficult to determine how well the method generalizes to different tasks. The evaluation also does not cover other widely used backbones, such as Qwen2. I would appreciate further clarification or additional experimental evidence on these points, where feasible within the rebuttal period, to better assess the generalizability and robustness of the method.
Questions for authors
Please refer to the Critical Weaknesses section.
4 comments
Response to DdGX (2/3): Benefit-Gate Ablations
(2) Necessity and contribution of the benefit gate.
The benefit gate provides an additional acceptance condition beyond native-relative ranking. Its contribution depends on the backbone configuration, as demonstrated by the following controlled comparisons.
For LLaVA, we conducted a paired replay on the same ten-candidate answer cache, holding the proposal, candidates, ranking scores, and native-relative ranking condition fixed. The only difference was whether the benefit threshold, approximately 0.0875, was applied.
| LLaVA configuration | Five-task mean score | Warp-selection rate | Harmful-switch rate |
|---|---|---|---|
| Ranking + benefit gate | 40.47 | 29.45% | 1.46% |
| Ranking only | 38.93 | 99.93% | 4.86% |
The gate improves the mean score by 1.54 points [95% paired CI: 1.32, 1.77] and reduces the harmful-switch rate by 3.40 percentage points. Harmful-switch rate denotes the fraction of all questions for which the selected warp scores below Native. In this configuration, a non-native candidate almost always outranks Native, so a zero ranking margin alone provides little rejection. The benefit gate supplies an additional check that substantially reduces harmful switches.
For InternVL, we agree with the reviewer that the threshold of makes the additional benefit check effectively inactive on the evaluated examples. The common interface is retained for implementation consistency. We verified its contribution through fresh paired inference on 2,500 questions, with 500 questions from each of the five datasets. Both arms use the same proposal, candidate construction, selector weights, and native-relative ranking condition.
| InternVL configuration | Five-task mean score | Warp-selection rate | Harmful-switch rate |
|---|---|---|---|
| Ranking + benefit gate | 81.78 | 18.16% | 0.92% |
| Ranking only | 81.78 | 18.16% | 0.92% |
Removing the gate changes neither the selected candidate nor the generated answer for any of the 2,500 questions. Native-relative ranking therefore determines selection in this evaluated configuration, and the additional gate can be removed without changing these results.
Response to DdGX (3/3): Additional Benchmarks and Backbone Coverage
(3) Additional benchmark evaluation and backbone coverage.
To broaden the evaluation beyond the original five benchmarks, we extended the InternVL-based comparison to MMVP, OCRBench, and AI2D, using existing checkpoints without additional training or benchmark-specific tuning for these evaluations. We also evaluated AttWarp, ViCrop, and the trained SD-RPN baseline.
The additional evaluation covers 4,388 questions: 300 MMVP questions forming 150 pairs, all 1,000 OCRBench questions, and all 3,088 AI2D test questions. The updated results are:
| Method | MMVP pair accuracy (%) | OCRBench accuracy (%) | AI2D accuracy (%) |
|---|---|---|---|
| Native | 56.67 | 82.10 | 81.99 |
| AttWarp | 46.67 | 82.50 | 81.61 |
| ViCrop | 57.33 | 81.20 | 81.77 |
| SD-RPN | 55.33 | 82.90 | 82.22 |
| QGVR | 58.00 | 83.60 | 82.67 |
QGVR achieves the highest numerical score among the evaluated methods on all three additional benchmarks. Relative to Native, the improvements are 1.33 percentage points on MMVP, 1.50 on OCRBench, and 0.68 on AI2D. Relative to the strongest evaluated baseline on each benchmark, the improvements are 0.67, 0.70, and 0.45 percentage points, respectively. These results extend the empirical evidence across paired visual discrimination, text recognition and understanding, and diagram question answering.
Regarding backbone coverage, our evaluation already spans two distinct language-backbone families and visual encoders:
| Evaluated MLLM | Language backbone | Visual encoder |
|---|---|---|
| InternVL3.5-8B | Qwen3 | InternViT |
| LLaVA-1.5-7B | Vicuna-7B-v1.5, from the LLaMA family | CLIP |
The existing results therefore provide evidence that QGVR’s benefits extend across both Qwen-based and LLaMA-based MLLM configurations, rather than being restricted to a single language-backbone family. We will make these architectural differences explicit in the revision.
These results suggest that the benefit gate is effectively redundant in the evaluated InternVL configuration, as removing it does not alter candidate selection or generation for any of the 2,500 questions. This raises concerns about the necessity of this component.
Response to DdGX (1/3): Proposal Verification and Skipped Updates
To accommodate the per-comment length limit, we organize our response into three parts, corresponding to your three concerns. We thank the reviewer for recognizing the motivation of visual reallocation, answer-based verification, and native-image fallback. We address the three concerns below with a training-trace audit, controlled gate ablations, and additional benchmark evaluations.
(1) Handling suboptimal proposals and skipped updates.
Stage I does not automatically distill its current proposal. Instead, answer-based verification determines whether an edited candidate provides a suitable training target. An edit must improve answer NLL over the base warp by at least 0.01 while remaining within a 0.01 NLL tolerance of the native image. If no edit qualifies and the base warp is worse than Native by more than 0.05 NLL, training uses a uniform-density target corresponding to identity sampling.
We audited all 3,200 example occurrences in the completed 200-step VJP control run, including 50 warmup steps:
The base warp exceeded the harmfulness threshold in 810 occurrences. Among these, 389 produced an admissible edit and 421 received identity-correction targets; none entered the skip branch. Thus, harmful base proposals were explicitly addressed through verified alternatives or corrective targets, rather than automatically reinforced.
For skipped examples, the available candidates do not justify a supervised update. Assigning zero weight avoids introducing an unsupported target, while the shared proposal network continues learning from other examples. Skipping an example therefore does not freeze its future prediction.
As complementary evidence, our matched VJP–Random control holds architecture, initialization, data order, optimization, and verification budget fixed. VJP increases Oracle@10 from 85.79 to 86.43, with a paired difference of +0.64 points [95% CI: 0.14, 1.15]. This supports more effective candidate discovery within the tested budget.