acceptodds
Under review as a conference paper at ICLR 2027

VeGo: Visual Evidence-Gated On-Policy Learning for Vision-Language Models

Abstract

Multimodal large language models still struggle with fine-grained visual understanding, particularly when answering questions that depend on small or inconspicuous image regions. Existing remedies either invoke cropping tools at inference, incurring considerable overhead from repeated tool calls and visual processing, or internalize region-focused perception through on-policy self-distillation, where a teacher observes a question-relevant crop while the student sees the full image. However, such distillation treats every generated token uniformly, although privileged visual evidence is informative for only a small subset of content-bearing tokens. Moreover, responses often continue unnecessarily after the answer has already been determined, resulting in redundant generation and inefficient inference. To address these limitations, we propose VeGo, which contrasts the teacher's next-token predictions under an evidence-rich crop and a degraded, evidence-blind view to estimate each token's reliance on fine-grained visual evidence. This enables distillation to prioritize evidence-sensitive tokens while preserving the overall loss magnitude. To reduce redundant generation, VeGo also identifies when the necessary visual evidence has been sufficiently encoded in the generated text and applies a length reward only to the subsequent region, leaving the evidence-writing prefix unaffected. Experiments across V, ZoomBench, Visual Probe, HR-Bench 4K and HR-Bench 8K show that VeGo achieves the highest average accuracy among open-source, agentic, and distillation baselines while producing the shortest responses.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Evidence-sensitive distillation, Decision-aware compression, Fine-grained visual perception

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_6rDzOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes VeGo, an extension of crop-based on-policy self-distillation. The teacher receives clean and degraded crops separately, and the divergence between its prediction distributions under the two views is used to weight teacher–student distillation. A second, answer-probe-based divergence identifies a decision cliff, after which the generated suffix is penalized to reduce redundant post-decision generation while maintaining accuracy. The method is trained on 6.5K examples using Qwen3.5-9B and evaluated on five high-resolution or fine-grained perception benchmarks. Compared with Vision-OPD, average accuracy improves from 75.77% to 77.35%, while average output length decreases from 116.74 to 86.39 tokens. Ablations compare weighting signals, normalization methods, suffix masks, and adaptive reference lengths.

Critical strengths
  1. The reported accuracy–length trade-off on a newer open-source model, Qwen3.5-9B, is attractive. Detailed component analyses also demonstrate the method's effectiveness. Across five benchmarks, the method improves accuracy and reduces generation length relative to Vision-OPD.
  2. The method has a sufficiently clear and important motivation. Diagnostic experiments in Figures 1 and 2 show that token-level dependence on visual evidence is highly concentrated and that residual evidence dependence drops sharply to near 0 once the answer is determined. These findings support the subsequent method design.
Critical weaknesses
  1. Fine-grained visual reasoning generally involves relatively short outputs. For example, the longest output reported in Table 2 does not exceed 400 tokens and may not incur excessive inference latency. Efficiency improvements may therefore not be among the more important issues in fine-grained visual reasoning. VeGo saves only a few dozen tokens, which may not translate into a significant inference speedup in practice.
  2. Many details need further attention, with multiple omissions and errors throughout the paper. For example:
    • The value of λ\lambda is inconsistent: Section 4.1 specifies λ=0.5\lambda = 0.5, while Appendix A states “default λ=1.0\lambda = 1.0.”
    • In Table 4, the Len direction for the HR-Bench 8K and Average columns is marked (↑\uparrow) but should be (↓\downarrow).
    • The question and answer options in Figure 8 do not match: “What black logo is visible on the yellow jersey? (A) left (B) right.”
    • The Table 3 caption contains the problematic wording “Bold values indicate the best and best results.”
    • Several leftover notes remain in the references.
    • The main text cites a nonexistent “Figure 2 (c).”
    • The sensitivity analysis cites the wrong figure.
    • Numerous extraneous characters and typos remain.
Questions for authors
  1. How exactly is the decision-cliff value defined? The paper gives only a rough description, without a specific formula or value.
  2. The introduction states that the method will “apply an accuracy-gated length objective.” Where is the accuracy gate reflected in the method?
  3. The Figure 3 caption explicitly states that the suffix after the cliff “gets no distillation gradient,” yet Eq. 4 sums over the entire sequence. How do the authors explain this inconsistency?
2 comments

We thank the reviewer for the constructive feedback.

W1: Resource savings from shorter outputs

In high-resolution perception, every output token repeatedly attends to a large visual context. Visual Probe and HR-Bench 4K/8K contain on average 13.8K–16.2K visual tokens per image. On HR-Bench 8K, one decoding step reads approximately 509 MB of KV cache, about 16× the volume for a 1K-token text prompt. Shortening post-decision generation therefore reduces repeated computation and KV-cache traffic.

Measurement. We count decoding FLOPs with Qwen3.5-9B on 50 randomly sampled questions per benchmark, reproducing the evaluation prompts and image preprocessing. The counted per-step cost is linear in context length: 15.87 GFLOP + 131,072 FLOP per context token. We apply this calibrated cost model to the saved output lengths of all questions and estimate KV-cache reads from the cache geometry. As in Table 2, outputs of ≥4096 tokens are excluded.

Decoding cost per question, Vision-OPD → VeGo. Avg. denotes the mean over these three benchmarks; reductions are computed from unrounded values.

MetricVisual ProbeHR-4KHR-8KAvg.
Output tokens104.3 → 55.3180.5 → 132.1142.8 → 103.3142.6 → 96.9
Decoding FLOPs (TFLOP)1.86 → 0.99 (−47.0%)3.17 → 2.32 (−27.0%)2.58 → 1.86 (−27.8%)2.54 → 1.72 (−32.2%)
KV-cache reads (GB)47.5 → 24.8 (−47.7%)72.1 → 51.5 (−28.5%)72.9 → 52.4 (−28.1%)64.1 → 42.9 (−33.1%)

VeGo reduces decoding FLOPs by 32.2% and KV-cache reads by 33.1%, saving approximately 21 GB of KV-cache traffic per question on these high-resolution benchmarks. This reduces bandwidth demand during decoding; shorter continuations also allow each request's cache to be released earlier, which is relevant to batched serving.

The efficiency gain accompanies higher five-benchmark accuracy (75.77% → 77.35%; Table 1). Short outputs are not inherent to these tasks: on the three benchmarks above, DeepEyes and SenseNova-MARS average 185 and 300 tokens, respectively, versus VeGo's 96.9 (Table 2). VeGo targets redundant post-decision text while retaining the evidence-writing prefix.

W2: Manuscript consistency

We apologize for the typographical and editorial errors and the confusion they caused. We will carefully proofread the entire manuscript and correct these issues in the revised version, including the captions, cross-references, and bibliography.

The actual coefficient is λ = 0.5; Appendix A's “default λ = 1.0” is a typo. The correct Len direction in Table 4 is ↓. Figure 8's question should read “Is the volleyball on the left or right side of the man with white cap?” Table 3's caption should read “Bold values indicate the best results.” The redundant-tail distribution is Figure 2(b), not Figure 2(c); the sensitivity analysis is Figure 6, not Figure 5. Reference notes and stray characters should be removed.

Q1: Definition of the decision cliff

Following Section 3.3, a fixed decision probe φ (e.g., The correct answer is: **) is appended to each generated prefix. Let P⁺ₜ and P⁻ₜ denote the EMA teacher's answer distributions given the question and this probed prefix, under the clean and degraded crops, respectively. Residual evidence dependence is:

Iₜ = JSD₀.₅(P⁺ₜ ‖ P⁻ₜ).

The decision cliff t_cliff is the position of steepest sustained descent of the Iₜ profile, after which Iₜ remains near zero. It is a trajectory-specific token index, not a universal divergence value. To preserve the complete evidence-writing sentence, compression starts after the sentence containing the cliff ends:

Eᵢ = {t : t > sent-end(t_cliff) in yᵢ}.

Q3: Distillation on the post-cliff suffix

Eq. 4 describes evidence-weighted distillation over the full response, ∑ₜ gₜ dₜ; it does not impose a hard suffix mask. Figure 3's “no distillation gradient” should instead read: “Evidence dependence weights distillation; the length objective acts only on the post-cliff suffix.” The two signals differ: gₜ concerns the next generated token, whereas Iₜ probes the answer decision. Thus, Iₜ ≈ 0 does not imply gₜ = 0. Suffix tokens can still receive weighted distillation gradients; tokens outside Eᵢ have no direct length-loss term.

Thank you to the authors for their detailed response. The authors' response has addressed my concerns regarding the clarity of the paper, but some concerns remain. In particular, the authors have not addressed Q2. Furthermore, although reducing output length may improve inference speed, it may also lead to performance degradation on more complex reasoning tasks, as these tasks typically require longer chains of thought. Therefore, I have decided to maintain my current rating.

Reviewer_nHWQOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes VeGo, a visual evidence-gated on-policy learning framework designed to improve both the effectiveness of fine-grained visual perception and the generation efficiency of vision-language models. Building upon Vision-OPD, the authors identify two limitations of conventional on-policy self-distillation: uniform token-level supervision fails to account for the uneven importance of visual evidence, and full-trajectory distillation may encourage unnecessarily long responses after sufficient visual evidence has already been incorporated. Experiments on five fine-grained visual understanding benchmarks demonstrate that VeGo improves average accuracy from 75.77% to 77.35% compared with Vision-OPD, while reducing the average number of generated tokens from 116.74 to 86.39. Additional ablation studies suggest that both evidence-sensitive reweighting and post-decision length compression contribute to the overall performance.

Critical strengths
  1. Important and practically relevant research problem.

The paper addresses two meaningful limitations of existing on-policy distillation methods for vision-language models: inefficient allocation of distillation supervision and redundant inference-time generation. Improving the effectiveness of privileged visual supervision while reducing inference overhead is an important direction for deploying multimodal models in real-world applications.

  1. Intuitive and well-motivated evidence-sensitive distillation.

The proposed token-level evidence dependence measure is conceptually appealing. By comparing teacher predictions under evidence-rich and evidence-degraded visual inputs, the method attempts to distinguish visual information sensitivity from general student–teacher disagreement. This provides a reasonable motivation for concentrating distillation supervision on tokens that are more sensitive to fine-grained visual information.

  1. Useful ablation studies and optimization analysis.

The paper evaluates several alternative token-weighting strategies, including student–teacher divergence selection and entropy-gap weighting. It also investigates the contributions of post-decision masking and adaptive reference lengths. The gradient similarity analysis provides an additional perspective on the interaction between distillation and length optimization objectives.

Critical weaknesses
  1. The central assumption behind the decision cliff is insufficiently validated.

One of the paper's key assumptions is that a near-zero residual evidence dependence indicates that sufficient visual evidence has already been incorporated into the generated prefix, making subsequent tokens redundant. However, agreement between the evidence-seeing and evidence-blind teachers does not necessarily imply that the correct visual evidence has been extracted or that the answer has been reliably determined. Both teachers may converge to the same incorrect prediction due to language priors, shared reasoning errors, or insufficient visual understanding. Moreover, the degraded visual input may still preserve enough information for the teacher to produce similar predictions. Therefore, a low divergence between the two teacher distributions is not sufficient to establish that subsequent reasoning is unnecessary.

  1. The experimental improvements over the most relevant baselines are relatively modest and lack statistical reliability analysis.

Although VeGo achieves the highest average accuracy, its improvement over Vision-OPD is approximately 1.58 percentage points across five benchmarks. On some individual benchmarks, the gains are relatively small, and VeGo does not consistently outperform all competing distillation methods. The paper does not report results across multiple random seeds, standard deviations, or confidence intervals. Consequently, it is difficult to determine whether the observed improvements are statistically robust.

  1. The gradient analysis does not fully substantiate the claimed reduction in optimization interference.

The authors argue that post-decision masking reduces interference between the distillation and length objectives, supported by an analysis of per-tensor gradient cosine similarities. However, the reported analysis emphasizes absolute cosine similarity. A larger absolute cosine similarity does not necessarily indicate stronger gradient conflict, since positively aligned gradients can be mutually beneficial. Gradient interference is more directly associated with negatively aligned gradient directions. Furthermore, the analysis is conducted at a single optimization step, making it difficult to determine whether the observed pattern persists throughout training.

Questions for authors
  1. Have the authors evaluated answer correctness when generation is terminated at the detected decision cliff? How often does the model produce the same correct answer without the post-cliff continuation?

  2. Can the authors provide multi-seed results and confidence intervals for the main comparisons, particularly against Vision-OPD and OPD-V?

  3. Can the authors report signed gradient cosine similarities and the proportion of conflicting gradients throughout training, rather than relying primarily on absolute cosine similarity at a single optimization step?

1 comment

We thank the reviewer for the constructive comments.

W1 / Q1: Is post-cliff reasoning really unnecessary?

We test this directly against ground-truth answers instead of teacher agreement. We sample 512 distinct training questions (no counting/spatial questions), generate a full response with Qwen3.5-9B, locate the decisive token, truncate the response at the end of its sentence, append "The correct answer is: **", and let the model answer immediately.

Full responseTruncated at cliff
Accuracy52.34%51.95%
Avg. output tokens145.880.1 (−45%)
  • Same final answer: 492/512 (96.1%).
  • Same correct answer: 261/268 (97.4%) of correct answers are preserved; only 7/512 flip from correct to wrong, while 5 flip from wrong to correct.
  • Accuracy change: −0.39 (paired bootstrap 95% CI [−1.76, +0.98]; McNemar pp=0.77).

The post-cliff continuation almost never changes the answer, whether right or wrong. This is exactly what the length objective relies on: it removes text that does not affect the decision. Since correctness is scored against ground truth, this conclusion does not depend on whether the two teachers agree.

W2 / Q2: Multiple seeds and confidence intervals

We trained three additional seeds (four in total, same configuration).

V*ZoomVPHR-4KHR-8KAvg
Vision-OPD93.7261.8954.6184.3884.2575.77
OPD-V91.6260.4755.7886.1283.6275.52
VeGo (mean±std)94.21±0.5364.85±0.6154.83±0.3085.83±0.6286.50±0.7577.25±0.17

Gains with 95% CI (one-sample t-test over seeds):

vs. Vision-OPDvs. OPD-V
Avg+1.48 [1.20, 1.76], p<0.001+1.73 [1.45, 2.01], p<0.001
Zoom+2.96 [1.99, 3.93], p=0.002+4.38 [3.41, 5.35], p<0.001
HR-8K+2.25 [1.05, 3.45], p=0.009+2.88 [1.68, 4.08], p=0.005
HR-4K+1.45 [0.46, 2.44], p=0.019−0.29 [−1.28, 0.70], p=0.42
V*+0.49 [−0.35, 1.33], p=0.16+2.59 [1.75, 3.43], p=0.002
  • Robust gain. The average gain over Vision-OPD is 8.5× the seed std, and every seed beats both baselines on average. Even if each baseline score is a single run with the same seed variance, the gains remain 7.6σ and 8.9σ.
  • Gains concentrate on the hardest fine-grained benchmarks (ZoomBench, HR-8K, HR-4K); V* is near ceiling (>93%).
  • Accuracy and efficiency together. VeGo uses 26% fewer tokens than Vision-OPD and 34% fewer than OPD-V. OPD-V is higher only on Visual Probe (−0.95), where VeGo's responses are 57% shorter.

W3 / Q3: Signed gradients throughout training

We agree that positively aligned gradients are beneficial to first order, and we will replace the absolute-cosine analysis with a signed one. At 7 checkpoints (steps 10–65; 8 questions × 2 rollouts each), we compute the signed cosine between the distillation gradient gDg_D and the length gradient gLg_L, globally and over 760 tensors. To measure interference directly, we then apply one AdamW length-only update (training lr 2e-6), re-evaluate the distillation loss LDL_D on the same data with fixed teacher targets, and restore the weights. ΔLD>0\Delta L_D>0 means the length update hurts distillation.

Stepcos: suffix / all-spanΔLD\Delta L_D (1e-4): suffix / all-span
10+0.04 / +0.190.37 / 4.38
20+0.16 / +0.075.35 / 6.41
30+0.53 / +0.776.93 / 11.94
40−0.05 / −0.1311.77 / 16.69
50−0.09 / +0.114.78 / 4.25
60+0.51 / +0.373.33 / 2.65
65+0.01 / +0.20−1.40 / 6.38

Finding 1: Signed cosine does not reflect actual interference. The global cosine is positive in 11/14 cases, and the fraction of negatively aligned tensors is similar for both variants (40.9% vs. 37.4%). Yet in 10 of the 11 positively aligned cases, the update still increases LDL_D. First-order alignment does not carry over to finite-step Adam updates, so we measure the loss change directly.

Finding 2: Post-decision masking reduces actual interference (question-cluster bootstrap 95% CI, unit 1e-4):

Suffix (VeGo)All-span
ΔLD\Delta L_D, whole response3.61 [−2.70, 10.36]6.70 [0.74, 12.58]
Δ\DeltaJSD, pre-cliff tokens0.67 [−2.44, 3.69]7.24 [0.98, 14.93]

All-span length optimization significantly increases the distillation loss. Post-decision masking reduces this increase by 46%, making it indistinguishable from zero. The difference is concentrated before the cliff, i.e., on the evidence-grounded reasoning that distillation should preserve: the paired difference is −6.57 [−13.89, −0.84], excluding zero. Restricting the length objective to the post-decision suffix therefore protects the pre-cliff distillation signal, consistent with Table 4, where applying it to the whole response lowers accuracy.

Reviewer_HucCOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents VeGo, an on-policy self-distillation framework for improving fine-grained visual reasoning while reducing unnecessarily long responses. A student observes the full image, while a teacher observes both an evidence-rich crop and a degraded version; disagreement between the teacher’s predictions under these views is used to emphasize tokens that depend strongly on visual evidence. The same signal is applied through a prefix-conditioned answer probe to identify a decision cliff, after which a length objective encourages the model to remove redundant reasoning without penalizing the evidence-bearing prefix. Experiments with Qwen3.5-9B on five visual benchmarks show a favorable aggregate accuracy–response-length trade-off compared with baselines.

Critical strengths
  • Continuous evidence-dependent weighting is more flexible than uniformly distilling every token or selecting a hard subset. The loss-sum normalization is also technically motivated to redistribute gradients without changing the overall loss scale.

  • Table 3 compares uniform weighting, hard top-token selection, entropy-based weighting, and two normalization variants which shows informative weighting ablations.

Critical weaknesses
  • The visual-evidence weighting closely resembles VA-OPD [1] which uses token-level visual advantage and V-Zero’s contrast between positive and negative visual evidence. VeGo’s main distinctive element appears to be the decision cliff length objective, but the paper does not clearly compare against or distinguish these methods.

  • Results are reported for only one 9B model and a 6.5K-example training set, without multiple seeds or significance test. The average accuracy improvement over Vision-OPD is only about 1.6 points, and VeGo does not outperform OPD-V on every benchmark.

  • Since the method require computing two teacher distributions, would it require additional computational cost?

[1] https://arxiv.org/abs/2605.21924

[2] https://arxiv.org/abs/2606.25319

Questions for authors

see weaknesses above.

3 comments

We thank the reviewer for the constructive comments.

W1: Relation to VA-OPD [1] and V-Zero [2] (concurrent arXiv work)

Key difference. Both [1] and [2] measure visual dependence by the log-prob gap on the student-sampled token, Δt=log⁡p+(yt)−log⁡p−(yt)\Delta_t=\log p^+(y_t)-\log p^-(y_t) ([1]: max⁡(Δt,0)\max(\Delta_t,0) with top-20% token grouping; [2]: rollout-mean Δt\Delta_t as a rollout gate). VeGo uses the full-distribution gt=JSD(p+∥p−)g_t=\mathrm{JSD}(p^+\|p^-), independent of the sampled token.

Flaw of the sampled-token gap. The gap satisfies Ey∼pS[Δt]=KL(pS∥p−)−KL(pS∥p+)\mathbb{E}_{y\sim p_S}[\Delta_t]=\mathrm{KL}(p_S\|p^-)-\mathrm{KL}(p_S\|p^+). It therefore measures whether the student agrees with the evidence-seeing teacher, not whether the position depends on evidence. Suppose the student writes an evidence-contradicted token, e.g., "black" for a gray helmet. Then p+p^+ assigns it a low probability, while the blind p−p^- does not, so Δt<0\Delta_t<0. [1] and [2] then down-weight exactly the tokens that most need correction.

Verification. On 1,024 on-policy rollouts, we take the first position where the student commits to a wrong answer option. We keep cases where p+p^+ prefers the ground truth, so the teacher can correct the error (n=90).

Wrong tokens written by the studentValue
Δt<0\Delta_t<083.3% (median −1.87)
[1]: in the high-VA top-20% group14.4% (chance: 20%)
[1]: token weight (uniform = 1×)0.89×
VeGo: gtg_t in the rollout's top-20%75.6% (chance: 20%)
VeGo: gtg_t / rollout mean4.1×

Each link of the argument holds:

  1. Errors get negative gaps. 83.3% of the wrong tokens have Δt<0\Delta_t<0. Wrong options scored counterfactually at the same positions agree (83.6% of 207).
  2. [1] thus down-weights errors. Only 14.4% enter its high-VA group, below the 20% chance level, so errors get less than uniform weight (0.89×).
  3. [2] down-weights erroneous rollouts. Wrong-answer rollouts get lower trajectory scores (Mann–Whitney p=5×10−5p=5\times10^{-5}), hence smaller gates.
  4. Yet these tokens are evidence-critical. VeGo's gtg_t ranks 75.6% of them in their rollout's top-20% (4.1× the rollout mean), emphasizing exactly the errors that [1] and [2] suppress.

Second use of the contrast. Read cumulatively (ItI_t), the same signal locates a decision cliff, which enables a post-cliff length objective. Neither [1] nor [2] controls response length.

Comparison with V-Zero on the shared benchmarks (V-Zero numbers taken from [2]):

V*HR-4KHR-8KZoomAvg
V-Zero89.0087.8082.6057.8079.30
VeGo94.2485.7586.0065.5682.89

W2: Seeds and significance

We trained three additional seeds, giving four in total:

V*ZoomVPHR-4KHR-8KAvg
Vision-OPD93.7261.8954.6184.3884.2575.77
OPD-V91.6260.4755.7886.1283.6275.52
VeGo (mean±std)94.21±0.5364.85±0.6154.83±0.3085.83±0.6286.50±0.7577.25±0.17
  • Average gain. Over Vision-OPD, the average gain is +1.48 (95% CI [1.20, 1.76]), which is 8.5× the seed std (one-sample t-test, p<0.001). Over OPD-V it is +1.73 (p<0.001). Every seed exceeds both baselines on average.
  • Per-benchmark gains over Vision-OPD. The gains are significant on the hardest high-resolution benchmarks: ZoomBench +2.96 (p=0.002), HR-8K +2.25 (p=0.009) and HR-4K +1.45 (p=0.02).
  • Shorter responses. These gains come with 26% fewer output tokens than Vision-OPD and 34% fewer than OPD-V (86.4 vs. 116.7 / 130.7).
  • Against OPD-V per benchmark. VeGo wins on V*, ZoomBench and HR-8K by +2.6, +4.4 and +2.9. The HR-4K difference (−0.29) is not significant (p=0.42). On Visual Probe, VeGo's responses are 57% shorter.
  • Controlled comparison. VeGo, Vision-OPD and all ablations use the same backbone and the same 6.5K training data, so the gains come from the objective. The evidence weighting alone improves over uniform distillation by +1.83 (Table 3).

W3: Cost of the negative teacher

The negative teacher adds one gradient-free forward pass on the short crop sequence (619 tokens vs. 3,460 for the student), and it reuses the EMA teacher's weights. Rollout generation and inference are unaffected. We measured one training step (Qwen3.5-9B, 4×H20, 32 rollouts, micro-batch size 1, gradient checkpointing enabled):

w/o negative teacherVeGoOverhead
Time per micro-batch (s)3.6473.771+3.4%
FLOPs per rollout (TFLOP)333.8344.8+3.3%
Peak memory (GB)39.1339.14+0.01

In our full-scale training logs, the negative teacher accounts for 5.5% of step time. VA-OPD and V-Zero likewise require two teacher evaluations per rollout.

[1] arXiv:2605.21924 [2] arXiv:2606.25319

Thanks for the response. The concerns regarding relation to prior works have been addressed. However, some concerns remain whether the proposed method works broadly to other vision language models remain, with its modest improvement. Thus, I remain my original score.

We thank the reviewer for the follow-up. To further examine whether VeGo generalizes beyond the original Qwen3.5-9B setting, we conducted additional experiments on Qwen3.5-4B, using Vision-OPD as the controlled baseline:

MethodBackboneV*ZoomBenchVisual ProbeHR-Bench 4KHR-Bench 8KAvg.
Vision-OPDQwen3.5-4B92.1557.1652.0483.2580.0872.94
VeGoQwen3.5-4B92.6760.8352.6585.3881.0074.51
Gain+0.52+3.67+0.61+2.13+0.92+1.57

VeGo improves Vision-OPD on all five benchmarks, with an average gain of +1.57 points. This is consistent with the +1.48-point average improvement on Qwen3.5-9B, suggesting that the benefit is not specific to a particular model scale or checkpoint. Together with our four-seed results on 9B, these experiments provide additional evidence of robustness across both backbone scales and random seeds.

We agree that the improvement is incremental rather than dramatic. However, it is consistent across all benchmark–backbone combinations, and on 9B it is achieved with 26% fewer output tokens than Vision-OPD. Thus, VeGo improves the combined accuracy–efficiency trade-off rather than obtaining accuracy gains through longer reasoning. We will include the 4B results in the revision and hope this additional evidence addresses the remaining concern.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.