acceptodds
Under review as a conference paper at ICLR 2027

SCOPD: Sparse-Context On-Policy Self-Distillation for Efficient Vision-Language Models

Abstract

Reasoning vision-language models (VLMs) process images and videos as long sequences of visual tokens, making inference expensive. Training-free token pruning can substantially reduce this cost, but performance degrades sharply under aggressive compression, commonly attributed to irreversible loss of task-relevant visual information. We show that this explanation is incomplete. In a fixed-context Pass@K analysis, we prune each visual representation once and repeatedly sample reasoning trajectories from the same sparse context. Although greedy Pass@1 drops substantially, Pass@K recovers many otherwise failed examples, suggesting that useful visual evidence can remain accessible but is not reliably utilized during reasoning. We call this the representation–utilization gap. Motivated by this observation, we introduce SCOPD, a sparse-context on-policy self-distillation framework in which a student model generates reasoning trajectories from pruned visual tokens while a privileged full-context teacher supervises those on-policy prefixes. SCOPD requires no ground-truth responses, architectural modifications, or inference-time computation. Building on SCOPD, we develop SCOPD+ to focus distillation where visual evidence matters most. A small visual-budget intervention identifies positions most sensitive to additional visual evidence and selectively distills them while backpropagating through only a fraction of responses. At 10% visual-token retention, the Vanilla base model retains only 86.37% of the unpruned model's performance across 13 benchmarks. SCOPD raises this normalized aggregate to 90.49%, while SCOPD+ further improves it to 92.43%. Across token budgets, benchmarks, and pruning operators, our results show that efficient reasoning VLMs depend not only on which visual information survives pruning, but also on how reliably the language model learns to use the sparse representation that remains.

Primary area:
foundation or frontier models, including LLMs
Keywords:
VLMs, On policy distillation, Inference optimization, Pruning, Post-training

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_PQe8OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The authors propose SCOPD, a post-training framework aimed at recovering the performance of LVLMs that have undergone aggressive visual token pruning. The work is predicated on the observation of a "representation-utilization gap"—the finding that a pruned visual context still yields correct answers under high-budget stochastic sampling (Pass@K), implying that the language model simply fails to reliably extract surviving visual evidence during greedy decoding. To address this, SCOPD employs a standard on-policy self-distillation approach, where a student model conditioned on sparse visual context is trained to match the token-level predictive distribution of a frozen, full-context teacher. A selective variant, SCOPD+, applies distillation only to a subset of tokens deemed "visually sensitive," which are identified by measuring the Jensen-Shannon Divergence (JSD) of the student’s output when the visual token budget is marginally increased.

Critical strengths
  1. It is important to distinguish “information absent from the retained tokens” from “information present but unreliably used by the decoder.”
  2. The authors evaluate their method across a wide array of 13 image benchmarks and 5 video benchmarks, testing generalization across different base models (Qwen2.5-VL and Qwen3-VL) and various pruning strategies (VisionZip, DivPrune, FastV).
Critical weaknesses
  1. The core “representation–utilization gap” claim is not causally established. The paper’s main claim is that sparse visual representations often retain sufficient task-relevant evidence, but the model fails to use this evidence reliably. However, the fixed-context Pass@K experiment does not rigorously establish this causal interpretation. The main evidence compares the setting with 10% retained visual tokens against a language-only baseline. The language-only control is potentially confounded. For image-dependent questions, VLM without image input may frequently produce responses such as “I cannot see the image,” refuse to guess, fail to follow the requested answer format, or otherwise behave conservatively. In that case, the near-zero language-only Pass@K does not necessarily show that language priors are insufficient.
  2. The comparison between “correct-image 10% tokens” and “no image tokens” is not the right causal control. It changes both the availability of relevant visual evidence and the model behavior. A more appropriate control should preserve the same number of visual tokens, while changing only whether the visual evidence is relevant. For example: ''10% task-relevant tokens'' with ''10% strictly task-irrelevant tokens''. Without proving that the recovery relies only on relevant tokens, this framework is likely just forcing the model to confidently hallucinate visual evidence when it actually cannot see the image clearly.
Questions for authors
  1. In SCOPD+, the "visual sensitivity" criterion is measured using the JSD between predictions under budgets b and b+δ. However, this metric essentially measures the model's output instability or sensitivity to additional tokenspruning results. It does not necessarily prove that the model is sensitive to task-relevant visual evidence. Could you provide deeper theoretical justification or empirical validation?

  2. Why does SFT underperform Vanilla at the full 100% visual-token budget, while improving over Vanilla at 20% and 10% token retention?

  3. The paper appears to be an extension of a NeurIPS 2026 workshop paper. Could the authors clearly describe the relationship between the two versions and enumerate the new contributions in the current submission?

5 comments

We thank the reviewer for the detailed review and concrete suggestions. We agree that the key question is whether task-relevant information is missing from the retained visual tokens or remains available but is used unreliably. To investigate this distinction more carefully, we have added the suggested controls and further analyses.


W1 & W2: Is the representation–utilization gap causal, and does recovery depend on task-relevant evidence?

Language-only control. We examined all 32,000 language-only responses generated from 500 unique questions, with 64 responses per question, using the GPT-6-Astra judge described in Appendix A. In 25% of responses the model mentions that there is no visual information available but still guesses an answer and in 20% of responses it refuses to answer. These response-level rates indicate that refusals could contribute to the low language-only score. We therefore added the matched-budget controls below to assess the dependence of recovery on image-specific input. Our Pass@K evaluation requires not only a correct final answer but also logically consistent reasoning supported by the original image (Sec. 3.1, Appendix A). Figure 5 provides examples of accepted and rejected reasoning trajectories.

Controls with the same visual-token budget. Following the reviewer's suggestion, we repeated the fixed-context analysis on the same 500 questions using three additional conditions, each retaining 10% visual tokens: (i) tokens from an unrelated image, (ii) the least important tokens from the correct image according to VisionZip, and (iii) randomly selected tokens from the correct image. Random and least-important token selection serve as additional controls rather than strictly irrelevant visual contexts. Random tokens may still preserve useful evidence, while the least-important tokens are determined by VisionZip's importance ranking, which does not necessarily imply that they contain no task-relevant information.

Retained visual tokens (10%)GreedyPass@64
VisionZip53.279.6
Random41.868.8
Least important, same image3060.6
From another image5.411.4
None (language-only, Fig. 1b)2.84.2

The results provide stronger evidence for our interpretation. Replacing the retained tokens with those from an unrelated image reduces Pass@64 from 79.6% to 11.4%, despite preserving the same visual-token count. In contrast, random tokens from the correct image recover 68.8%, and even the least important tokens recover 60.6%. These results suggest that recovery depends strongly on image-specific visual evidence, rather than simply on the presence of visual tokens or a greater tendency to generate answers. While this does not establish which individual tokens contain the required evidence, it strengthens the representation–utilization interpretation. We have added these controls to Fig. 1b, and Sec 3.1.

Regarding the concern that adaptation may encourage confident hallucinations, we also observe improvements on HallusionBench at both 10% retention (58.57 for Vanilla, 60.46 for SCOPD, and 60.99 for SCOPD+) and 20% retention (61.51, 63.83, and 65.72, respectively). While not a direct test of hallucination frequency, these results provide additional supporting evidence that the gains are not limited to answer-generation behavior.


Q1: Does the sensitivity score reflect task-relevant evidence?

We agree that our visual sensitivity metric measures how much the model's predictions change under a small increase in the retained visual context, rather than directly identifying task-relevant visual evidence. However, we would like to clarify that our goal is not to identify which positions correspond to task-relevant evidence. Instead, as hypothesized at the end of Sec. 3.2, our motivation is that positions sensitive to changes in the visual context may be particularly useful for applying full-context teacher supervision, helping the student better utilize its sparse visual representation.

We validate this hypothesis empirically. Table 2 compares our sensitivity-based selection against several alternatives, and Appendix H extends these comparisons to all 13 benchmarks and additional baselines. SCOPD+ achieves 92.43 Avg13, compared with 90.48 for random selection, 91.29 for Top-KL, and 86.35 for selecting the least sensitive positions. Moreover, Masked SCOPD, which excludes the top 10% most sensitive positions and distills the remaining 90%, achieves only 89.46, even below dense SCOPD (90.49).

Together, these experiments support our hypothesis that visual-budget sensitivity is a useful criterion for selecting where to apply distillation, without requiring the stronger claim that the metric explicitly identifies task-relevant visual evidence.

Q2: Why is SFT below Vanilla at full context but above it at 20% and 10%?

SFT is trained exclusively at 10% visual-token retention using reference reasoning traces from LLaVA-CoT. One plausible explanation for the observed trade-off is that fine-tuning adapts the model to sparse inputs and fixed reference traces, which can also change its performance when evaluated with full visual context.

Importantly, the degradation at full context is relatively small: SFT achieves 99.17 Avg13 compared with 100.00 for Vanilla, with the largest drops on MathVerse and LogicVista (6.98 and 4.69 points). At lower retention budgets, the benefit of sparse-context adaptation outweighs this degradation: SFT improves over Vanilla from 93.42 to 95.22 at 20% retention and from 86.37 to 88.82 at 10%.

SCOPD is also trained at 10% retention but retains its full-context performance (100.67 Avg13). Unlike SFT, SCOPD uses on-policy trajectories and supervision from an EMA teacher with access to the full visual context, rather than fixed external reasoning traces. This difference in supervision may help preserve the model's original reasoning behavior while adapting it

Dear Reviewer,

We understand how busy this period is, and sincerely appreciate the time you have devoted to reviewing our paper.

With less than three days remaining in the discussion period, we wanted to briefly follow up on our rebuttal and revised manuscript. In particular, we have added the matched-budget visual controls you suggested to strengthen the evidence for our representation–utilization hypothesis, along with additional experiments and clarifications addressing your other questions.

We would be grateful if you could let us know whether these additions address your concerns, or if there is anything further we could clarify before the discussion period ends.

Thank you again for your time and thoughtful feedback!

Reviewer_Bei4OpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper identifies a representation–utilization gap through Pass@K analysis and proposes SCOPD. This on-policy self-distillation framework uses a full-visual-context teacher to supervise a student operating on pruned visual tokens. This work further introduces SCOPD+, which selectively distills visually sensitive reasoning positions. Experiments on 13 benchmarks demonstrate consistent improvements under aggressive pruning without introducing additional inference-time overhead.

Critical strengths
  1. The fixed-context Pass@K analysis is particularly interesting, as it suggests that reasoning failures can occur even when useful evidence remains accessible in the sparse visual representation. This shifts the focus from designing better pruning strategies to improving the model's ability to utilize compressed information.
  2. SCOPD+ introduces visual-budget interventions to distinguish visually sensitive reasoning positions from ordinary teacher–student disagreement. This is a useful methodological refinement over uniformly applying distillation or selecting positions solely based on KL divergence.
Critical weaknesses
  1. Although the motivation is clear, the pruning technique is leveraged from previous work, VisionZip, and this work only utilizes a normal distillation method to fine-tune the model to adapt to the pruned context.
  2. There are previous works that explicitly investigate fine-tuning vision-language models to adapt to compressed or pruned visual contexts with a similar distillation idea, such as [1] and [2]. It would be better to compare with those methods.

[1] Wen, Z., Wang, S., Zhou, Y., Zhang, J., Zhang, Q., Gao, Y., ... & Zhang, L. (2026). Efficient multi-modal large language models via progressive consistency distillation. Advances in Neural Information Processing Systems, 38, 69726-69753.

[2] Guo, X., Luo, G., Zhang, J., Wang, K., & Wang, W. (2026). Token-Budget Distillation: Transferring Full-Token Semantics to Compressed Video Vision-Language Models. arXiv preprint arXiv:2608.28138.

Questions for authors

See weakness.

4 comments

We thank the reviewer for the constructive feedback. We clarify the distinction from existing distillation methods and report an additional baseline comparison in the revised submission.


W1: Use of VisionZip, and how SCOPD differs from standard distillation

We are glad the reviewer finds that our fixed-context analysis shifts the focus from designing better pruners to improving how models use compressed visual information. This is the focus of our paper, and why we build on an existing pruner rather than proposing a new one. Additionally, we find that the learned adaptation transfers to unseen pruners (Table 3). With SCOPD+, even random pruning (88.85 Avg6) slightly outperforms the original Vanilla model with VisionZip (88.74), and also exceeds DivPrune (83.47) and FastV (81.47). This suggests that adapting the language model to sparse visual contexts can be as important as, or even more important than, the choice of pruning operator.

We also clarify how SCOPD differs from standard supervised distillation:

  1. On-policy trajectories. Standard supervised distillation trains on fixed responses, such as reference answers or teacher-generated outputs. In SCOPD, the student generates its own reasoning from the question and sparse visual context, and the teacher supervises those same trajectories. The two baselines trained on fixed reference responses, SFT and EPIC, perform below SCOPD at all three budgets in Table 1 (at 10% retention: 88.82 and 86.45 vs. 90.49). In particular, our on-policy self-distillation substantially outperforms EPIC's progressive hidden-state distillation under the same evaluation setup.

  2. The model as its own teacher. The teacher is an exponential moving average (EMA) of the student's weights, given the full visual context. Its privileged information consists of the visual tokens removed by pruning, so SCOPD requires no external teacher, ground-truth answers, or reasoning traces.

  3. Selective supervision. SCOPD+ distills only the 10% of response positions most sensitive to additional visual evidence. This differs from distilling every position or selecting positions by teacher-student KL alone. Among the selection rules evaluated, including the adapted VA-OPD baseline, teacher-student KL, TIP, and random selection, SCOPD+ achieves the highest Avg13 (Appendix H of the revised submission).


W2: Additional baselines

We thank the reviewer for pointing us to these two works.

[1] EPIC (Wen et al.). This method was already cited and compared in Table 1 (baseline (iii) in Sec. 4.1) under its original name. At 10% retention, EPIC reaches 86.45 Avg13, compared with 90.49 for SCOPD and 92.43 for SCOPD+. This is a gain of 4.04 and 5.98 points, respectively, demonstrating the effectiveness of our on-policy self-distillation compared with EPIC's progressive hidden-state distillation.

[2] Token-Budget Distillation. This recent work appeared on arXiv shortly before the original submission deadline, and we had missed it. While ICLR guidelines do not require comparisons with contemporaneous or arXiv-only work, we nevertheless implemented and evaluated it to provide a more comprehensive comparison. Since its released code supports LLaVA-Video and LLaVA-OneVision with FlashVID video compression, we reimplemented its objective for our image-based VisionZip setting, following the paper and released training code while keeping its loss terms, weights, and thresholds. We used the same data, base model, LoRA configuration, and 10% VisionZip budget as our other baselines (see Appendix C for the detailed implementation details). Like SFT and EPIC, it trains on reference responses, so we also indicate which labels each method uses.

MethodExternal reference supervisionAvg13 at 10% retention
Vanilla (no training)–86.37
SFTreference traces and answers88.82
EPIC [1]reference traces and answers86.45
Token-Budget Distillation [2], our reimplementationreference traces and answers91.6
SCOPDnone90.49
SCOPD+none92.43
SCOPD, teacher also given the ground-truth answer (App. F)reference answers92.86

Avg13 is the 13-benchmark average normalized to the unpruned model, as in Table 1. Token-Budget Distillation is a strong baseline, outperforming SCOPD, while SCOPD+ achieves higher performance without using reference answers or reasoning traces. We have added this baseline, its experimental results, and the corresponding discussion to Section 4 of the revised submission.

Thanks to the authors for the detailed rebuttal. I'll maintain my score.

We would appreciate it if you could clarify which concerns remain unresolved, as we tried to directly address both of your main weaknesses:

1. Novelty beyond standard distillation: We clarified the differences in SCOPD’s on-policy training, and selective supervision in SCOPD+. We also provided empirical comparisons with SFT, EPIC, and alternative token-selection strategies.

2. Missing comparisons: EPIC was already included in Table 1, and we additionally implemented Token-Budget Distillation under the same experimental setup. While this baseline outperforms SCOPD, SCOPD+ achieves higher performance without requiring ground-truth answers or reasoning traces.

Given these clarifications and additional experiments, we would be grateful to understand what specific concerns still justify the original rating.

We would be happy to further clarify any outstanding points.

Dear Reviewer,

We understand how busy this period is and appreciate the time you have devoted to our paper.

With less than three days remaining in the discussion period, we wanted to briefly follow up on our previous message. We have made a substantial effort to address your two main concerns, particularly by clarifying the methodological differences from existing distillation approaches and adding a direct comparison with Token-Budget Distillation in our revised manuscript.

We would be grateful if you could clarify which concerns remain unresolved despite these additions, so that we have an opportunity to address them before the discussion period ends.

Thank you again for your time and consideration!

Reviewer_3252OpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates why vision-language models degrade under aggressive visual-token compression. A fixed-context sampling analysis shows that correct, visually grounded responses can remain recoverable from the same pruned representation, suggesting a gap between retained evidence and its reliable utilization. Motivated by this finding, SCOPD trains a sparse-context student through on-policy distillation from a full-context teacher. SCOPD+ further selects supervision positions using their sensitivity to a small increase in the visual-token budget. Experiments evaluate performance recovery across compression budgets, pruning operators, two Qwen model variants, and image and video benchmarks. I think this paper is interesting.

Critical strengths

1.The paper provides an informative empirical finding with useful controls. Holding the pruned representation fixed makes the recoverability analysis directly relevant to visual-token compression. The language-only comparison and reasoning-validity checks strengthen the evidence that some pruning failures arise from unreliable utilization of surviving visual information. This offers a useful perspective for designing compressed VLMs.

2.The finding motivates a practical adaptation method with encouraging transfer results. At 10% visual-token retention, SCOPD+ improves the normalized average performance across 13 image benchmarks from 86.37 to 92.43. Transfer to other pruning operators and video benchmarks supports the usefulness of the learned adaptation beyond the primary training setup. The method requires neither a teacher nor the visual-budget intervention at inference.

Critical weaknesses

1.The distinctive contribution relative to closely related approaches needs stronger experimental isolation. Recoverability after compression has been studied in ShortOPD, while VA-OPD uses visual interventions to identify informative distillation positions. The paper acknowledges these connections and introduces a different compression setting and sensitivity criterion. However, a controlled comparison with an adapted visual-advantage selection or weighting baseline would better establish the benefit of its specific design.

  1. The consistency of SCOPD+'s incremental improvement remains uncertain. Its normalized average gain over Top-KL in Table 2 is 0.74 points, and its performance is nearly identical to dense SCOPD at 5% retention in Table 7. Independent training-seed results and uncertainty estimates are not reported. These would help establish when the proposed selection rule provides a reliable advantage.
Questions for authors

1、Could you compare the proposed sensitivity criterion with an adapted visual-advantage selection or weighting baseline under the same pruning, teacher, and training configuration? This would clarify the contribution of the local visual-budget intervention.

  1. How many independent training runs underlie the main results and selection ablations? Could you report means and standard deviations for SCOPD, SCOPD+, and Top-KL at representative retention budgets?

  2. Could you add more releated work in Towards Efficient Multimodal Large Language Models: A Survey on Token Compression for enhanced efficient vision-language models literature review

3 comments

We thank the reviewer for the careful review and are glad they found the fixed-context analysis informative and the method practical. We address each point below.


W1 / Q1: Comparison with a visual-advantage baseline

Thank you for suggesting this comparison. The two rules measure different signals. VA-OPD scores a position by how much the original image, relative to a low-resolution copy, increases the teacher's probability of the student's token, and ignores positions where it decreases. SCOPD+ instead measures how much the student's own next-token distribution changes when its visual-token budget increases from 10% to 11%. It therefore directly probes sensitivity to the visual evidence affected by pruning and captures changes in either direction.

To compare these supervision rules under a common training setup, we trained an adapted VA-OPD variant of SCOPD using the same VisionZip 10% pruning, EMA teacher, forward KL, data, LoRA configuration, and number of updates; only the position weighting differs. We compute visual advantage as in VA-OPD using a low-resolution image copy, then follow its grouped objective: the top 10% of positions by advantage form one group (VA-OPD uses 20%; we use 10% to match SCOPD+), and the remaining 90% form the second. The loss is λ\lambda times the mean KL over the first group plus (1−λ)(1 - \lambda) times the mean KL over the second, with λ=0.5\lambda = 0.5 as in VA-OPD. Since SCOPD+ selects positions within each response, we do not use VA-OPD's response-level weighting.

As also suggested in W2, we expand this comparison from the original 6-benchmark subset to all 13 image benchmarks used in our main evaluation to better assess the consistency of the selection criteria across tasks.

Positions distilled (10% retention)Avg6 (Table 2)Avg13
All positions (dense SCOPD)94.4490.49
Random 10%93.5690.48
Top-KL 10%94.5191.29
TIP 10%94.2191.68
Least sensitive 10%89.1386.35
VA-OPD weighting (adapted)95.491.71
SCOPD+95.2592.43

The adapted VA-OPD baseline scores 95.40 on the original six-benchmark subset, compared with 95.25 for SCOPD+. Across all 13 image benchmarks, SCOPD+ scores 92.43, compared with 91.71 for the adapted baseline. SCOPD+ also does so without requiring an additional low-resolution pass through the vision encoder, and distills only the selected 10% of response positions, while the adapted VA-OPD objective still applies supervision across all positions. We have updated Q1 in Sec 4.3 and added an Appendix H to the revised submission to report the full 13-benchmark comparison and added the corresponding discussion.


W2 / Q2: Seeds and the size of the gains

Our original results were based on a single training run per configuration. As suggested we are currently running two additional seeds for SCOPD, SCOPD+, and Top-KL to assess training variability. All runs use 10% visual-token retention during training and will be evaluated at 10% and 20% visual-token retention. We will report the mean and standard deviation across the three training runs for each method and evaluation budget.

Alongside the additional training runs, we examine how the observed gains vary across tasks and retention budgets. We expanded the selection ablations from six to 13 image benchmarks. As shown in the updated comparison under W1, SCOPD+ achieves the highest Avg13 at 92.43%, outperforming Top-KL (91.29%), TIP (91.68%), and VA-OPD (91.71%). Notably, its advantage over Top-KL increases to 1.14 points, compared with 0.74 points in the original Table 2. We have included the expanded results in Appendix H of the revised submission.

Performance at extreme pruning. At 5% retention, SCOPD+ and SCOPD perform nearly identically (83.66 vs. 83.63 Avg8), although both substantially outperform the pruned Vanilla model (75.99%). Both methods were trained at 10% retention and evaluated at 5% without further adaptation. The advantage of SCOPD+ is largest at its training budget (+1.94 points at 10%), decreases at 20% (+0.45 points), and becomes negligible at 5%. We acknowledge that selective supervision does not provide uniform gains at every pruning budget and does not generalize to budgets lower than it was trained on. We have clarified this limitation in the revised paper at Appendix D.


Q3: Related work from token-compression surveys

Thank you for pointing us to this survey. We have added both Yao et al. (2026) and Shao et al. (TMLR 2026) to our related work, along with additional representative compression methods including DeCo, FOLDER, and FiCoCo. These complement the pruning methods already discussed in the paper, while SCOPD focuses on adapting the language model to better utilize the retained visual representation, independently of the compression strategy.

Thanks to the authors for hard working in the rebuttal. All my concerns are resolved. I'll maintain my positive score.

Thank you very much for your positive feedback. We are glad to hear that your concerns have been resolved.

We also wanted to follow up on the additional training-seed experiments mentioned in W2. We have now completed three independent training runs at 10% visual-token retention, with ρ=10\rho=10% for the selective methods. The results remain consistent across seeds:

MethodOriginal seedSeed 43Seed 44Mean ± SD
SCOPD90.4990.9790.3790.61 ± 0.32
SCOPD+92.4391.8992.2592.19 ± 0.27
Top-KL91.2991.1790.8991.12 ± 0.21

All values are Avg13 normalized performance at 10% visual-token retention. SCOPD+ consistently outperforms both dense SCOPD and Top-KL across all three seeds, further supporting the reliability of the gains from visually selective supervision.

We have also added these results and the corresponding discussion to Appendix H of the revised submission. Thank you again for your constructive feedback.

Comments on the paper

2 comments

Dear reviewers,

the authors have now submitted their responses. Please review them and share your thoughts on whether your concerns have been addressed. I encourage you to engage with one another’s assessments, discuss any remaining disagreements, and update your reviews and scores as appropriate.

Thank you.

AC

Summary of Rebuttal and Revised Submission

Dear Area Chair,

As the discussion period closes, we would like to briefly summarize the status of each review and the main additions to our revised submission.

Reviewer 3252 (weak accept) explicitly confirmed that all their concerns were resolved following our additional experiments, including the adapted VA-OPD baseline, expanded 13-benchmark ablation, and related work. We also completed the requested training-seed experiments, confirming that SCOPD+ consistently outperforms SCOPD and Top-KL across three independent runs (Appendix H).

Reviewer Bei4 (weak reject) maintained their initial score. We addressed both main concerns: clarifying how SCOPD differs from standard distillation and evaluating the suggested baselines. EPIC was already included, while we reimplemented Token-Budget Distillation and added it to Table 1. SCOPD+ outperforms this baseline (92.43 vs. 91.60 Avg13) without requiring reference labels. We followed up to ask whether any concerns remained.

Reviewer PQe8 (weak reject) acknowledged that our responses improved their assessment but maintained concerns about the representation–utilization gap and potential hallucination. We conducted two additional experiments directly following their suggestions:

  • Matched-budget controls (Fig. 1b, Sec. 3.1): Replacing retained visual tokens with those from an unrelated image reduces Pass@64 from 79.6% to 11.4%, despite preserving the same visual-token budget. This supports our interpretation that recovery depends on image-specific visual evidence.
  • Wrong-image training (Appendix I): Training SCOPD with mismatched visual evidence reduces Avg13 to 75.90%, compared with 90.49% for SCOPD and 86.37% for Vanilla. Additional HallusionBench and POPE evaluations further support the importance of correct visual supervision.

Overall, we have substantially strengthened the submission with new experiments addressing the reviewers' specific concerns, including matched-budget recoverability controls, wrong-image training, Token-Budget Distillation, VA-OPD comparisons, and training-seed analyses. All have been incorporated into the revised manuscript.

We would be grateful if these additions and the detailed rebuttal discussions could be considered in the final assessment.

Thank you for your time and consideration.

The authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.