acceptodds
Under review as a conference paper at ICLR 2027

See What I See: On-Policy Distillation of Visual Evidence, Not Just Words

Abstract

On-policy distillation (OPD) has become a core technique in the post-training of large language models and is increasingly applied to multimodal ones. It trains a student with dense token-level feedback from a teacher on the student's own rollouts. However, this feedback supervises what the student says, not what it sees: even a correct token may draw little from the image, or draw from the wrong region. We find that during OPD the student's output distribution converges to the teacher's, while an evidence gap persists. The gap has two sides: (i) a few load-bearing tokens carry most of the teacher's evidence, yet their supervision is diluted across the whole response; (ii) on these tokens the student's evidence misses the ground-truth region, even where its next-token distribution already matches the teacher's. To close this gap, we propose SWIS ( See What I See ), which distills the teacher's visual evidence directly rather than through its next-token distribution alone. SWIS models evidence as a joint distribution over token–patch pairs and aligns the student's with the teacher's, matching both which tokens carry the evidence and where on the image it lands. Across ten benchmarks, SWIS improves average accuracy by 1.8 pp, gaining most where answers depend on the image, and closes the evidence gap under the same readout that exposed it.

Primary area:
foundation or frontier models, including LLMs
Keywords:
VLM, Computer Vision

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_5WM2OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies on-policy distillation (OPD) of multimodal LLMs. The authors show that current multimodal OPD fails to transfer how the teacher uses the image: the student's outputs converge to the teacher's, but the image regions behind its tokens stay wrong. They call this the evidence gap. Their method, SWIS, scores how much each token relies on each image patch. These scores form a joint token-patch distribution. SWIS adds one KL divergence between teacher and student on this distribution. By the chain rule, it splits into two terms: which tokens use the image, and where the evidence lands. No weighting between them is needed. Experiments cover 10 benchmarks, 2 student sizes, and 2 teacher setups. SWIS gains about 1.8 points of average accuracy over vanilla OPD. It also moves evidence onto the annotated regions, close to the teacher's level. A masking check that avoids the readout backs both claims.

Critical strengths
  1. The paper identifies a real gap in the field. Existing OPD methods match only the teacher's per-token output distribution, leaving how the image is used unsupervised. Cross-modal alignment is a central problem for current multimodal LLMs, so this question matters beyond distillation. The authors verify that the problem exists through a fairly detailed diagnostic analysis in the OPD setting.
  2. The experiments around the method are thorough.They show the gain and the bottleneck of each component: mass and placement terms separately and jointly (Tab. 2), the choice of readout (Tab. 3), the teacher's privileged input (Tab. 4), and the loss weight (Tab. F2).
  3. The paper is clearly written and easy to follow.The mass/placement framing is introduced early and used consistently from the diagnosis to the ablations, and most claims are tied to a specific figure or table.
Critical weaknesses
  1. The target quality is bounded by the teacher, and errors can propagate. The paper itself notes that "SWIS inherits where the teacher looks: its Energy-PG approaches the teacher's ... for right and wrong answers alike" (App. G.4). The teacher's own grounding is moderate (Energy-PG 33.8 vs. chance 14.9), so when the teacher attends to spurious regions, the loss teaches the student to do the same. Since the training data has ground-truth boxes, gating the evidence loss by teacher evidence quality seems worth exploring.
  2. The improvements in Table 1 are not clearly beyond noise.The paper claims "the best average accuracy in all four settings" and, at 4B, "the only method that improves on its vanilla recipe on all ten benchmarks" (Sec. 4.2). Yet on MMVP (n=300), a 2-point gain is about 6 questions; the 2B-OPD margin over the strongest baseline is 0.66 points; and vanilla OPD improves over the base model by only 0.21 points (Tab. 1). These claims are stronger than the margins support.
  3. The grounding gains may reflect optimization toward the metric.The abstract concedes the gap is closed "under the same readout that exposed it." The evidence loss, the Energy-PG metric, and the V verdict of the process evaluation ("V holds when the response's Energy-PG reaches τ", App. H) are all built from this readout, so the student's in-box share rises almost by construction. Consistently, at λ=0.2 Energy-PG reaches 33.4 while accuracy drops (Tab. F2). An evaluation the loss does not touch, such as an off-family attribution, would make the grounding claims convincing.
  4. Benchmark images overlap the training set. App. D states: "We did not remove benchmark images from this training set, which all compared methods share." The overlap is never quantified, and it is concrete: POPE is built on COCO images while the training set is largely COCO-derived, and Geometry3K is used in full and feeds MathVista. Because the evidence loss binds tokens to image regions, memorization may interact with evidence supervision rather than add to it, so sharing the data does not fully neutralize the comparison.

Taken together, these weaknesses suggest that the paper reliably shows the student can be made to look where the teacher looks; what is not yet established is that this transfer brings reliable gains on the task itself.

Questions for authors
  1. Why does better placement improve the task? The premise of the paper is that the student fails to inherit the teacher's cross-modal ability. Yet one could argue that both student and teacher largely exploit global image information, and the observed gap reflects only where the chosen readout concentrates its mass. Beyond the correlations in Figs. 6–7, please provide a logical account of why optimizing placement — making the evidence land in the annotated region — translates into task-level gains.
  2. Independent evaluation metrics. The evidence loss, the Energy-PG metric, and the process-evaluation verdict are all derived from the same readout, so part of the improvement may reflect optimization toward the metric itself. Please supplement an evaluation that the loss does not touch, for example re-scoring the trained student with an off-family attribution (patch ablation, LIME, or Grad-CAM), to rule out this misleading effect.
  3. Decontaminated comparison. Please remove the benchmark-overlapping portion from the evaluation and add the corresponding comparison experiments, so that the method's effect on the tasks is not distorted by images seen during training. The concrete overlaps are POPE–COCO and MathVista–Geometry3K. These three questions are central to my assessment; a convincing answer to each would raise my rating.
8 comments
AuthorsOpenReview ↗

Response to Reviewer 5WM2 (Part 1/4)

Q1. "Why does better placement improve the task? [...] one could argue that both student and teacher largely exploit global image information, and the observed gap reflects only where the chosen readout concentrates its mass. [...] please provide a logical account of why optimizing placement [...] translates into task-level gains."

The evidence loss makes the student read the part of the image that the teacher reads, so the gain appears on the questions that a prior cannot answer. In these tasks the answer is read from a small part of the image. Next-token matching gives full credit to a token produced without reading that part, from a prior or from the context, and such a token is right only where the prior holds. All four are measured in accuracy, not with the readout.:

  1. The teacher depends on the region, not on the image as a whole. Deleting the annotated region, 14.9% of the image, lowers its accuracy from 64.42 to 22.54; a random region of the same size lowers it only to 61.84.
  2. Next-token matching leaves the student's dependence on the region unchanged, and SWIS raises it. The same deletion costs vanilla OPD 15.96 accuracy points, about as much as the base model, and SWIS 30.57 (Table R2, Part 2/4).
  3. The gain follows the target. With a random box given to the self-teacher, the student falls below vanilla OPSD (Table 4 of the paper). Placement turns into accuracy on the tokens that read the image: +0.35 matched token by token, +1.28 with the mass as a second loss, and +1.90 in the joint loss, which weights tokens by how much they draw from the image (Table 2 of the paper).
  4. The gain concentrates on the questions that need the image. We split the questions by whether the base model answers them correctly from a blank image.

Table R1: Accuracy (%) on the questions that the base model answers correctly from a blank image (N = 3,881) and on the others (N = 8,994); gain over the vanilla recipe in parentheses. Qwen3.5-4B, seven benchmarks; the three binary ones are excluded, since a guess is right half of the time.

MethodBlank-solvableImage-needed
OPSD
Vanilla OPSD77.3361.37
SWIS (Ours)77.45 (+0.12)62.19 (+0.82)
OPD
Vanilla OPD77.7460.28
VA-OPD78.85 (+1.11)61.05 (+0.77)
SWIS (Ours)77.94 (+0.20)62.53 (+2.25)

Under OPD, SWIS gains +2.25 points on the image-needed questions and +0.20 on the others; under OPSD the direction is the same. VA-OPD, the strongest OPD baseline, gains no more on the image-needed questions than on the others, so the concentration is a property of SWIS and not of the split.

Revision. This account and Table R1 are in Appendix J.6, and Analysis 3 of Sec. 5 refers to it (blue).

W1. "The target quality is bounded by the teacher, and errors can propagate. [...] when the teacher attends to spurious regions, the loss teaches the student to do the same. Since the training data has ground-truth boxes, gating the evidence loss by teacher evidence quality seems worth exploring."

SWIS transfers the teacher's evidence by design, and Appendix G.4 reports what this implies. On the questions the teacher answers wrongly, SWIS passes on fewer errors than vanilla OPD. Where the teacher looks off the box, the student's evidence moves little, and its accuracy is above vanilla OPD (Table G2 of the paper).

  • Teacher wrong. Where the teacher's evidence hits the box, vanilla OPD falls below the base model (24.55 vs. 29.41), so next-token matching already passes on the teacher's errors. SWIS reaches 35.81.
  • Teacher off the box. The student's Energy-PG moves by only 2.9–3.7 points, and its accuracy is above vanilla OPD whether the teacher is right or wrong.
  • Teacher quality. The teacher's Energy-PG of 33.8 pools every token of a response, including reasoning tokens that are not about the region. On the load-bearing tokens its in-box share is 45.2% (Fig. 3c). SWIS improves over its vanilla recipe with each of the three teachers we tested, and a Qwen3-VL-8B teacher on a second backbone (Appendix J.4).
  • The suggested gate. We leave it to future work for two reasons. It would remove the evidence loss exactly where the student still gains over vanilla OPD, and in the OPD setting it would make SWIS depend on boxes, which the loss otherwise never uses.

Revision. The gate and a second direction, refreshing the self-teacher from the improved student, are added to Appendix I (blue).

AuthorsOpenReview ↗

Response to Reviewer 5WM2 (Part 2/4)

Q2. "The evidence loss, the Energy-PG metric, and the process-evaluation verdict are all derived from the same readout, so part of the improvement may reflect optimization toward the metric itself. Please supplement an evaluation that the loss does not touch, for example re-scoring the trained student with an off-family attribution (patch ablation, LIME, or Grad-CAM) [...]."

We ran the kinds of evaluation the review suggests: patch ablation, a perturbation test like LIME (Ribeiro et al., 2016); Grad-CAM (Selvaraju et al., 2017) with its high-resolution variant HiResCAM (Draelos & Carin, 2020); and the input gradient (Simonyan et al., 2014). None of them uses attention weights or Eq. (2).

Under patch ablation, HiResCAM and the input gradient, SWIS depends on the annotated region significantly more than vanilla OPD. On the share-based measures it covers 73–82% of the distance from vanilla OPD to the teacher, close to the 80% under Energy-PG.

(1) Patch ablation. On the same 2,951 held-out examples, we delete the visual tokens of the annotated region, of a random region of equal size, or of the whole image.

Table R2: Region deletion (Qwen3.5-4B, OPD). Drop: accuracy lost. Δ log p: log-likelihood of the model's own response lost per token (nats) without the region. Region share: Δ log p for the region over that for the whole image.

ModelAcc.Drop w/o region ↑Drop w/o randomΔ log p ↑Region share (%) ↑
Teacher (9B)64.4241.882.580.28454.7
Base Model57.4014.941.930.12529.8
Vanilla OPD59.6715.961.760.13832.1
SWIS (Ours)62.9330.572.340.22448.6

Without the region, SWIS loses 14.61 more accuracy points than vanilla OPD, while a random region makes no significant difference. The region share measures the same allocation as Energy-PG without the readout.

(2) Gradient-based attribution. We use Grad-CAM, HiResCAM, which keeps the gradient of every token instead of pooling it, and the gradient with respect to the pixels.

Table R3: Energy-PG (%) on the held-out split under attributions that the loss does not use (chance: 14.9).

ModelEq. (2)Grad-CAMHiResCAMInput gradient
Teacher (9B)33.8215.0723.8119.10
Base Model24.7014.8918.9816.64
Vanilla OPD27.0915.0719.5217.02
SWIS (Ours)32.5015.2323.0418.66

Grad-CAM stays within 0.4 points of chance for every model, the teacher included, so it cannot separate them. HiResCAM and the input gradient localize the region, and under them SWIS is above vanilla OPD by 3.52 and 1.64 points. The grounding gain therefore holds under evaluations that the loss never touches.

Revision. Tables R2 and R3 with their protocols are in Appendix J.3 (blue).

References

  • Selvaraju et al. Grad-CAM: Visual explanations from deep networks via gradient-based localization. ICCV, 2017.
  • Draelos & Carin. Use HiResCAM instead of Grad-CAM for faithful explanations of convolutional neural networks. arXiv:2011.08891, 2020.
  • Simonyan et al. Deep inside convolutional networks: Visualising image classification models and saliency maps. ICLR Workshop, 2014.
AuthorsOpenReview ↗

Response to Reviewer 5WM2 (Part 3/4)

W3. "The grounding gains may reflect optimization toward the metric. The abstract concedes the gap is closed 'under the same readout that exposed it.' The evidence loss, the Energy-PG metric, and the V verdict [...] are all built from this readout, so the student's in-box share rises almost by construction. Consistently, at λ=0.2 Energy-PG reaches 33.4 while accuracy drops (Tab. F2)."

The evaluations that do not use the readout are in Part 2/4.

The in-box share does not rise by construction: it follows the teacher's evidence in both directions. In the OPD setting, the loss never uses the box. It matches the student's evidence to the teacher's, while Energy-PG (Wang et al., 2020; applied to MLLMs by Gong et al., 2026) compares the evidence with a human annotation. The share rises by 12.7–13.9 points on the questions where the teacher's evidence hits the box and by only 2.9–3.7 where it does not (Table G2). It falls to 21.84, below the base model's 24.70, when the self-teacher is given a random box (Table 4 of the paper).

  • λ = 0.2. As λ grows, Energy-PG approaches the teacher's 33.82 without exceeding it (now stated in Appendix F, blue). A larger weight has little evidence left to transfer, while the next-token loss, on which accuracy depends, loses relative weight.
  • The V verdict is the only one of the four verdicts that uses the readout. Its counterpart without the readout, the region share of Table R2 (Part 2/4), gives the same picture as V in Fig. 7, and Appendix H now says so (blue).
  • The abstract now ends: "... and narrows the evidence gap, under the readout that exposed it and under a region-deletion test that does not use it." (blue)

References

  • Wang et al. Score-CAM: Score-weighted visual explanations for convolutional neural networks. CVPR Workshops, 2020.
  • Gong et al. Saliency-R1: Enforcing interpretable and faithful vision-language reasoning via saliency-map alignment reward. CVPR, 2026.
Reviewer_9p5JOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper investigates whether multimodal on-policy distillation transfers not only a teacher’s predictions but also the visual evidence underlying them. It identifies an evidence gap, where the student outputs become similar to the teacher’s while relying on different image regions, and proposes SWIS. SWIS represents visual evidence as a joint distribution over response-token and image-patch pairs. By aligning this distribution, it supervises both how much visual evidence each token uses and where that evidence is located.

Critical strengths
  • The evaluation covers ten benchmarks, two model sizes, external-teacher and self-teacher settings, and multiple OPD baselines. The reported improvements are relatively consistent across benchmarks rather than concentrated in one dataset.

  • The paper separately tests mass and placement, joint versus separate losses, alternative attention/evidence readouts, teacher privileges, and loss weights. These experiments support the individual design choices.

Critical weaknesses
  • The broad idea and motivation of output-only distillation lacking, and thus transferring teacher attention has been already well established in prior works. The paper mentions prior attention distillation works such as RAL, CompoDistill, its novelty appears more narrowly attributable to the contribution-based joint evidence distribution and its mass/placement decomposition.

  • Prior work TRAG [1] seem to have a similar motivation for vision-language attention distillation, where it argues that response tokens exhibit different visual-grounding patterns and require token-adaptive attention distillation. Could the authors discuss regarding this and possibly compare with an entropy-adaptive TRAG-style attention baseline?

  • The method reuses the teacher and student passes already required by OPD, avoiding an additional model forward or backward pass. However, the paper does not report the memory, throughput, or wall-clock cost of retaining and processing attention weights and value states. For example, does such method hinder the use of flash-attention and thus require more compute cost?

Questions for authors

see weaknesses above.

6 comments
AuthorsOpenReview ↗

Response to Reviewer 9p5J (Part 1/3)

W1. "The broad idea and motivation of output-only distillation lacking, and thus transferring teacher attention has been already well established in prior works. [...] its novelty appears more narrowly attributable to the contribution-based joint evidence distribution and its mass/placement decomposition."

The motivation is shared with prior work, and the paper credits RAL and CompoDistill for it (lines 070, 117, 308; Table B1). What we claim is the part the reviewer names: the contribution-based evidence and the joint distribution with its mass/placement decomposition.

In OPD, transferring attention gains at most 0.82 points and leaves most of the evidence gap open. The contribution-based evidence and the joint distribution together gain 1.90.

(1) Attention is one factor of the evidence. Eq. (2) multiplies the attention weight by what the attended patch adds to the token's logit, so a student can match the teacher's attention while what it reads does not support the token (Fig. C1).

(2) Direct comparison. We add attention distillation to vanilla OPD in two forms: the per-token losses of RAL and TRAG with their original settings (Part 2/3), and attention inside our joint loss. We also take our own loss apart. Drop and Region share use no readout: they come from deleting the annotated region on 2,951 held-out examples (accuracy lost; share of the log-likelihood loss that falls on the region).

Table R1: Attention and evidence distillation added to vanilla OPD (Qwen3.5-4B student, 9B teacher). Evidence: Eq. (2) instead of attention. Joint: one token-patch distribution instead of per-token maps. Teacher-weighted: placement weighted by the teacher's evidence share, no mass term.

MethodEvidenceJointAvg.Energy-PGDrop w/o regionRegion share (%)
Vanilla OPD––72.3927.0915.9632.1
+ RAL✗✗72.27 (−0.12)27.5217.4234.5
+ TRAG✗✗72.71 (+0.32)28.3420.3736.2
+ Attention, last layer✗✓72.96 (+0.57)28.4320.4037.7
+ Attention, all layers✗✓73.21 (+0.82)29.0222.1338.5
+ Placement channel✓✗72.74 (+0.35)30.3724.9741.7
+ Placement, teacher-weighted✓✗73.12 (+0.73)30.9826.5344.3
+ Both, as two losses✓✗73.67 (+1.28)31.4627.7544.2
+ SWIS (Ours)✓✓74.29 (+1.90)32.5030.5748.6

Attention transfer alone. RAL and TRAG reach their own objective: the attention divergence to the teacher falls by 43% and 51%. The student changes little: accuracy moves by −0.12 and +0.32 points, and deleting the region costs 17.42 and 20.37 accuracy points, against 15.96 for vanilla OPD and 30.57 for SWIS.

The evidence and the joint distribution. They act together, and neither is obtained by transferring attention. Inside the same joint loss, replacing attention by Eq. (2) raises the gain from +0.82 to +1.90. Matched token by token, the evidence gains +0.35, about as much as TRAG; weighted by the teacher's evidence share it gains +0.73, with the mass added as a second loss +1.28, and with mass and placement in one divergence +1.90.

Revision. Table R1 and this analysis are in Appendix J.5, and Sec. 3.3.3 refers to the comparison (lines 314–315, blue).

AuthorsOpenReview ↗

Response to Reviewer 9p5J (Part 2/3)

W2. "Prior work TRAG [1] seem to have a similar motivation for vision-language attention distillation, where it argues that response tokens exhibit different visual-grounding patterns and require token-adaptive attention distillation. Could the authors discuss regarding this and possibly compare with an entropy-adaptive TRAG-style attention baseline?"

We take [1] to be TRAG (Jang et al., ECCV 2026). TRAG shows that response tokens differ in how they use the image and that the response-to-vision attention is the place to supervise. SWIS shares this observation and changes two things: what is matched on these tokens (evidence instead of attention) and how the tokens are weighted (by how much each draws from the image). We also ran the entropy-adaptive TRAG loss as a baseline: it gains 0.32 points in OPD, against 1.90 for SWIS.

Token-adaptive in two different senses. TRAG adapts the divergence within each token. Every token still enters the loss with equal weight, and each map is normalized per token, so how much a token draws from the image is not a target. In SWIS this quantity is the mass term of the loss, and each token's placement term is weighted by the teacher's evidence share on that token (Prop. 1). On the held-out split, 6.8% of the tokens hold 60% of the teacher's evidence (Fig. 3a), so the constraint concentrates on these tokens.

The TRAG baseline. TRAG trains its student with teacher forcing on ground-truth answers and names on-policy distillation as future work (its Appendix H). The port follows the original: response-to-vision attention averaged over heads and over the softmax-attention layers at relative depth 0.3–0.6 (layers 11, 15 and 19 of Qwen3.5-4B), and the entropy-based mix of the two KL directions with running entropy bounds.

Result. The TRAG loss works as intended: the attention divergence of the student to the teacher falls by 51% during training. The student itself changes little: average accuracy rises by 0.32 points to 72.71, against +1.90 for SWIS, and deleting the annotated region costs the TRAG student 20.37 accuracy points, against 15.96 for vanilla OPD and 30.57 for SWIS. When an already trained MLLM is post-trained, aligning the response-to-vision attention moves the student much less than aligning the evidence.

Revision. TRAG is added to Sec. 2 (lines 117–118) and to Appendix B.3 (Table B1), and the comparison is in Appendix J.5 (blue).

Reference

  • Jang et al. Token-level response-visual attention guidance for multimodal LLMs knowledge distillation. ECCV, 2026.
AuthorsOpenReview ↗

Response to Reviewer 9p5J (Part 3/3)

W3. "[...] the paper does not report the memory, throughput, or wall-clock cost of retaining and processing attention weights and value states. For example, does such method hinder the use of flash-attention and thus require more compute cost?"

SWIS adds about 4% to the wall-clock time (throughput about 4% lower) and 1.4% to peak memory. It does not hinder FlashAttention, which stays enabled.

FlashAttention. The student and the teacher keep FlashAttention in both runs, and the readout does not take attention weights from it. Instead, a side path in the eight softmax-attention layers recomputes the query, key and value projections and evaluates the attention only from the response tokens to the image patches, in chunks, so no attention matrix over the full sequence is formed or stored. This path runs without gradients on the teacher and with gradient checkpointing on the student.

We run one epoch of vanilla OPD and of SWIS on 8 H200 GPUs, with the Qwen3.5-4B student and the 9B teacher.

Table R2: Training cost of one epoch (Qwen3.5-4B, OPD, 8 H200 GPUs).

MethodTime (h)Peak memory / GPU (GB)
Vanilla OPD2.6124.2
SWIS (Ours)2.7 (+4%)126.0 (+1.4%)

Wall-clock time rises from 2.6 h to 2.7 h (+4%). Both runs take the same number of updates, so throughput falls by the same share. Peak memory rises from 124.2 to 126.0 GB per GPU (+1.4%), which the chunked evaluation and the gradient checkpointing keep low. Inference is unchanged.

Revision. Table R2 and the description of the side path are in Appendix D.2 (blue).

Reviewer_yNSBOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

On-policy distillation trains a multimodal student on its own rollouts with the teacher's next-token distribution. This paper argues that matching those tokens leaves an evidence gap: the student's visual evidence does not move onto the region the teacher uses. SWIS adds a KL term on a joint token–patch evidence distribution, read out from all softmax-attention layers as the rectified contribution of each patch to each token logit. Across multiple benchmarks, SWIS raises average accuracy by 1.8 points over its vanilla recipe and also raises the share of evidence inside the annotated region.

Critical strengths
  • The method is simple, and the paper is easy to read and follow.
  • The evaluation covers multiple benchmarks at two scales, with ablations and several evidence analyses and visualizations.
Critical weaknesses
  1. The claim that next-token feedback supervises what the student says and not what it sees is too strong. The reverse KL depends on the logits, and those logits depend on attention, so the output loss does send gradient into the visual attention.
  2. SWIS largely combines on-policy next-token distillation with attention distillation, which prior work already studies, such as Align-KD, CompoDistill and TRAG. This makes SWIS less novelty.
  3. Every student and teacher is Qwen3.5. The readout sums only the softmax-attention layers of this hybrid backbone, and the Gated DeltaNet layers are excluded from both the diagnosis and the loss. There is no result on a standard Transformer or on another hybrid model.
  4. The paper does not measure the time or memory cost of the evidence loss.
Questions for authors

Please see the weaknesses above.

7 comments
AuthorsOpenReview ↗

Response to Reviewer yNSB (Part 1/4)

W1. "The claim that next-token feedback supervises what the student says and not what it sees is too strong. The reverse KL depends on the logits, and those logits depend on attention, so the output loss does send gradient into the visual attention."

This sentence states the motivation of the paper. Similar observations are reported by VA-OPD (Liu et al., 2026), RAL (Li et al., 2026a), TRAG (Jang et al., 2026) and CompoDistill (Kim et al., 2026). The next-token loss does send gradient into the visual attention, but only through the output distribution, so a token that already matches the teacher receives no further correction of its evidence. Three results in the paper show that the evidence stays in place under next-token matching.

The next-token loss reaches the visual attention only through the output distribution. Specifically, the gradient of LoutL_{out} with respect to a token's logits is pS⊙(log⁡pS−log⁡pT−KL(pS∥pT))p_S \odot (\log p_S - \log p_T - \mathrm{KL}(p_S \Vert p_T)), and every layer upstream receives its signal through this factor. The factor vanishes once the token matches the teacher, whatever evidence produced it. A matched token therefore receives no further correction, even if it draws little from the image or draws from the wrong region.

Evidence in the paper. Three results show that the evidence stays in place under next-token matching.

  1. Over training, the next-token divergence of vanilla OPD falls by 55%, while its evidence divergence KL(QT∥QS)\mathrm{KL}(Q_T \Vert Q_S) falls by 3%, against 28% with the evidence loss (Fig. 2, Table G1).
  2. On the load-bearing tokens whose next-token distribution already matches the teacher's, 21.6% of the evidence of vanilla OPD lies inside the annotated region, compared with 20.1% for the base model, 38.7% for SWIS and 45.2% for the teacher (Figs. 3c and G1c).
  3. The evidence maps show the same. In Fig. 3d, on two tokens whose distributions already match, the teacher puts 67% and 75% of the evidence inside the box and the student 18% and 6%. In Fig. 8, vanilla OPD reaches the teacher's answer while its evidence falls on the arches instead of the lamps.

Revision. Sec. 1 cites the four works and states this point in one sentence (lines 061–064), and the gradient is derived in Appendix B.4 (blue).

References

  • Liu et al. Visual-advantage on-policy distillation for vision-language models. arXiv:2605.21924, 2026.
  • Li et al. Reinforced attention learning. arXiv:2602.04884, 2026.
  • Jang et al. Token-level response-visual attention guidance for multimodal LLMs knowledge distillation. ECCV, 2026.
  • Kim et al. CompoDistill: Attention distillation for compositional reasoning in multimodal LLMs. ICLR, 2026.
AuthorsOpenReview ↗

Response to Reviewer yNSB (Part 2/4)

W2. "SWIS largely combines on-policy next-token distillation with attention distillation, which prior work already studies, such as Align-KD, CompoDistill and TRAG."

Align-KD, CompoDistill and TRAG distill attention maps: the student matches where the teacher's tokens attend over the image (prompt tokens in Align-KD and CompoDistill, response tokens in TRAG). SWIS differs in what is matched and in how the tokens are weighted. With attention as the target, the same OPD pipeline gains at most 0.82 points, against 1.90 for SWIS.

  • What is matched. Eq. (2) multiplies the attention weight by what the attended patch adds to the token's logit, so it records whether what a token reads supports that token (Fig. C1).
  • How the tokens are weighted. SWIS matches one joint distribution over tokens and patches, so the constraint concentrates on the tokens that read the image (Prop. 1).

Comparison. We add the attention losses of TRAG, the response-level method among the three, and of RAL (Li et al., 2026a; the closest work in our setting) to vanilla OPD on the student's own rollouts, the on-policy setting that TRAG lists as future work (its Appendix H). Both keep their original design and loss weight, and everything else follows Table 1 of the paper. Drop and Region share use no readout: they come from deleting the annotated region on 2,951 held-out examples (accuracy lost; log-likelihood lost without the region relative to that lost without the image).

Table R1: Attention and evidence distillation added to vanilla OPD (Qwen3.5-4B student, 9B teacher). Evidence: Eq. (2) instead of attention. Joint: one token-patch distribution instead of per-token maps. Teacher-weighted: placement weighted by the teacher's evidence share, no mass term. †/‡: Tables 3/2 of the paper.

MethodEvidenceJointAvg.Energy-PGDrop w/o regionRegion share (%)
Vanilla OPD––72.3927.0915.9632.1
+ RAL✗✗72.27 (−0.12)27.5217.4234.5
+ TRAG✗✗72.71 (+0.32)28.3420.3736.2
+ Attention, last layer†✗✓72.96 (+0.57)28.4320.4037.7
+ Attention, all layers†✗✓73.21 (+0.82)29.0222.1338.5
+ Placement channel‡✓✗72.74 (+0.35)30.3724.9741.7
+ Placement, teacher-weighted✓✗73.12 (+0.73)30.9826.5344.3
+ Both, as two losses‡✓✗73.67 (+1.28)31.4627.7544.2
+ SWIS (Ours)✓✓74.29 (+1.90)32.5030.5748.6
  1. Aligning attention changes the student little. The attention divergence to the teacher falls by 43% (RAL) and 51% (TRAG), yet accuracy moves by −0.12 and +0.32 points, and the drop without the region rises from 15.96 to 17.42 and 20.37, against 30.57 for SWIS.
  2. Neither change alone gives the gain of SWIS. Attention inside our joint loss gains at most +0.82, and evidence without the joint distribution +0.35 to +1.28.
  3. Together they give +1.90, so the gain of SWIS does not come from adding attention distillation to OPD.

Revision. Align-KD and TRAG are added to Sec. 2 (lines 117–118) and Table B1, and Table R1 with this analysis is in Appendix J.5 (blue). CompoDistill and RAL are already discussed in the paper (lines 070, 117, 308).

Reference

  • Feng et al. Align-KD: Distilling cross-modal alignment knowledge for mobile vision-language large model enhancement. CVPR, 2025.
AuthorsOpenReview ↗

Response to Reviewer yNSB (Part 3/4)

W3. "Every student and teacher is Qwen3.5. The readout sums only the softmax-attention layers of this hybrid backbone, and the Gated DeltaNet layers are excluded from both the diagnosis and the loss. There is no result on a standard Transformer or on another hybrid model."

The paper discusses both points in Appendix I (lines 1402–1408). We now add a result on a standard Transformer and a test on Qwen3.5 that involves every layer.

On Qwen3-VL, where every layer uses softmax attention, SWIS gains 1.45 points over vanilla OPD. On Qwen3.5, the loss also trains the Gated DeltaNet layers, and in a test through every layer that uses no readout, removing the annotated region costs the SWIS student 30.57 accuracy points, against 15.96 for vanilla OPD.

A standard Transformer. We repeat the OPD experiment with a Qwen3-VL-2B student and a Qwen3-VL-8B teacher trained with GRPO.

Table R2: Qwen3-VL-2B student distilled from a Qwen3-VL-8B teacher (OPD).

ModelMathHalluc.GeneralAvg.Energy-PG
Base Model37.0173.7751.2951.5023.40
Vanilla OPD35.9374.9150.7251.1225.12
SWIS (Ours)37.9275.0252.3752.5729.24

SWIS improves the average by +1.45 points over vanilla OPD, against +1.05 on Qwen3.5-2B, so the gain does not depend on the hybrid design.

The Gated DeltaNet layers. These layers form no attention weights over patches, so they cannot be read out. They are not excluded from the loss: the evidence at a softmax-attention layer is computed from the outputs of the layers below it, so the gradient of the evidence loss passes through the Gated DeltaNet layers. To check the result with a test that involves every layer and uses no readout, we delete the annotated region from the input, and compare with deleting a random region of equal size.

Table R3: Region deletion on 2,951 held-out examples (Qwen3.5-4B, OPD). Drop: accuracy lost. Δ log p: log-likelihood of the model's own response lost per token (nats) without the region. Region share: Δ log p for the region over that for the whole image.

ModelAcc.Drop w/o region ↑Drop w/o randomΔ log p ↑Region share (%) ↑
Teacher (9B)64.4241.882.580.28454.7
Base Model57.4014.941.930.12529.8
Vanilla OPD59.6715.961.760.13832.1
SWIS (Ours)62.9330.572.340.22448.6

Measured through all layers of the hybrid backbone, SWIS loses 30.57 accuracy points without the region, about twice the 15.96 of vanilla OPD, and the share of its image dependence that falls on the region rises from 32.1% to 48.6%. A random region of the same size costs every model at most 2.58 points.

Revision. Table R2 is in Appendix J.4 and Table R3 in Appendix J.3, and Appendix C now states that the evidence loss also trains the Gated DeltaNet layers (lines 960–962, blue).

Comments on the paper

9 comments
AuthorsOpenReview ↗

Response to Reviewer mvH2 (Part 1/5)

W1. "Unquantified benchmark overlap limits the generalization claims. [...] The paper provides neither image/question overlap statistics nor results after decontamination."

Q1. "What exact and near-duplicate image/question overlap exists with each benchmark and the 3K diagnostic split? Please report results after removing overlaps, including image-level separation of diagnostic examples."

Thank you for raising this concern. All methods in Table 1 of the paper are trained on one shared set built from public sources (Visual-CoT, Geometry3K, Vision-OPD-6K), as in prior multimodal OPD work: VA-OPD (Liu et al., 2026), for example, trains all methods on public sets that include Geometry3K and evaluates them on MathVista and MathVerse. As the review notes, a shared set supports the comparison across methods but does not show that the gains persist on unseen images, so we now measure the overlap with each test set and re-score every model without it.

Six of the ten benchmarks share no image or question with the training set, and SWIS gains +1.04 to +3.72 points on them. Without the overlapping samples of the other four, its gain over the vanilla recipe does not shrink in any setting (+1.06 to +2.96 points).

How the overlap is measured. Following prior overlap audits (Radford et al., 2021; Barz & Denzler, 2020), we compare every test image with every training image by file and pixel hashes (exact duplicates) and by perceptual hashes and DINOv2 similarity (near-duplicates). We verify every candidate pair manually and check whether the question is also shared.

Table R1: Overlap between the training set and each test set (number of test samples).

Test setSamplesShared imageSame image & question
MathVerse3,940365 (9.3%)35 (0.9%)
POPE9,000360 (4.0%)0
MathVista1,00038 (3.8%)8 (0.8%)
MMStar1,50014 (0.9%)4 (0.3%)
Other six benchmarks7,68600
Held-out split3,00049 (1.6%)0

The overlap is confined to four benchmarks: 0.9% to 9.3% of their samples share an image with the training set, and at most 0.9% also share the question. The 360 POPE samples, for example, come from 20 COCO photos that appear in training with other questions. The held-out split shares 49 images and no question, and the paired test in Part 3/5 excludes them.

Results without the overlap. We remove every overlapping test sample and re-score all models of Table 1 with the same checkpoints and evaluation settings.

Table R2: Avg. accuracy (%) before and after removing the overlap; gain of SWIS over its vanilla recipe in parentheses. Zero-overlap (6): the six benchmarks without overlap.

MethodOriginalOverlap-freeZero-overlap (6)
Qwen3.5-4B, OPSD
Vanilla OPSD72.7272.6867.66
ViGOS72.9172.8767.78
SWIS (Ours)73.98 (+1.26)73.96 (+1.28)68.83 (+1.17)
Qwen3.5-4B, OPD
Vanilla OPD72.3972.3567.32
VA-OPD73.1073.0667.80
SWIS (Ours)74.29 (+1.90)74.27 (+1.92)69.26 (+1.94)
Qwen3.5-2B, OPSD
Vanilla OPSD61.2261.1753.26
ViGOS62.3362.2854.58
SWIS (Ours)64.15 (+2.93)64.13 (+2.96)56.98 (+3.72)
Qwen3.5-2B, OPD
Vanilla OPD64.9064.8658.59
VA-OPD65.2965.2558.93
SWIS (Ours)65.95 (+1.05)65.92 (+1.06)59.63 (+1.04)

The gain does not depend on the overlapping samples. Without them, SWIS gains +1.28, +1.92, +2.96 and +1.06 points over its vanilla recipe in the four settings, and +1.04 to +3.72 points on the six benchmarks with no overlap at all. Its margin over the strongest baseline is 0.67–1.85 points without the overlap.

Revision. Tables R1 and R2 and this analysis are in Appendix J.1 (blue), and the sentence quoted from Appendix D now refers to it.

References

  • Radford et al. Learning transferable visual models from natural language supervision. ICML, 2021.
  • Barz & Denzler. Do we train on test data? Purging CIFAR of near-duplicates. Journal of Imaging, 2020.
  • Liu et al. Visual-advantage on-policy distillation for vision-language models. arXiv:2605.21924, 2026.
AuthorsOpenReview ↗

Response to Reviewer mvH2 (Part 2/5)

W2. "[...] Together with the absence of uncertainty estimates across runs, these limitations leave unclear whether the modest accuracy gains reliably indicate a broader change in visual reliance."

Q4. "Can you provide repeated-seed or paired-bootstrap uncertainty [...]?"

We provide both kinds asked for: repeated seeds for the variation across training runs, and a paired bootstrap for the variation across test questions.

SWIS is above vanilla OPD in every seed, by 1.83–1.90 points, and the 95% interval of its gain over the vanilla recipe excludes zero in all four settings.

(1) Repeated seeds. We repeat the Qwen3.5-4B OPD runs of vanilla OPD and SWIS with two more seeds, three in total (2024, 2025 and 2026). Data, teacher, hyperparameters and evaluation are unchanged.

Table R3: Three seeds (Qwen3.5-4B, OPD). Seed 1 is the run reported in the paper.

MethodSeed 1Seed 2Seed 3Mean ± Std
Avg. accuracy (%)
Vanilla OPD72.3972.0372.5172.31 ± 0.25
SWIS (Ours)74.2973.8674.3674.17 ± 0.27
Energy-PG
Vanilla OPD27.0926.4727.5027.02 ± 0.52
SWIS (Ours)32.5031.8332.7532.36 ± 0.48

SWIS is above vanilla OPD in every seed: by 1.90, 1.83 and 1.85 points in average accuracy and by 5.41, 5.36 and 5.25 points in Energy-PG. The mean gains are about seven and ten times the largest standard deviation across seeds.

(2) Paired bootstrap over the test questions. In all four settings of Table 1 of the paper, we resample the 23,126 questions of the ten benchmarks. This gives the 95% interval of the gain of SWIS in average accuracy, against its vanilla recipe and against the strongest baseline of the setting (ViGOS under OPSD, VA-OPD under OPD).

Table R4: Gain of SWIS in Avg. accuracy with 95% bootstrap intervals.

Settingvs. Vanilla95% CIvs. best baseline95% CI
4B-OPSD+1.26[+0.57, +1.93]+1.07[+0.39, +1.76]
4B-OPD+1.90[+1.22, +2.59]+1.19[+0.52, +1.85]
2B-OPSD+2.93[+2.17, +3.70]+1.82[+1.08, +2.58]
2B-OPD+1.05[+0.31, +1.80]+0.66[−0.07, +1.41]

Against the vanilla recipe, the interval excludes zero in all four settings. Against the strongest baseline it excludes zero in three settings; at 2B-OPD the margin of 0.66 points is within sampling noise, and Sec. 4.2 now says so.

Revision. Tables R3 and R4 are in Appendix J.2, and Sec. 4.2 now reports both results (lines 367–372, blue).

AuthorsOpenReview ↗

Response to Reviewer mvH2 (Part 3/5)

W2. "[...] Figure C2 provides useful validation independent of the readout, although it compares different subsets of newly correct answers."

Q2. "On one fixed held-out example set, does SWIS improve a paired region-deletion or counterfactual-image test relative to vanilla OPD, with uncertainty estimates? Please distinguish relative evidence allocation from absolute visual influence."

We agree that a test independent of the readout is important for this claim, which is why the paper includes the masking check of Appendix C.2. As the reviewer notes, that check compares different subsets, so we now run it as a paired test on one fixed set.

On one fixed set of 2,951 held-out examples, SWIS relies on the annotated region more than vanilla OPD, both in relative allocation and in absolute influence.

Setup. All four models are tested on the same 2,951 held-out examples: the 3,000 of the held-out split without the 49 that share an image with training (Table R1, Part 1/5). Following Appendix C.2, we delete the visual tokens of the annotated region, of a random region of equal size, or of the whole image, and evaluate again; deleting tokens avoids masking artifacts (Jain et al., 2022). The absolute influence of the region is measured by the accuracy drop and by the drop in the log-likelihood of the model's own response (Δ log p). The relative allocation is measured by the region share: Δ log p for the region over that for the whole image.

Table R5: Region deletion on the same 2,951 held-out examples (Qwen3.5-4B, OPD). Drop: accuracy lost. Δ log p: nats per token, without the annotated region.

ModelAcc.Drop w/o region ↑Drop w/o randomΔ log p ↑Region share (%) ↑
Teacher (9B)64.4241.882.580.28454.7
Base Model57.4014.941.930.12529.8
Vanilla OPD59.6715.961.760.13832.1
SWIS (Ours)62.9330.572.340.22448.6
  • Relative allocation. The region share rises from 32.1% for vanilla OPD to 48.6% for SWIS (+16.5 points, 95% CI [14.6, 18.4], paired bootstrap over examples). The teacher's is 54.7%.
  • Absolute influence. Without the region, SWIS loses 30.57 accuracy points, 14.61 more than vanilla OPD (95% CI [12.47, 16.71]), and Δ log p rises from 0.138 to 0.224 nats per token (+0.086 [0.074, 0.098]).
  • Control. A random region of equal size costs every model at most 2.58 points, and SWIS and vanilla OPD do not differ there (0.58 [−0.56, 1.64]). The larger drop is specific to the annotated region.

Vanilla OPD stays close to the base model on every measure, while SWIS covers more than half of the distance to the teacher on each. Because this test perturbs the input and reads the output, it uses no readout, and it covers the Gated DeltaNet layers and the paths through earlier tokens that Eq. (2) leaves out. The Energy-PG gap is also unchanged on these 2,951 examples: 32.53 for SWIS and 27.06 for vanilla OPD, against 32.50 and 27.09 on the full split.

Revision. Table R5 and its protocol are in Appendix J.3 (blue).

Reference

  • Jain et al. Missingness bias in model debugging. ICLR, 2022.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.