acceptodds
Under review as a conference paper at ICLR 2027

Evidence-Aligned Multimodal On-Policy Self-Distillation for Fine-Grained Visual Understanding

Abstract

Fine-grained visual understanding requires models to recognize small details within complex images. Multimodal on-policy self-distillation (OPSD) addresses this challenge by using a teacher conditioned on evidence-centered crops to supervise a student conditioned on original images along student-generated trajectories. Ideally, teacher corrections, the distributional changes from the student toward the privileged teacher, should be driven by task-relevant visual evidence. However, the designs that make the teacher effective also introduce other interference. Using a lagged or frozen teacher improves training stability but introduces a model-state gap from the evolving student, while cropping enhances task-relevant evidence but also loses the visual context. These two sources of interference make the teacher corrections not purely rely on the visual evidence. We introduce Evidence-Aligned multimodal on-policy self-Distillation (EAD), which retains the crop-conditioned teacher as the target but constructs a separate evidence reference for weighting the corrections. To exclude the effect of lagged model-state from this reference, EAD measures prediction changes using the current student. To avoid crop-induced context changes, EAD masks the evidence region in the original image while preserving the other visual context. The change from the student's masked-image prediction to its original-image prediction provides a controlled reference for the direction in which the visual evidence shifts the student's prediction. EAD weights each teacher correction by its cosine alignment with the reference, i.e., retaining aligned corrections and downweighting the rest. Retaining only 6% of the supervision mass of dense OPSD, EAD consistently outperforms previous state-of-the-art methods. We also show that with small-scale backbones, EAD achieves competitive performance compared to substantially larger open-weight and closed-source models.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Vision-Language Models, Multimodal Learning, On-Policy Self-Distillation, Visual Reasoning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_6NuHOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces Evidence-Aligned Distillation, a framework designed to improve fine-grained visual understanding. It employs a crop-conditioned EMA teacher to guide a full-image student using prefixes generated by the student itself. Evidence is estimated from differences in the student's predictions between original and evidence-masked images, and distillation weights are assigned based on the positive directional agreement between teacher corrections and this evidence signal. Experiments with 4B and 9B models demonstrate improvements in fine-grained evaluation performance, despite retaining only about 6% of the original unweighted loss mass along the same trajectories.

Critical strengths
  • The approach distinguishes the privileged supervision target from the evidence-based signal used to determine its weight. Ablation studies on model states, visual contrast, normalization, and clipping provide further support for these design choices.
  • EAD outperforms Vision-OPD on fine-grained tasks by a large margin.
Critical weaknesses
  • Equation 8 modifies the selection of supervision signals and the overall magnitude of the distillation loss. Without controls using uniformly scaled dense losses and randomly shuffled weights, it is difficult to determine whether the improvements specifically arise from evidence alignment.

  • EAD achieves relatively modest average improvements over VAD. However, the absence of results across independent training seeds and paired uncertainty estimates makes it unclear whether these improvements over the strongest baseline are consistent and statistically reliable.

Questions for authors
  • Does EAD outperform globally downweighted dense distillation and shuffled EAD weights? These controls would separate selection from loss attenuation.

  • How stable is the EAD–VAD difference across training seeds? Paired evaluation intervals would also clarify the strength of the ranking.

  • Does the evidence reference remain useful with another masking fill or an irrelevant-region mask? Mean-color occlusion inside the ground-truth box may introduce effects beyond evidence removal.

  • What are the additional training time, memory, and annotation requirements? The 6% loss-mass figure does not measure computational savings.

Reviewer_KXNgOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper addresses fine-grained visual understanding in multimodal large language models (MLLMs), such as answering questions about small details in high-resolution images. It builds on Vision-OPD, a self-distillation method in which the model generates an answer from the full image and is trained to match, token by token, the predictions of a teacher that only sees a crop around the ground-truth evidence region. The teacher's weights are an exponential moving average (EMA) of the student's. The authors argue that the teacher's corrections (the differences between its predictions and the student's) are not only due to the visual evidence: they also reflect the gap between teacher and student weights, and the context lost by cropping. Their method, EAD, keeps this training target but weights the loss of each token by how well the teacher's correction agrees with an evidence reference: the change in the student's own prediction when the evidence region is masked in the full image. The weight is the cosine similarity between the two, set to zero when negative. With Qwen3.5-4B/9B, EAD obtains the best average on six fine-grained benchmarks, ahead of the base model and four other post-training methods retrained by the authors, including the concurrent OPD-V and VAD. It also loses less average accuracy than Vision-OPD on four held-out general benchmarks.

Critical strengths

S1. Simplicity. EAD is simple to implement on top of Vision-OPD: the per-token weights only require one extra forward pass, without gradients.

S2. Clean evaluation protocol. The authors retrain all post-training methods with the same backbone and data for one epoch. They also identify and fix a real bug in the released Vision-OPD answer extractor (Appendix A.3), and re-evaluate the released Vision-OPD checkpoint with the corrected evaluator (Table 6).

S3. Ablations of the main design choices. Each main component of the method is ablated separately, and changing any of them lowers the average score: the model and images used to build the evidence reference, the probability scaling, and the clipping of negative weights (Tables 2–4).

Critical weaknesses

W1. The comparison with the concurrent methods is not conclusive.

  • (a) EAD beats VAD by only +0.63 (4B) and +0.88 (9B) average points, and it is difficult to assess the significance of these gains (see also (b)).
  • (b) The results appear to come from a single training and evaluation run, as is standard in this line of work. In that case, reporting scores with two decimals (e.g. 77.53) suggests more precision than the results have; one decimal would be more appropriate and easier to read.
  • (c) The retrained OPD-V seems weaker than in its own paper: at 4B, it beats Vision-OPD by 2.9 points there but only by 0.2 here, while VAD's margin is reproduced (2.4 vs. 2.2).

W2. The paper does not clearly state how EAD differs from two close concurrent works. OPD-V (arXiv 2608.05131) already masks part of the teacher's crop to weight each token, and VAD (arXiv 2607.28590) already keeps only the part of the teacher's correction that agrees with an evidence direction (the change in the teacher's prediction when the crop is downsampled). The related work covers both in a single shared sentence, and Sec. 5.4 only contrasts EAD's image pair with VAD's. Please state how EAD differs from each (e.g. a reference computed from the student itself, masking the ground-truth region in the full image, weighting each token's loss instead of rebuilding the target as VAD does).

W3. The paper does not show that the teacher's lag harms training. The paper names the lag of the EMA teacher behind the student as one of two sources of "interference" in the teacher's corrections (Abstract, Sec. 1). Fig. 2a only shows that the lag changes which words get the loss, and Table 2 that mixing the two models when computing EAD's weights hurts (75.98 vs. 77.53); neither shows that the part of the corrections due to the lag is harmful. Vision-OPD even finds that a frozen teacher, which has the largest lag, works about as well as the EMA teacher (Yuan et al., 2026, Table 3). Please support this claim, e.g. by comparing EAD's gain over Vision-OPD with a frozen and with an EMA teacher, or present it as a hypothesis.

W4. Undefined terms and notation make the paper hard to follow.

  • (a) Some terms are used before being explained: "evidence restoration" (Fig. 1) and "supervision mass of dense OPSD" (Abstract, Sec. 1; explained only in Sec. 5.3).
  • (b) "Teacher correction" means both a per-token loss (Sec. 3.1, Fig. 3) and a vector of probability differences (Eq. 6).
  • (c) The notation is heavy (e.g. pθ,i,tRp^R_{\theta,i,t}) and not always consistent: t is used before being defined, several letters have two meanings (R, C, B), and variant names change between tables (e.g. "No JS Scaling" in Table 4 vs. "Vanilla Cosine" in Table 9).

Please define each term once, in plain words, where it is first used.

Questions for authors

Q1. Two ablations would test two key differences from OPD-V (W3), namely where to mask and how to turn the masking effect into a weight: (i) masking a random rectangle inside the ground-truth box instead of the whole box; (ii) replacing EAD's weight with OPD-V's per-token score (how much masking lowers the probability of the generated token, zero if it does not). Could you run them?

Q2. Since the evidence reference is computed by the student, it reflects the student's own reading of the image. When the student misreads a detail (e.g. "black" for a blue object), the reference can point against the teacher's correction, which then gets low or zero weight (Eq. 7), although this is exactly the error to fix. Could you report the weight on answer tokens separately for correct and wrong training answers (about 36–40% are wrong, Table 7)?

Q3. Sec. 5.1 fixes the first generated token to "The" to avoid a collapse toward answers that only give the option letter. Why does this collapse arise with EAD, given that the released Vision-OPD code does not use such a fix? Was it also used for the retrained baselines?

Q4. The retrained OPD-V gains much less over Vision-OPD than in its own paper (W2c). Could you explain this gap, or evaluate OPD-V's public Qwen3.5-4B/9B checkpoints, as done for Vision-OPD in Table 6?

Q5. The evidence reference needs an extra forward pass on the full-resolution masked image. How much does this add to training time compared with Vision-OPD and VAD? Both VAD and OPD-V report such numbers.

Q6. Sec. 1 concludes that OPSD "is most effective when teacher corrections are selectively retained according to their alignment with privileged evidence". Yet every variant in Tables 2–4, even the weakest (Original–Crop, 75.16), beats Vision-OPD (74.70 at 4B), so part of the gain may not depend on which tokens are kept. Could you add a random baseline, e.g. EAD's weights randomly shuffled among the tokens of each answer?

Q7. Minor:

  • (a) In Table 1, the base Qwen3.5-9B already scores 74.84, above GPT-5.2/5.4 and most of the listed models, so "competitive with substantially larger models" mostly reflects the strength of the backbone. Please rephrase the claim.
  • (b) Figs. 2 and 4 are hard to read at print size.
  • (c) Table captions almost touch the tables (e.g. Tables 1–3 and 5). Please restore the template's default spacing.
Reviewer_J154OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper builds on Vision-OPD, where a slowly updated copy of the model sees a zoomed crop of the key region and, word by word, teaches the normal model that sees the full image. It argues the teacher's corrections are partly noise, because the teacher has older weights and the crop removes the surroundings. EAD keeps the same teacher and loss but weights each word by how well the teacher's correction agrees with what the current model's prediction changes when the key region is painted over, reporting the best average on six fine-grained benchmarks at 4B and 9B.

Critical strengths
  1. A useful diagnosis: the paper measures that the older teacher weights alone shift which words get supervised by 25.3% after 25 steps, and that zooming and masking emphasize different words by 42–44%.
  2. Consistent gains: EAD improves over Vision-OPD by 2.8 points at 4B and 1.3 at 9B, has the best average at both sizes, and keeps general abilities close to the base model.
Critical weaknesses
  1. The only change is a hand-designed per-word weight, and its benefit is not separated from simply training less. EAD keeps only about 6% of Vision-OPD's loss while using the same learning rate and one epoch, so it trains roughly 16 times more weakly and concentrates 96% of its loss on the top 10% of words. Smaller updates alone could explain both the better fine-grained scores and the smaller loss of general abilities. There is no control with the dense loss scaled down to the same size, random word weights of the same total, or keeping only the highest-loss words.
  2. The noise the paper diagnoses is not removed from what the student learns. EAD only decides how much each word counts; on the words it keeps, the student still learns the full crop-teacher prediction, including the parts caused by the teacher's older weights and by the crop losing the surroundings. The paper motivates EAD by these two sources of noise, but never measures how much of them remains in the corrections it keeps. Its ablations only change which signal sets the weight, not the target the student learns toward.
Questions for authors
  1. Does EAD still beat Vision-OPD when Vision-OPD's loss is scaled down to the same 6% of supervision, or when words are weighted randomly with the same total weight, which would show whether the evidence-based weight itself matters?
  2. Do the training images contain a red box around the evidence that the test images lack, as the description of the masked image suggests, and if so how does this affect what the model learns?
  3. How large are the older-weights and the zoom-and-lost-surroundings parts of the teacher's correction on the words EAD keeps, for example measured with the current model on the crop and on the full image with only the box drawn?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.