acceptodds
Under review as a conference paper at ICLR 2027

PReViD: Interventional Prefix Replay for Visual On-Policy Distillation

Abstract

Visual distillation uses a teacher with better visual evidence to supervise responses sampled by a student model, but early visual mistakes remain in the response history and distort later supervision. We identify this missing pathway by showing that corrected visual evidence can change later predictions through the teacher's language context. Based on this finding, we introduce PReViD, which replays the same student response under controlled image and evidence contexts while preserving alignment with the original response. PReViD separates visual and contextual corrections, uses a placebo prefix with no visual evidence to filter generic text effects, and gives more weight to context changes that support the teacher's correction. Across benchmarks for detailed visual understanding, model sizes, and teacher settings, PReViD outperforms strong distillation baselines without changing inference. Code is available at https://anonymous.4open.science/r/E02D2.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Vision-Language Model, On-Policy Distillation, Evidence Prefix

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_CsGHOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

PReViD prevents error propagation in visual on-policy distillation by using contrastive, crop-based evidence prefixes to clean up the teacher's token targets. The authors claim to outperform standard visual OPD baselines across multiple high-res vision benchmarks with almost no training overhead.

Critical strengths
  1. Low cost and solid controls. PReViD has zero inference footprint and negligible training overhead, with steady gains across all benchmarks and model scales. The random gate and shuffled prefix baselines are well-chosen and cleanly demonstrate that informative prefix content is what actually moves the needle.

  2. Clean, transferable design. Inserting the prefix as an unfinished assistant turn preserves rollout alignment, while the placebo direction replaces arbitrary thresholding with a grounded baseline. The authors are also refreshingly honest that these are empirical input contrasts rather than causal claims, making the framework easily reusable for other privileged-context distillation problems.

Critical weaknesses
  1. Underperforming baselines vs. published results: The re-implemented baselines (Vision-OPD, VAD) score noticeably below their officially published numbers, to the point where PReViD actually trails the original reported figures. Because the official baseline checkpoints are publicly available, the authors should simply run those released weights on their own harness to clarify whether this gap comes from evaluation differences or undertrained baselines.

  2. The decomposition order artificially inflates the prefix path: Measuring the image step dtxd^x_t with a degraded text prefix e−e^- already in context introduces a huge confound. Because models naturally defer to text over visual evidence under conflict, the visual contrast gets suppressed, handing the remaining 76–81% variance to the prefix by construction. Standard OPD runs without any prefix ((x−,∅)→(x+,∅)(x^-, \varnothing) \to (x^+, \varnothing)), yet this baseline and the reversed path order are omitted entirely. The label flips in Appendix C confirm that this "pathway" is mostly just the familiar text-override effect rather than a newly isolated supervision channel. There is also zero visibility into what e−e^- actually says (whether it's actively wrong or just vague).

  3. Novelty framing & missing composite baseline: The core concept of feeding privileged text hints to the teacher is already established by π-Distill (ICLR 2026). The authors should reframe their contribution as merging OPSD's textual hints with Vision-OPD's visual crops, and run an "OPSD + crop" baseline to isolate the gain from their gated decomposition. Also, verify the Section 2.2 claim about Masking-KD, as it already intervenes on teacher-side prefixes.

  4. Missing basic hyperparameters and reproducibility details: Section 4.1 points to Appendix A for training setup, but the appendix is missing nearly all of it. There are no learning rates, batch sizes, optimizers, rollout settings, or epoch counts for Table 1, nor any clarification on whether tuning is full-parameter or LoRA. Several core variables (λ,M,ε\lambda, M, \varepsilon) and the exact implementation of the importance weight wtw_t (clipped or standard ratio?) are also completely absent.

Questions for authors
  1. Is the scale-matched teacher simply a frozen base model (per Appendix A.2), or was it trained on privileged information (Sections 4.2)?
  2. Are the 70.24 MME-CN entries in Table 1 and Table 2 distinct runs?
4 comments

Thank you for your detailed review. We appreciate your comments “Low cost and solid controls” and “Clean, transferable design”. Below, we respond to your concerns and questions.

W1 on baseline results and reproduction settings

Section 4.1 of the original paper specifies the same training data, student initialization at each model size, and evaluation protocol for the trained methods. Appendix A.1 lists the supervision inputs for each method. Table 1 reports results under this shared setup. Each method retains its own supervision inputs and target construction.

Our shared training setup may differ from the full training procedures in the baseline papers. To check the sources of the reported results, we evaluated the official weights under our evaluation configuration. The table below compares results from the baseline papers, official weights, and models trained under our shared setup. “Retrained under our shared setup” refers to models trained for the original paper. For this revision, we retained those weights and added two inference evaluations with new seeds.

The means and standard deviations for official weights and our retrained models each come from three inference evaluations with fixed weights and different seeds. The supplement provides the three inference evaluations for Vision-OPD and VAD. Section 4.1 of the revised paper states this repeated inference protocol. Table 1 has been updated accordingly.

The official Vision-OPD weights give results closer to those reported in the VAD paper. For the 9B model, the original Vision-OPD paper and the VAD paper report average scores of 79.68 and 76.88, respectively. Under our evaluation configuration, the official weights score 77.89±0.18. Retraining under our shared setup gives 76.03±0.15. The original VAD paper reports an average score of 79.93. Its official weights score 78.03±0.06 under our evaluation configuration. Retraining under our shared setup gives 77.37±0.15. These differences may arise from training configurations, inference implementations, and hardware environments. Vision-OPD and VAD have clear motivations and simple supervision procedures that offer advantages in training cost. We use their evaluation benchmarks to compare the methods. PReViD achieves higher average performance in the comparison under shared training settings. It also adds evidence prefix generation and four teacher replays. These methods reflect a tradeoff between performance and training cost.

Method and result sourceV*ZoomHR4KHR8KMME-ENMME-CNAvg
Qwen3.5-4B
Vision-OPD (original paper)92.1559.7684.5080.3874.8870.7677.07
Vision-OPD (VAD paper)89.5359.4181.7580.1274.5170.1775.92
Vision-OPD (official weights)89.70±0.8059.57±0.0781.63±0.3378.25±0.7872.87±0.1569.93±0.2375.32±0.33
Vision-OPD (retrained under our shared setup)87.26±0.3053.41±0.1479.88±0.6678.00±0.7667.19±0.1168.08±0.1872.30±0.27
VAD (original paper)92.1560.5985.3883.3876.9771.4778.32
VAD (official weights)93.72±0.0061.38±0.2785.50±0.1381.42±0.1972.53±0.0671.43±0.3177.66±0.06
VAD (retrained under our shared setup)90.40±0.6153.06±0.3077.92±0.8375.79±0.4769.93±0.1067.81±0.1172.48±0.33
PReViD (ours)90.93±0.3061.42±0.1284.50±0.4581.13±0.7671.89±0.0470.26±0.0876.69±0.13
Qwen3.5-9B
Vision-OPD (original paper)94.7665.8088.1385.5073.4070.4679.68
Vision-OPD (VAD paper)92.7259.2685.1383.7571.7668.6576.88
Vision-OPD (official weights)90.58±0.5264.62±0.4185.46±0.3184.33±0.6271.95±0.0670.38±0.0677.89±0.18
Vision-OPD (retrained under our shared setup)90.75±0.3058.70±0.3184.67±0.2682.54±0.7571.30±0.0468.20±0.1176.03±0.15
VAD (original paper)94.7662.4987.8885.7576.5672.1379.93
VAD (official weights)94.76±0.0061.18±0.2085.42±0.3882.50±0.2273.19±0.0771.15±0.0578.03±0.06
VAD (retrained under our shared setup)93.02±0.6058.38±0.4985.29±0.2685.63±0.1372.28±0.0269.64±0.2777.37±0.15
PReViD (ours)93.54±0.3064.34±0.1886.13±0.1384.84±0.1974.38±0.0570.42±0.0478.94±0.06

W2 on path order and the interpretation of degraded prefixes

Thank you for pointing out the effects of path order and prefix context on the decomposition. Section 3.2 of the original paper defines the two paths as controlled model input contrasts, rather than natural direct or indirect effects. Figure 2 and Appendix B report the 76%/81% results for the image first diagnostic. Appendix B also gives the score definition and subset selection rule.

We have extended the original diagnostic with the reverse decomposition and interaction term in Appendix B and Table 11 of the revised paper. This tests the effect of path order. The analysis reuses the cached scores for all four input combinations. All samples and the original fixed joint endpoint subset remain unchanged. Both image conditions use the same saved prefixes. The clear prefix e+e^+ is still generated from the clear image. It is not regenerated when the image is switched. The reverse path is

\qquad \delta'_e=\ell^{-+}-\ell^{--},\quad \delta'_x=\ell^{++}-\ell^{-+}.$$ We retain the log-odds score of the correct option relative to the set of valid incorrect options. We report mean scores and components. The prefix share is the mean prefix component divided by the mean endpoint contrast. |Sample group|Path order|Samples|Mean endpoint contrast|Mean image component|Mean prefix component|Prefix share| |---|---|---|---|---|---|---| |All samples|Image first|6241|3.444|0.820|2.623|76.179%| |All samples|Prefix first|6241|3.444|0.477|2.967|86.151%| |Original joint endpoint subset|Image first|2077|11.686|2.168|9.518|81.448%| |Original joint endpoint subset|Prefix first|2077|11.686|1.155|10.531|90.114%| The interaction term measures the difference between components under the two orders. Here, $\delta_e=\ell^{++}-\ell^{+-}$ and $\delta_x=\ell^{+-}-\ell^{--}$. $$I=\ell^{++}-\ell^{+-}-\ell^{-+}+\ell^{--} =\delta_e-\delta'_e =\delta'_x-\delta_x.$$ |Sample group|Mean interaction I| |---|---| |All samples|-0.343| |Original joint endpoint subset|-1.013| The reverse path gives prefix shares of 86.151% and 90.114%, higher than those under the original order. The mean prefix component is positive under both orders. **A large mean prefix component is therefore observed under both path orders.** In both sample groups, the mean image component is larger under degraded prefixes than under clear prefixes. Both mean interactions are negative. These results indicate that component sizes depend on prefix context. Section 3.1 and Appendix A.2 of the original paper define how clear, degraded, and placebo prefixes are generated. Figure 5 in Appendix B of the revised paper adds [evidence prefix examples](https://anonymous.4open.science/r/E02D2/supplement/replay.pdf) from three samples. These examples pair image inputs with the generated text. Each sample shows the clear evidence input, degraded evidence input, and uniform RGB placebo image, together with their evidence prefixes. The examples show that degraded inputs can produce incorrect observations.

W3 on the method contribution and combined baselines

Thank you for pointing out the connections to prior work. Sections 2.1 and 2.2 of the original paper discuss π-Distill/OPSD, Vision-OPD, and VAD. Section 4.1 and Appendix A.1 specify that OPSD uses answer labels as textual privileged information. Section 4.4 explains that Vision-OPD follows a training procedure similar to OPSD, replaces textual privileged information with visual information, and supervises the full sequence.

If “OPSD + crop” means replacing answer label text with a clear evidence crop, it corresponds to Vision-OPD in our main table. Both use self-distillation with the current policy. They differ in the form of privileged information. Section 4.1 of the revised paper clarifies this correspondence. Section 4.2 gives a consistent description of the teacher based on the current policy.

Sections 3.2 to 3.4 of the original paper describe PReViD. It replays the same student response in four teacher contexts and selectively rebuilds the target. Student history is fixed. Evidence prefixes appear only in teacher inputs. Two replays pair the clear image with clear or degraded prefixes. The others pair the degraded image with a degraded prefix, and the clear image with a placebo prefix.

The path (x−,e−)→(x+,e−)→(x+,e+)(x^-,e^-)\rightarrow(x^+,e^-)\rightarrow(x^+,e^+) defines image and prefix corrections. We remove the image-aligned component from the prefix direction to avoid duplicate visual corrections. A deterministic gate weights the prefix correction using the placebo direction magnitude. One-sided projection forms rtr_t from components aligned with the endpoint correction. The anchor separately regularizes the student toward pt+,+p_t^{+,+} and is not gated.

We also examined whether the gain can be explained by supplying the teacher with the additional context alone. The original Table 3 in Section 4.4 tests prefix selection through the no gate, random gate, and shuffled prefix controls. We added two objective controls in Section 4.4 of the revised paper, with both model sizes reported in Table 4. Direct endpoint OPD and PReViD (no anchor) were newly trained at 4B and 9B using their respective objectives. Student initialization and the training protocol are shared within each size. For each method, we used the same weights for three inference evaluations with different seeds. Vision-OPD and Ours retain the weights from their original training.

The two controls correspond to the two terms of the original Section 3.4 loss. We compare matching the constructed target with matching the teacher endpoint. The token loss is

ℓt=DJS(pθ,t∥rt)+λDJS(pθ,t∥pt+,+)\ell_t=D_{\mathrm{JS}}(p_{\theta,t}\Vert r_t)+\lambda D_{\mathrm{JS}}(p_{\theta,t}\Vert p_t^{+,+}).

The constructed target is matched in the first term. The anchor in the second term regularizes the student toward the teacher endpoint.

The endpoint distribution pt+,+p_t^{+,+} uses the clear image and clear evidence prefix. Four teacher replays, projection, and gating form the target rtr_t. PReViD (no anchor) removes endpoint matching and retains only the first term. Direct endpoint OPD omits target construction and uses only the second term with unit weight. Full PReViD uses the two terms together. Results from the three inference evaluations for the target construction controls are provided in the supplement.

ModelV*ZoomHR4KHR8KMME-ENMME-CNAvg
Qwen3.5-4B
Vision-OPD87.26±0.3053.41±0.1479.88±0.6678.00±0.7667.19±0.1168.08±0.1872.30±0.27
Direct endpoint OPD88.83±0.3056.73±0.1880.79±0.1979.00±0.3370.73±0.0668.71±0.1974.13±0.08
PReViD (no anchor)90.05±0.5259.57±0.2581.83±0.3179.42±0.5271.04±0.0669.38±0.1275.22±0.17
Ours90.93±0.3061.42±0.1284.50±0.4581.13±0.7671.89±0.0470.26±0.0876.69±0.13
Qwen3.5-9B
Vision-OPD90.75±0.3058.70±0.3184.67±0.2682.54±0.7571.30±0.0468.20±0.1176.03±0.15
Direct endpoint OPD91.80±0.3060.63±0.4985.08±0.2683.67±0.5172.46±0.0669.72±0.2277.23±0.17
PReViD (no anchor)92.50±0.3062.09±0.3885.63±0.3884.50±0.3373.74±0.0770.12±0.1078.09±0.11
Ours93.54±0.3064.34±0.1886.13±0.1384.84±0.1974.38±0.0570.42±0.0478.94±0.06

PReViD improves over Direct endpoint OPD by 2.56 percentage points at 4B and 1.71 at 9B. The no anchor variant also outperforms Direct endpoint OPD. This supports an additional benefit from selective target construction with the same teacher context.

We retain the discussion of these methods and Masking-KD in Sections 2.1 and 2.2 of the original paper. Section 2.1 of the revised paper further explains that OPSD uses the same policy to compute distributions under different information contexts. Masking-KD changes the student prefix. PReViD adds evidence to the teacher context and replays the original response. The new target controls further test the benefit of our target construction.

W4 and Q1 on training parameters and teacher identity

Section 4.1 of the original paper gives the student sizes, initialization, training data, and hardware. Appendices A.1 and A.2 list the supervision inputs and generation procedure. We have added a consolidated hyperparameter table in Appendix A.1 of the revised paper. Table 5 includes the learning rates, optimizer, and update settings.

Section 4.3 of the original paper identifies the 9B PReViD checkpoint from Table 1 as the fixed teacher. However, the descriptions of the teacher parameter state in the main experiments were inconsistent. We apologize for the inconsistent descriptions in the original paper.

In the main PReViD experiments and ablations, the current student policy also provides teacher supervision. The two roles use shared parameters under different contexts. The student receives the full image and its own history; teacher inputs contain controlled images and evidence prefixes. Teacher distributions and reconstructed targets have their gradients stopped. Backpropagation acts only on the student distributions, so the teacher role follows the student's parameter updates. For Table 2, supervision comes from the frozen 9B PReViD checkpoint. The student is initialized from Qwen3.5-4B. Each setting uses its corresponding teacher to generate evidence prefixes. We have corrected the frozen teacher wording for the main experiments. The teacher and generator descriptions are now consistent across Sections 4.1 to 4.3 of the revised paper.

Section 3.4 of the original paper defines wtw_t as the stored-rollout importance weight. Appendix A.1 of the revised paper makes its probability ratio explicit. It is the current student probability of the sampled token divided by its stored rollout probability. We use full fine-tuning and Adam. The original training seed is 42. The additional inference evaluations use different fixed seeds. The table below lists the settings.

ParameterSetting shared by 4B and 9B, except learning rate
Fine-tuningFull fine-tuning
OptimizerAdam
Adam coefficientsβ₁=0.9,β₂=0.999
Original training seed42
Learning rate4B 5×10⁻⁷,9B 2×10⁻⁷
Training epochs2
Global question batch size64
Microbatch size per GPU1
Gradient accumulation4 steps
Weight decay0.01
Warmup ratio0.05
Maximum gradient norm1.0
Student rollouttemperature=1,top-p=1
Maximum prompt/response length8192/4096 tokens
Evidence prefix generationEvery training batch, greedy decoding, at most 64 tokens
Hardware/precision/sharding16 H20 GPUs (96 GB)/BF16/FSDP full shard
Distillation divergence/candidate supportJSD/Top-64 plus remaining probability mass
Anchor weight λ0.1
Target shift clipping limit M20
Projection stability term ε10⁻³
Gate margin/logistic scale0/1

Q2 on the two MME-CN values and repeated inference evaluations

Section 4.3 and Table 2 of the original paper specify a fixed 9B PReViD teacher. This is a different training setup from the main experiments in Table 1, which use a teacher of the same model size. Section 4.2 of the revised paper clarifies that 4B PReViD in Table 1 uses the current student policy to construct teacher supervision with gradients stopped. Table 2 retains the fixed 9B PReViD teacher trained for Table 1. The table below reports three MME-CN inference evaluations for each setting, with the means and sample standard deviations. Section 4.1 of the revised paper describes the repeated inference protocol. Tables 1 and 2 have been updated separately.

Experimental settingOriginal inference evaluationNew seed inference evaluation 1New seed inference evaluation 2Mean and standard deviation
Table 1, 4B self-distillation with the current policy70.2470.3470.1970.26±0.08
Table 2, fixed 9B teacher distilling a 4B student70.2469.8570.3170.13±0.25

Thank you for your detailed comments on baseline comparisons, path interpretation, and reproduction settings. We have added diagnostics and reproduction details to the paper, building on the original shared comparison and method definitions. The supplement also provides results from evaluating the official weights. Your suggestions helped us present the experimental evidence more fully and improve the clarity and reproducibility of the paper.

Reviewer_3gWqOpenReview ↗
Rating
3(+1)
Confidence
1
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This submission presents PReViD, Interventional Prefix Replay for Visual On-Policy Distillation. In visual on-policy distillation, the teacher scores each token conditioned on the student's own history, so an early visual error still shapes every later target. PReViD keeps the student's response unchanged and has the teacher replay it with an added one-sentence evidence prefix. Comparing replays under clear, degraded and placebo image/prefix contexts separates an image-driven correction from a prefix-driven one. The prefix correction is gated against the placebo, and the corrections are combined into a training target for the student. On Qwen3.5-4B and 9B, PReViD achieves the best average accuracy across six fine-grained visual benchmarks. It does so against eight baselines and in a fixed-teacher setting, with no change at inference.

Critical strengths

This submission is well written and easy to follow even for someone without any related backgrounds, like me. The motivation is strong: it begins from a very clear observation and then design the method around this observation. Finally, it shows strong, consistent results. At 4B, gains over baseline are +10.5 on V*, +13.5 on Zoom and +9.2 on MME-EN. PReViD has the best average at both scales and in the fixed-teacher setting, with nothing extra at inference.

Critical weaknesses
  1. The direct baseline is missing: PReViD's teacher sees privileged information the other visual-OPD baselines do not, i.e. a text description e⁺ produced by a separate generator G from the clear crop. Therefore, an obvious baseline is a plain OPD with the teacher conditioned on (x⁺, e⁺), i.e. distilling directly to p^{+,+}. This is the anchor term used alone, with no decomposition, projection or gate. Without it, the gains cannot be split between "the teacher received a helpful textual hint" and "the decomposition and gating help." The no-gate row in Table 3 still includes visual projection, so it does not answer this.
  2. I failed to find the definition of Generator G. The paper calls G a "frozen observation generator" but does not name the model, its size, or its prompt.
Questions for authors

How does plain OPD to the (x⁺, e⁺) teacher compare with PReViD? Which model is used as the generator? What happens if e⁺ is generated by the teacher itself?

The rebuttal answered these questions, so I decide to raise the score.

3 comments

Thank you for reviewing our paper. We appreciate your comments that the paper is “well written and easy to follow” and that “The motivation is strong”. We address your concerns and questions below.

W1 and Q1

Thank you for suggesting this direct comparison. Sections 3.2 to 3.4 of the original paper define the four teacher replays, selective target construction, and anchor term. The original Table 3 in Section 4.4 tests visual projection and prefix selection. It includes the no gate, random gate, and shuffled prefix controls.

In Table 3, Vision-OPD improves over Baseline on average by adding clear visual privileged information. Visual projection and teacher evidence prefixes add further gains. Random gates and shuffled prefixes reduce performance. This supports the role of prefix content and selection. Full PReViD also improves over “all prefix shifts”, which keeps projection and target reconstruction.

Reviewer CsGH also commented on these controls. “The random gate and shuffled prefix baselines are well-chosen and cleanly demonstrate that informative prefix content is what actually moves the needle.”

We added Direct endpoint OPD and PReViD (no anchor) at 4B and 9B to revised Section 4.4 and Table 4. Each control was trained with its own objective and the corresponding student initialization under the shared training protocol. Rows report means and standard deviations from three inference evaluations with fixed weights and different seeds. Vision-OPD and Ours retain their original trained weights.

These controls test the effects of teacher context and target construction. The token loss in Section 3.4 of the original paper is

ℓt=DJS(pθ,t∥rt)+λDJS(pθ,t∥pt+,+)\ell_t=D_{\mathrm{JS}}(p_{\theta,t}\Vert r_t)+\lambda D_{\mathrm{JS}}(p_{\theta,t}\Vert p_t^{+,+}).

The first term fits the constructed target. The second anchors the student to the teacher endpoint.

pt+,+p_t^{+,+} denotes the teacher distribution with the clear image and clear evidence prefix. Four teacher replays, projection, and gating produce rtr_t. PReViD (no anchor) keeps the first term alone. Direct endpoint OPD keeps the second term alone, with a coefficient of 1. Ours combines the two terms. The three inference evaluations for the target construction controls are available in the supplement.

ModelV*ZoomHR4KHR8KMME-ENMME-CNAvg
Qwen3.5-4B
Vision-OPD87.26±0.3053.41±0.1479.88±0.6678.00±0.7667.19±0.1168.08±0.1872.30±0.27
Direct endpoint OPD88.83±0.3056.73±0.1880.79±0.1979.00±0.3370.73±0.0668.71±0.1974.13±0.08
PReViD (no anchor)90.05±0.5259.57±0.2581.83±0.3179.42±0.5271.04±0.0669.38±0.1275.22±0.17
Ours90.93±0.3061.42±0.1284.50±0.4581.13±0.7671.89±0.0470.26±0.0876.69±0.13
Qwen3.5-9B
Vision-OPD90.75±0.3058.70±0.3184.67±0.2682.54±0.7571.30±0.0468.20±0.1176.03±0.15
Direct endpoint OPD91.80±0.3060.63±0.4985.08±0.2683.67±0.5172.46±0.0669.72±0.2277.23±0.17
PReViD (no anchor)92.50±0.3062.09±0.3885.63±0.3884.50±0.3373.74±0.0770.12±0.1078.09±0.11
Ours93.54±0.3064.34±0.1886.13±0.1384.84±0.1974.38±0.0570.42±0.0478.94±0.06

W2

Section 4.1 of the original paper specifies the Qwen3.5-4B and 9B model sizes. Appendix A.2 states that generator G also serves as the replay teacher. Section 4.3 specifies that the fixed teacher is the frozen 9B PReViD checkpoint trained for Table 1. However, our descriptions of the teacher parameter state in the main experiments were inconsistent. We apologize for this lack of clarity.

We have corrected and aligned the teacher and generator descriptions in Sections 4.1 to 4.3 of the revised paper. The main experiments and ablations use a copy of the current Qwen3.5-4B or 9B student policy with gradients stopped. This policy generates both evidence prefixes and replay distributions. The fixed teacher experiments use the frozen Qwen3.5-9B PReViD checkpoint for both roles.

Appendix A.2 of the original paper provides the full prompts and generation procedure. The prompt asks for one sentence of visual observation. It asks G to omit a separate final answer field and to avoid repeating the answer options. G receives the question and a clear, degraded, or uniform RGB placebo image. It receives neither answer labels nor student responses. Generation uses greedy decoding with a limit of 64 tokens. The prefixes are used only for teacher supervision. The student never receives them at inference. Appendix A.3 of the revised paper retains these rules. It also clarifies that all three views share the same prompt and decoding rules, and specifies the clear image input for each teacher setting.

Thank you again for your suggestions. The new target controls and clarified teacher setup build on the original ablations and generation protocol. Your comments helped clarify our contribution and sources of supervision.

Thank you for your reply, it resolves my concerns, I will raise the score.

Thank you very much for your professional review. Your valuable comments have greatly contributed to improving the quality of this manuscript, and we hereby express our sincere gratitude to you.

Reviewer_GSmbOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes PReViD, a visual on-policy distillation method motivated by the observation that a student’s early visual errors remain in its response history and can influence the teacher’s later supervision. PReViD preserves the student’s sampled response while replaying it under teacher contexts that vary the visual input and a generated evidence prefix. It constructs a selective training target from the resulting image- and prefix-related changes, using a placebo prefix to modulate the prefix contribution and an additional anchor to the privileged teacher. The auxiliary contexts are used only during training, leaving the student’s inference procedure unchanged. Experiments cover six fine-grained visual benchmarks, Qwen3.5 models at two scales, a fixed-teacher setting, and component ablations. An ordered-path analysis further shows that changing the evidence prefix can substantially affect the teacher’s answer probabilities.

Critical strengths
  1. Sections 3.1–3.4 introduce generated visual evidence before the unchanged student history, allowing the teacher to provide supervision under a different context without replacing the sampled response. This is a focused and potentially useful extension of fixed-history visual distillation. Because the added contexts are teacher-side and training-only, the student’s inference procedure remains unchanged.

  2. Table 1 reports the best average performance at both model scales, with gains on several benchmarks. Table 2 provides an additional fixed-teacher comparison, while Table 3 includes ungated, random-gate, and shuffled-prefix controls. Table 8 reports that the evidence-prefix path accounts for a substantial share of the measured endpoint change under the paper’s specified ordered decomposition. Together, these results support the usefulness of the overall recipe and suggest that example-specific evidence prefixes matter, although they do not fully isolate the contribution of each mechanism.

Critical weaknesses
  1. The contribution of the target-construction mechanism is not sufficiently isolated from the benefit of additional teacher context. PReViD supplies the teacher with an evidence prefix, but the paper does not compare it against ordinary on-policy distillation using the same clear image and clear prefix. The Table 3 ablations test several components, but do not establish whether the gains come primarily from providing this additional information or from PReViD’s replay-based decomposition and selective target construction. This comparison is important for identifying the method’s specific contribution.

  2. The prefix intervention may conflate evidence content with sequence-position effects. The generated prefixes can contain up to 64 tokens and are inserted before the unchanged student response history. If prefix lengths differ across the clear, degraded, and placebo conditions, the positions at which the teacher processes the response tokens may also differ. The paper should clarify whether lengths and positional effects are controlled. Otherwise, the measured changes may reflect not only prefix content but also changes in the teacher’s conditioning sequence and token positions. A length-matched control or an analysis controlling for prefix length would help isolate the semantic contribution of the evidence.

  3. The evidence for robustness of the reported gains is incomplete. Tables 1–3 report point estimates, but the paper does not appear to report variability across independent training runs. The confidence intervals in the path analysis quantify uncertainty across examples, not variation due to training stochasticity. Since the differences between methods are central to the effectiveness claim, results across multiple seeds—or another suitable measure of run-to-run variability—would help establish whether the improvements are robust.

  4. The teacher and prefix-generator protocol needs clarification. The description does not make it sufficiently clear which checkpoint generates the prefixes and replay distributions in each experiment, or whether those parameters remain fixed or are refreshed during training. The descriptions in Sections 4.2 and 4.4 and Appendix A.2 should be clarified. This matters because the teacher’s state determines the supervision available to the student and affects how the results should be interpreted and reproduced.

Questions for authors
  1. How does PReViD compare with ordinary on-policy distillation using the identical clear-image, clear-prefix teacher distribution? Please hold the teacher, prefix generator, student initialization, divergence, and training budget fixed. This comparison would help distinguish the benefit of improved teacher conditioning from that of PReViD’s target construction.

  2. Do the clear, degraded, and placebo prefixes have matched lengths, or can their lengths differ? If they differ, how do you ensure that changes in the teacher’s next-token distributions are attributable to evidence content rather than shifts in the positions of the replayed response tokens? A length-matched control or a controlled analysis would materially strengthen the prefix-intervention claim.

  3. Were the reported results averaged across independent training seeds? If so, please report the run-to-run variation for the main comparisons. If not, can the authors provide repeated runs for the central Table 1 comparisons? The current example-level confidence intervals in the path analysis do not quantify training variability.

  4. Which checkpoint or parameter state generated the prefixes and replay distributions in Tables 1 and 3? Were these parameters fixed throughout training or refreshed from the evolving student? Please reconcile Sections 4.2 and 4.4 with Appendix A.2 and the configurations used for the reported runs.

  5. What evidence supports the claim that prefix replay specifically mitigates the influence of earlier visual errors, beyond the general benefit of adding informative context? For example, do the gains differ for student histories that contain an identifiable earlier visual error versus histories that do not? Also, Table 8’s 76%/81% result is an ordered contrast, with the prefix effect measured after changing to the clear image; it is not an order-independent causal attribution. Please clarify how narrowly the paper intends this decomposition to support the mechanism claim.

4 comments

Thank you for your careful review and helpful comments. We appreciate your view of the method as a “focused and potentially useful extension of fixed-history visual distillation”. Thank you also for recognizing that the student inference procedure remains unchanged. We address your concerns and questions below.

W1 and Q1 on target construction and sources of improvement

Thank you for asking us to separate improved teacher conditioning from the effect of target construction. Sections 3.2 to 3.4 of the original paper describe four teacher replays, selective target construction, and the anchor term. Section 4.4 reports component ablations in Table 3.

Those ablations show successive gains from clear visual privileged information, visual projection, and teacher evidence prefixes. Random gates or shuffled prefixes reduce performance, indicating the role of relevant prefix content and its selection. PReViD also outperforms “all prefix shifts”, which retains projection and target reconstruction. This comparison supports selective use of the evidence. Reviewer CsGH expressed a related view of these controls. “The random gate and shuffled prefix baselines are well-chosen and cleanly demonstrate that informative prefix content is what actually moves the needle.”

Your requested comparison tests target construction under the same teacher context. We therefore trained Direct endpoint OPD and PReViD (no anchor) at both 4B and 9B using their respective objectives. Section 4.4 and Table 4 of the revised paper report these controls. Within each model size, student initialization, teacher and prefix generation protocols, JSD, and training update counts are shared. Each row summarizes the first inference evaluation and two additional evaluations with new seeds, all using one fixed set of weights. Vision-OPD and Ours keep the weights trained for the original paper.

For these controls, we use the two terms in the loss from Section 3.4 of the original paper. Their roles can be seen in the token loss

ℓt=DJS(pθ,t∥rt)+λDJS(pθ,t∥pt+,+)\ell_t=D_{\mathrm{JS}}(p_{\theta,t}\Vert r_t)+\lambda D_{\mathrm{JS}}(p_{\theta,t}\Vert p_t^{+,+}).

Matching the constructed target is the first term. Teacher endpoint matching is the second, weighted by the anchor coefficient.

The clear image and clear evidence prefix define the teacher distribution pt+,+p_t^{+,+}. The target rtr_t comes from four teacher replays followed by projection and gating. Removing the anchor leaves only the first term in PReViD. Direct endpoint OPD uses the second term alone with its coefficient set to 1. The full method uses both terms. The supplement reports the three inference evaluations for the target construction controls.

ModelV*ZoomHR4KHR8KMME-ENMME-CNAvg
Qwen3.5-4B
Vision-OPD87.26±0.3053.41±0.1479.88±0.6678.00±0.7667.19±0.1168.08±0.1872.30±0.27
Direct endpoint OPD88.83±0.3056.73±0.1880.79±0.1979.00±0.3370.73±0.0668.71±0.1974.13±0.08
PReViD (no anchor)90.05±0.5259.57±0.2581.83±0.3179.42±0.5271.04±0.0669.38±0.1275.22±0.17
Ours90.93±0.3061.42±0.1284.50±0.4581.13±0.7671.89±0.0470.26±0.0876.69±0.13
Qwen3.5-9B
Vision-OPD90.75±0.3058.70±0.3184.67±0.2682.54±0.7571.30±0.0468.20±0.1176.03±0.15
Direct endpoint OPD91.80±0.3060.63±0.4985.08±0.2683.67±0.5172.46±0.0669.72±0.2277.23±0.17
PReViD (no anchor)92.50±0.3062.09±0.3885.63±0.3884.50±0.3373.74±0.0770.12±0.1078.09±0.11
Ours93.54±0.3064.34±0.1886.13±0.1384.84±0.1974.38±0.0570.42±0.0478.94±0.06

At both model sizes, Direct endpoint OPD outperforms Vision-OPD. This supports the value of additional teacher context. PReViD (no anchor) further outperforms Direct endpoint OPD. Matching the constructed target rtr_t is thus more effective than directly matching the teacher endpoint in these comparisons. Ours also outperforms the no anchor variant. This supports the complementary roles of target construction and endpoint regularization.

W2 and Q2 on prefix length and position

Appendix A.2 of the original paper states that prefixes are generated greedily in each training batch, with a limit of 64 tokens. Section 3.1 specifies that the evidence prefix is placed before the full student history. Thank you for suggesting that we test the effects of length and response position. The 64 token limit does not ensure equal lengths across the three prefixes. Lengths vary across samples and views during generation. Appendix A.3 of the revised paper retains these generation rules.

To control response positions, we take the maximum length Lmax⁡L_{\max} of the three prefixes after generation for each sample. We add Lmax⁡−LbL_{\max}-L_b PAD slots to a prefix of length LbL_b. We trained separate evidence pre-PAD and post-PAD controls at 4B and 9B, and compared them with original PReViD. Pre-PAD slots follow the assistant start boundary and precede the evidence text. Post-PAD slots follow the evidence text and precede the student history. Both controls preserve the original evidence text and student history. PAD tokens are masked. Response positions are aligned in both controls.

Each control uses its trained weights for three inference evaluations with different seeds. Original PReViD retains its original trained weights and adds two inference evaluations with new seeds. The table below reports the means and sample standard deviations across the three inference evaluations. Appendix A.5 describes the controls and training setup. Table 8 reports the results. The supplement provides the three inference evaluations for the PAD controls.

SettingV*ZoomHR4KHR8KMME-ENMME-CNAvg
Qwen3.5-4B
Ours90.93±0.3061.42±0.1284.50±0.4581.13±0.7671.89±0.0470.26±0.0876.69±0.13
Masked evidence pre-PAD with aligned response positions90.23±0.6060.20±0.4283.83±0.5180.38±0.3371.28±0.1169.82±0.1375.96±0.11
Masked evidence post-PAD with aligned response positions90.58±0.5261.26±0.2583.71±0.8979.50±0.3871.46±0.1069.75±0.1076.04±0.21
Qwen3.5-9B
Ours93.54±0.3064.34±0.1886.13±0.1384.84±0.1974.38±0.0570.42±0.0478.94±0.06
Masked evidence pre-PAD with aligned response positions92.15±0.5264.30±0.4585.38±0.9484.42±0.4773.93±0.0870.00±0.1878.36±0.31
Masked evidence post-PAD with aligned response positions92.67±0.5263.71±0.7685.63±0.4584.46±0.5174.06±0.0970.11±0.1078.44±0.25

W3 and Q3 on experimental robustness

Thank you for your suggestion on experimental robustness. Section 4.1 of the original paper specifies an inference temperature of 0, native benchmark metrics, and a fixed judge configuration for ZoomBench. We retained this evaluation protocol and the original trained weights for each method. We added two inference evaluations with different seeds to the original evaluation. The table below reports results on six benchmarks across the three inference evaluations using the same weights. The main trends remain consistent. The means and sample standard deviations reflect variation in inference evaluation with fixed weights. Section 4.1 of the revised paper states this protocol. Tables 1 to 4 have been updated accordingly. The supplement provides the three inference evaluations for the main experiments and ablations.

ModelV*ZoomHR4KHR8KMME-ENMME-CNAvg
Qwen3.5-4B
Baseline80.98±0.6047.85±0.0783.38±0.3380.13±0.3362.69±0.0563.65±0.1269.78±0.18
SFT81.15±0.0049.63±0.1883.54±0.2678.63±0.4556.39±0.0759.57±0.1768.15±0.03
GRPO79.23±0.3054.91±0.1282.96±0.2978.75±0.1366.61±0.0560.01±0.1870.41±0.05
OPSD82.55±0.3052.27±0.2581.88±1.1580.67±0.4470.64±0.0369.11±0.1972.85±0.31
VA-OPD85.17±0.3053.02±0.3178.09±0.6475.71±0.4767.64±0.0468.92±0.1471.42±0.11
V-Zero84.82±0.5252.23±0.2579.21±0.5076.25±1.6268.22±0.0568.27±0.0271.50±0.35
Vision-OPD87.26±0.3053.41±0.1479.88±0.6678.00±0.7667.19±0.1168.08±0.1872.30±0.27
VGS87.96±0.5257.55±0.0781.92±0.3179.46±0.1967.55±0.0666.85±0.0873.55±0.04
VAD90.40±0.6153.06±0.3077.92±0.8375.79±0.4769.93±0.1067.81±0.1172.48±0.33
Ours90.93±0.3061.42±0.1284.50±0.4581.13±0.7671.89±0.0470.26±0.0876.69±0.13
Qwen3.5-9B
Baseline86.56±0.3052.39±0.1884.21±0.0780.21±0.3170.85±0.0267.53±0.1073.62±0.10
SFT87.08±0.3052.23±0.1484.21±0.3881.88±0.1371.53±0.0468.19±0.0874.19±0.06
GRPO86.21±0.3056.02±0.4284.21±0.4082.71±0.1971.52±0.0768.80±0.0974.91±0.12
OPSD89.18±0.3055.42±0.0783.13±0.3881.42±0.3172.46±0.0968.68±0.1175.05±0.07
VA-OPD90.05±0.9159.96±0.2584.04±0.1982.29±0.1571.86±0.0469.94±0.3476.36±0.16
V-Zero89.88±0.3056.41±0.1883.71±0.0784.25±0.5473.23±0.2969.82±0.1676.22±0.10
Vision-OPD90.75±0.3058.70±0.3184.67±0.2682.54±0.7571.30±0.0468.20±0.1176.03±0.15
VGS89.53±0.5261.42±0.2483.96±0.4782.67±0.5272.28±0.0469.91±0.2476.63±0.12
VAD93.02±0.6058.38±0.4985.29±0.2685.63±0.1372.28±0.0269.64±0.2777.37±0.15
Ours93.54±0.3064.34±0.1886.13±0.1384.84±0.1974.38±0.0570.42±0.0478.94±0.06

W4 and Q4 on consistency of the teacher description

Section 4.1 of the original paper specifies a teacher of the same model size for the main experiments. Section 4.3 identifies the 9B PReViD checkpoint from Table 1 as the fixed teacher. The original wording was inconsistent about the teacher's parameter state in the main experiments. We are sorry that these descriptions were unclear.

For Tables 1 and 3, PReViD uses the current policy for self-distillation. Teacher and student roles share parameters while evaluating different input contexts. The student is conditioned on the full image and its own history. Controlled images and evidence prefixes define the teacher context. Teacher distributions and reconstructed targets are computed without gradients. The loss propagates gradients only through the student distributions. Updating the student therefore also updates the parameters used in the teacher role.

Table 2 uses the 9B PReViD checkpoint from Table 1 as a fixed teacher for a student initialized from Qwen3.5-4B. In both settings, the corresponding teacher generates the evidence prefixes. We have corrected the frozen teacher description for the main experiments in Section 3 of the revised paper. Sections 4.1 to 4.3 now give consistent teacher and generator definitions. Appendix A.3 retains the generation and replay protocol from Appendix A.2 of the original paper.

Q5 on path order and the interpretation of early errors

Section 3.2 of the original paper defines the two paths as controlled model input contrasts. Figure 2 in Section 1 and Appendix B report the 76%/81% results for the image first order. Appendix B also gives the score definition and selection rule for the joint endpoint subset. We have added the reverse comparison with the prefix first order in Appendix B and Table 11 of the revised paper. This tests the effect of path order. The analysis reuses the original samples, scores for all four input combinations, and saved prefixes. The reverse path is (x−,e−)→(x−,e+)→(x+,e+)(x^-,e^-)\rightarrow(x^-,e^+)\rightarrow(x^+,e^+). We compute the prefix share by dividing the mean prefix component by the mean endpoint contrast.

Sample groupPrefix share with image firstPrefix share with prefix firstMean interaction I
All samples76.179%86.151%-0.343
Original joint endpoint subset81.448%90.114%-1.013

Both orders yield positive mean prefix components, showing that evidence prefixes can change later supervision while the student history remains fixed. The higher reverse shares and negative mean interactions indicate dependence on image and prefix context. The 76%/81% values are prefix shares in an ordered contrast, rather than causal attributions that are independent of path order.

Figure 1 in Section 1 and Section 4.2 of the original paper explain that early errors remain in the student history. Evidence prefixes aim to reduce their effect on later supervision. To illustrate this mechanism, Figure 5 in Appendix B of the revised paper adds evidence prefix examples from three samples. A degraded evidence prefix identifies a vending machine as a guard booth. Another describes a blue crane as red. These examples show that degraded inputs can produce incorrect observations, rather than only vague descriptions. During training, PReViD retains the student history. It selectively incorporates corrections supported by clear evidence into the supervision target to reduce the effect of incorrect observations on later predictions. Improvements across multiple benchmarks support the benefit of the method.

Thank you for these specific and helpful suggestions. We have retained the original component ablations and ordered path analysis. We have added newly trained target and position controls, reverse path diagnostics, and protocol clarifications to the paper and appendices. Your comments helped strengthen the experimental evidence and clarify the scope of our interpretation.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.