- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies how erroneous student-generated prefixes can override privileged visual evidence during multimodal on-policy distillation. OPD-Aha contrasts the same frozen teacher under a localized evidence crop and a visual null, then reconstructs the distillation target by amplifying their log-probability difference. Distillation still uses unchanged student prefixes. Experiments report improved visual accuracy and reasoning transfer, accompanied by more frequent self-interruption and renewed visual support.
Critical strengths
The paper is clearly written and identifies an interesting failure mode. The prefix-length diagnostics, token-shuffling controls, alternative visual nulls, and qualitative examples provide informative analyses. The reported improvements over Vision-OPD are consistent across six visual benchmarks at both evaluated scales.
Critical weaknesses
- Target stability and reconstruction-strength selection are insufficiently characterized. Equation (7) can concentrate the reconstructed target on tokens with very small null probabilities. Please specify any numerical safeguards. Figure 3(d) tests only , with no account of validation-based selection or behavior beyond this range. Table 2 fairly fixes both reconstruction signals at , but should also test whether its conclusion holds at the main setting or across a shared tuning grid.
- The additional training cost is not reported. OPD-Aha requires two teacher evaluations rather than the single teacher branch of Vision-OPD. Please report training throughput, GPU-hours, and peak memory.
- The reflection analyses are suggestive but do not establish causality. Please specify the reflection-token set, diagnostic sample sizes, and matching procedure for non-reflection positions. Controlled suppression or insertion of reflection markers, with matched continuation budgets, would help distinguish causal visual recovery from correlated changes in response style or length.
- Table 3 shows substantial post-training transfer regressions for the 4B GRPO and Vision-OPD baselines, rather than demonstrating progressively worsening training. Please clarify their checkpoint selection and evaluation settings.
Questions for authors
See weakness.
4 comments
Author Response (W2): Training Cost
W2: What the Second Teacher Pass Costs
- Vision-OPD scores each student rollout once with the teacher; OPD-Aha scores the same rollout twice, with the crop and with the visual null. The student still generates eight rollouts per prompt (Table 4), so the extra work is one gradient-free teacher forward per rollout. Inference cost is unchanged: the student runs alone on the original image.
- We measured both 4B arms of the full-image ablation on the same eight A100s over the same early-training measurement window:
Tab. R2: Training Cost of the Second Teacher Pass (4B, Measured Training Window)
| 4B, same hardware | Rollouts / s | Median step (s) | Null-teacher forward (median s / step) | GPU-hours (measured window) | Highest sampled memory / GPU |
|---|---|---|---|---|---|
| Standard target (, one teacher pass) | 1.45 | 526 | 0 | 10.56 | 75.6 GiB |
| Real–null target (, two teacher passes) | 0.98 | 775 | 62 | 15.64 | 79.2 GiB |
- Interpreting the cost difference. The null-teacher pass accounts for part of the measured step time. Longer responses also contribute to the total cost difference in this early window (Tab. R2b), consistent with Fig. 3c. Memory was sampled with nvidia-smi across all GPUs, and the table reports the largest observed sample during the measurement window. We will supplement these windowed measurements with full-run cost and peak-memory measurements for the main crop configuration in the revision.
Tab. R2b: Response Length and Timing Context for the Cost Measurements
| Measurement | Value |
|---|---|
| Standard-target median response length | 145 tokens |
| Real–null median response length | 186 tokens |
| Null-teacher share of real–null step time | 8% |
| Device-memory sampling interval | Approximately 30 s |
Takeaway: The extra teacher pass and longer student responses both contribute to the measured early-training cost. These windowed measurements will be supplemented with full-run measurements in the revision.
Author Response (W3): Reflection Diagnostics
W3: Lexicon, Matching, and the Intervention
Reflection lexicons. The reflection rate (Fig. 4b) counts wait, actually, however, look again, and looking again. The event analysis (Fig. 5) anchors on the first match of wait, hold on, looking again, look again, look more carefully, reexamine, doublecheck, now I see, made a mistake, misread, or closer inspection. Matching is case-insensitive and uses whole words. Reexamine and doublecheck also match with a space or hyphen.
Matched non-reflection positions. Each cue response is paired with a cue-free response from the same benchmark with the same final-answer correctness and enough remaining tokens. The control anchor has the same token offset as the end of the observed cue. Among eligible controls, we prefer least prior reuse, then matching category, then longer responses, with deterministic tie-breaking. Each curve subtracts the mean of the 16-token window preceding its anchor. We will include the diagnostic sample sizes in the revised figure captions.
Diagnostic quantities. For the emitted token , continuation support is and visual-evidence support is , evaluated at the same prefix. These diagnostics measure changes in textual and visual support around the cue.
- Suppression (inference only, OPD-Aha 4B and 9B). On HR-Bench-4K and 8K we set the logits of wait, wrong and however (all capitalizations) to −∞ throughout generation. Both arms use our evaluation decoding (non-thinking, greedy, 8,192-token budget, seed 42) and the same LLM judge; the baseline runs the same code with a non-existent word suppressed. Tab. R3 reports the questions whose baseline answer used one of the three words, the only ones suppression can affect.
Tab. R3: Suppressing Three Reflection Words on HR-Bench-4K/8K
| Model | Baseline acc. (%) | Suppressed acc. (%) | Net change, pp [95% CI] | McNemar p |
|---|---|---|---|---|
| 4B | 62.3 | 60.7 | −1.6 [−6.0, +2.8] | 0.60 |
| 9B | 64.3 | 63.8 | −0.5 [−5.7, +4.7] | 1.0 |
-
What suppression shows. Neither change is significant, and at 4B answers flip about as often as between two unsuppressed runs. Suppression also does not remove reflection: at 4B, alternative cues such as actually, check and look again still appear. The test therefore cannot show that these words are necessary for recovery.
-
Pilot. Inserting Wait versus Next at shared prefixes with a matched 2,048-token continuation budget yields similar accuracy with the real image and no advantage for Wait with a null image. The next-token real–null TV difference favors Wait, but its confidence interval includes zero at both scales (Tab. R3b). The pilot is small and its prefixes were not filtered for a verified perceptual error, so it remains inconclusive.
Tab. R3b: Next-Token Real–Null TV Difference in the Insertion Pilot (Wait − Next)
| Model | TV difference | 95% interval includes zero |
|---|---|---|
| 4B | +0.064 | Yes |
| 9B | +0.045 | Yes |
- What we will claim. Neither intervention shows that the reflection words are necessary or sufficient for re-anchoring. We will revise Section 4.3 to describe reflection as a marker that accompanies renewed visual reliance rather than its driver, and keep the training-side result (Section 4.2), which does not depend on this question.
Takeaway: Both lexicons and the matching rule are now stated, and the matched-budget interventions do not establish causality, so we will present reflection as a marker of renewed visual reliance rather than its cause.
Author Response (W4): Checkpoints and Evaluation Settings behind Table 3
W4: Checkpoints and Evaluation Settings behind the Table 3 Regressions
- Checkpoints. Every Table 3 model is the final checkpoint of its perception run (one epoch over Vision-OPD-6K, 65 steps at batch 96), with no selection on any math benchmark. The same checkpoints produce Table 1.
Evaluation mode and budget. Our initial MathVision evaluation uses non-thinking mode, following Vision-OPD and OPD-V, with a 4,096-token first-pass budget. Tab. R4a summarizes the related protocols.
Tab. R4a: Evaluation Settings Reported in Related Visual-Distillation Work
| Work | Reported setting | Relevance to our evaluation |
|---|---|---|
| Vision-OPD [1], Section 4.1 | Qwen3.5 training and evaluation use non-thinking mode | Supports the choice of non-thinking mode |
| OPD-V [2], Section 4.1 and Appendix E | Training and evaluation use non-thinking mode | Also uses the non-thinking convention |
| S²VOPD [3], Section 4.1 | Perception: greedy, 4,096 tokens. Mathematics: 24,576 tokens, temperature 0.3 | Uses a longer budget for math evaluation |
| Our initial MathVision pass | Non-thinking, 4,096 tokens | A limited-budget transfer evaluation |
Effect of a longer budget. Fresh base-model evaluations show that a longer non-thinking budget improves accuracy at both scales (Tab. R4b).
Tab. R4b: MathVision testmini Base-Model Accuracy under Non-Thinking Single-Pass Evaluation
| Generation budget | Qwen3.5-4B | Qwen3.5-9B |
|---|---|---|
| 4,096 tokens | 43.8 | 47.0 |
| 32,768 tokens | 58.9 | 63.8 |
- Evaluation settings, and an inconsistency we are correcting. The original first pass used non-thinking mode on MathVision testmini with a 4,096-token budget. Our logs show that the 4B base and Vision-OPD rows additionally had truncated answers regenerated at 8,192 tokens, while the GRPO row did not. Under the same first pass the picture is:
Tab. R4c: 4B MathVision testmini under Matched Evaluation Passes
| 4B MathVision testmini, accuracy % | Base | GRPO | Vision-OPD |
|---|---|---|---|
| Non-thinking, 4,096 first pass (original logs) | 43.4 | 46.1 (+2.6) | 41.1 (−2.3) |
Under the matched first-pass budget, Vision-OPD remains below the base model. The GRPO regression in Table 3 is a protocol artifact: GRPO is above base when both receive the same pass. We apologize for this inconsistency and will complete the uniform-protocol evaluation for all Table 3 methods in the revision.
Base and OPD-Aha under a shared thinking protocol. We evaluated the released base and OPD-Aha checkpoints on MathVision testmini with a shared thinking protocol, model-card style boxed-answer prompt, and 40,960-token budget (Tab. R5).
Tab. R5: MathVision testmini Accuracy under a Shared Thinking Protocol
| Model | Qwen3.5-4B | Qwen3.5-9B |
|---|---|---|
| Base | 64.5 | 75.3 |
| OPD-Aha | 70.7 | 74.3 |
Scope of the transfer claim. Table 3 compares final checkpoints with the base models. It establishes post-training differences, rather than progressively worsening performance over training. We will revise the wording accordingly.
Independent evidence of transfer regression. S²VOPD [3] also reports in its Table 3 a MathVision decline for the Vision-OPD student alongside a MathVista improvement, using the same Vision-OPD-6K data and recipe (Tab. R6). This illustrates that perception gains need not translate to mathematical reasoning.
Tab. R6: Reported 4B Results from Table 3 of S²VOPD [3]
| Model | MathVision | MathVista |
|---|---|---|
| Base | 65.3 | 75.8 |
| Vision-OPD | 62.5 | 79.4 |
| Other privileged methods | MathVision change, 4B | MathVision change, 9B |
|---|---|---|
| OPSD | −9.3 | −7.9 |
| ZwZ | −13.1 | −9.9 |
OPD-Aha improves MathVision at both scales under the original Table 3 protocol, and its gain under the shared thinking protocol is concentrated at the smaller scale (Tab. R5).
Takeaway: The reported transfer results compare final checkpoints under specific evaluation settings. Matching the evaluation passes removes the apparent GRPO regression, while Vision-OPD remains below the base model. These comparisons do not establish progressive deterioration during training.
References
[1] Vision-OPD: Learning to See Fine Details for Multimodal LLMs via On-Policy Self-Distillation, 2026.
[2] OPD-V: Visual On-Policy Self-Distillation with Modality Balance, 2026.
[3] Self-Supervised Visual On-Policy Distillation, 2026.
Author Response (W1): Target Stability and the Choice of β
We thank Reviewer aRhc for four concrete requests. We address target stability, training cost, reflection diagnostics, and transfer evaluation below.
Comparison at the main strength. We trained the teacher–student target at β=4 with the same privileged crop, data, and training recipe as OPD-Aha. Tab. R1a compares both targets at step 30 using the same scorer. The real–null target leads on all six benchmarks and both aggregate metrics. Thus, the advantage of the intra-teacher contrast also holds at the main reconstruction strength.
Tab. R1a: Teacher–Student versus Real–Null Targets at β=4 (Qwen3.5-4B, Step 30)
Mean₄ averages the first four benchmarks, and Avg₆ averages all six.
Selection of β. We stopped at 4 because the accuracy increments had diminished, while the transient response-length increase grew with β (Fig. 3c). We will clarify the data split and selection procedure in the revision.
Behavior beyond β=4. A further β=5 run, evaluated at step 30 with the same scorer, shows no consistent accuracy improvement on the completed benchmarks (Tab. R1b). Responses also become longer, including late in training (Tab. R1c). These results support retaining β=4.
Tab. R1b: Real–Null Reconstruction at β=4 and β=5 (Qwen3.5-4B, Step 30)
Tab. R1c: Response Length at the Main and Higher Reconstruction Strengths
Takeaway: Log-space normalization addresses numerical computation, while concentration on small-null-probability tokens remains a separate stability question. The real–null target retains its advantage at the main strength, and stronger reconstruction brings longer responses without consistent accuracy gains.