acceptodds
Under review as a conference paper at ICLR 2027

Self-Boosting Vision-Language Models with Noisy Student On-Policy Self-Distillation

Abstract

Post-training enables vision-language models (VLMs) to understand human instructions and perform various downstream tasks. Current post-training methods usually rely on human-annotated data, distillation from external models, reinforcement learning with human feedback, or verifiable answers. This limits their ability to improve without external supervision. To tackle this, we propose NOPD (Noisy Student On-Policy Self-Distillation), a simple yet effective self-distillation approach that improves VLMs without any external models or ground-truth answers. Our key insight is that prediction discrepancies between clean and corrupted inputs naturally induce a self-supervision signal. In NOPD, the model learns from corrupted inputs while using its own predictions under clean inputs as token-level supervision. We show the effectiveness of NOPD on five visual reasoning tasks; it can match and even outperform reinforcement learning approaches or distillation from external models. Notably, when trained with 2.1K samples from Geometry3K, NOPD improves Qwen2.5-VL-7B by 20 points on its validation set. It also shows generalization on out-of-distribution test sets and achieves 7.4 point gains on MathVista. Furthermore, we demonstrate that NOPD is a general approach to enhance VLMs, achieving improvements across three models on 12 benchmarks.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
on-policy distllation, post-training

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_ZBVnOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Noisy Student On-Policy Self-Distillation (NOPD) for post-training vision-language models without ground-truth answers or an external teacher. The student generates on-policy responses from corrupted images, while the same model, conditioned on the corresponding clean images and student-generated prefixes, provides token-level supervision. Experiments report improvements over base models across various benchmarks, with performance comparable to or better than RLVR and external-teacher on-policy distillation on several reasoning benchmarks.

Critical strengths
  1. The paper presents a simple approach to VLM self-improvement without ground-truth answers or an external teacher, which is a relevant research topic on post-training of VLM.
  2. The ablations support key design choices, and the experiments cover both reasoning and broader VLM benchmarks.
  3. The paper is generally well-written and easy to follow.
Critical weaknesses
  1. Comparisons with SSL4RL [1] and Visual Jigsaw [2] are missing, as both enable post-training without human-provided answers or external teachers. Image Jigsaw reports 84.45 on MMBench-EN, versus NOPD's 83.6 on MMBench-EN, which raise a doubt on the effectiveness of NOPD.
  2. Some formatting issues, the margin between Sec.5 and Sec.6 are quite small which could be adjusted for readability.

[1] Guo et al. SSL4RL: Revisiting Self-supervised Learning as Intrinsic Reward for Visual-Language Reasoning. ICML 2026.

[2] Wu et al. Visual Jigsaw Post-Training Improves MLLMs. ICLR 2026.

Questions for authors
  1. Can the authors compare the performance NOPD with SSL4RL and Visual Jigsaw under matched backbones, data?
  2. Will the authors release the training and evaluation code in the revised version of this paper?

I would consider raising my score if the questions are addressed.

Reviewer_KtN9OpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors propose a simple and effective self-training scheme that uses clean and corrupted images without requiring additional models or ground-truth supervision. The model acts as the teacher when processing clean inputs and as the student when processing corrupted inputs. Experiments demonstrate that training on clean–corrupted input pairs can substantially improve response quality. The method can also be combined with other approaches to further improve performance.

Critical strengths

The paper uses prediction discrepancies between clean and corrupted inputs to provide a self-supervision signal. I am particularly interested in understanding what useful information these discrepancies contain and how it benefits learning. I find the empirical effectiveness of this simple idea appealing and consider it a key strength of the pape. But i need a clearer experimental analysis to support this underlying mechanism.

Critical weaknesses

As noted in the strengths, the authors state, “Our key insight is that prediction discrepancies between clean and corrupted inputs naturally induce a self-supervision signal.” The analysis and empirical evidence supporting this central claim are central to my evaluation of the paper.

Questions for authors

1.The current ablation studies support the conclusion that introducing input corruption during self-training is beneficial. However, I would like to better understand what the model learns from the prediction discrepancies between clean and corrupted inputs. Do these discrepancies help the model learn alternative approaches to answering questions, or do they provide some other benefit? Is there further experimental evidence clarifying this mechanism?

2.In addition, given that the method relies on self-training, have the reliability of the teacher’s predictions and the potential for error accumulation been examined? Specifically, how many student errors are corrected by supervision derived from the model’s responses to clean images, and how many new errors does this supervision introduce?

3.Could using the same model’s predictions under clean inputs as supervision lead to a performance bottleneck? In addition, when the two branches become increasingly consistent but still answer incorrectly, where does the signal for further improvement come from?

Reviewer_uUVJOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes NOPD, a label-free post-training method for vision-language models. The model samples a response from a corrupted image (Gaussian noise, gamma transformation, or 2×/4× downsampling). It is then trained with a token-level reverse KL that pulls its next-token distribution under the corrupted image (student) toward its own distribution under the clean image (teacher: same weights, stop-gradient, synchronized every step). No ground-truth answers, rewards or external teachers are used. With 100 training steps on 2.1K Geometry3K samples, NOPD improves Qwen3.5-2B and Qwen2.5-VL-7B on five math and multidisciplinary benchmarks. The paper also ablates the divergence, teacher update rule and corruption type. However, the paper lack support of main claim, so more experiments is a must.

Critical strengths
  • Simple, cheap and label-free. NOPD needs only unlabeled image–question pairs and one rollout of ≤2,048 tokens per prompt. It needs no reward, verifier or stronger teacher, so it also applies to non-verifiable tasks (general VQA, charts, captioning). If the gains were shown to be genuine and specific to the method, this would be of clear practical value.
Critical weaknesses

The mechanism is unverified, the key baselines are missing, and the reported numbers are not reliably comparable. I therefore do not consider the main conclusions established; fixing this would require a substantially new set of experiments.

  1. The central claim is not supported: the gains are plausibly generic self-sharpening rather than “self-induced supervision” from clean/corrupted discrepancies.

    • The teacher is the model’s own clean-image distribution, which is exactly what is evaluated at test time. Matching it yields corruption invariance, and that need not improve clean-image accuracy. The paper’s explanations (“difficult environments”, privileged information) are not tested.

    • The paper’s own ablations point to sharpening:

      • a fixed clean teacher gives only +0.8 on MathVista, versus +4.3 with a synchronized teacher (Table 2c);
      • 1,024-token rollouts, ≈13% of the base model’s average MathVista response, rarely reach the final answer, yet already give 3.3 of the 4.3 points (Table 2d);
      • mode-seeking reverse KL works best (Table 2a);
      • outputs shrink by 26–58% (Fig. 3).
    • The decisive label-free baselines are missing:

      • entropy minimization (S. Agarwal et al., NeurIPS 2025);
      • majority-vote self-rewarding RL: TTRL, and MM-UPT (Wei et al., NeurIPS 2025), which reports MathVista 66.3→72.9 on Qwen2.5-VL-7B;
      • training-free visual contrastive decoding (VCD; Leng et al., CVPR 2024), which exploits the same clean-vs-noised discrepancy.
    • Novelty is overstated:

      • a clean teacher supervising a noised student is the core of Noisy Student (Xie et al., CVPR 2020; the name is reused but the paper is not cited), and of UDA, FixMatch, Mean Teacher and DINO;
      • clean-vs-corrupted signals for VLMs already appear in VCD, STIC (Deng et al., 2024) and PAPO (Wang et al., ICLR 2026).
  2. The evaluation cannot support the headline claims.

    • Inconsistent protocols.

      • The Qwen2.5-VL-7B base scores differ between Tables 1 and 4: MMMU 58.6 vs. 51.3, MMMU-Pro 38.3 vs. 34.6, MathVision 25.1 vs. 22.4.
      • NOPD’s Table 4 MMMU (58.1) is below the base model’s Table 1 score, so the Table 4 gains may reflect a weak base-model setup.
      • Table 1 also takes the maximum of direct and CoT prompting for MMMU(-Pro) (App. A.1), which is a selection made on the test set.
    • No variance. All results are single runs, while many conclusions rest on gaps of about 1 point (Tables 1–3, 9).

    • Weak baselines and unsupported efficiency claims. Baselines appear untuned (a single lr of 1e-6 for all methods; Table 7). The claimed data efficiency (Sec. 3, “see Sec. 4.2”) is not tested anywhere, and no compute cost is reported.

    • No contamination check. MathVista contains problems from Geometry3K, GeoQA+, ChartQA and VQA2.0, all of which are in the training mix, and a label-free method can fit test images directly.

Questions for authors

Major: See weekness;

Minor:

“permutation” should be “perturbation”. GLM-4.6V-Flash improves on 9/12 benchmarks, not 10/12 (Table 4). The CapArena claim that a 7B model “can match frontier models” is overstated: losses exceed wins against both Gemini-3-Flash and GPT-5, with n=100. Data filtering uses Qwen3-VL-8B-Instruct, which should be acknowledged next to “without any external models”.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.