acceptodds
Under review as a conference paper at ICLR 2027

Visual-OPSD: Cross-Modal On-Policy Self-Distillation for Efficient Unified Multimodal Reasoning

Abstract

Unified multimodal models (UMMs) interleave generated "visual thoughts" (VTs) with text reasoning to improve spatial tasks. This incurs roughly an order-of-magnitude inference cost from multi-step diffusion. We find this cost yields limited direct benefit. On ThinkMorph, removing or noising VTs barely changes accuracy across nine benchmarks. Once rendered, attention concentrates on the VT regardless of content. Yet a KL diagnostic shows that conditioning on a privileged VT trace shifts the model's completion distribution. This suggests the generation pathway encodes useful reasoning beyond the rendered pixels. Motivated by this gap, we propose Visual On-Policy Self-Distillation (Visual-OPSD). Teacher and student share identical weights but differ in context: the teacher sees privileged VTs while the student sees only the question. Token-level JSD distillation on on-policy student trajectories transfers the teacher's reasoning to a text-only student. Across nine benchmarks, Visual-OPSD improves over its generative teacher by pp with speedup (10.0s vs. 142.8s per sample) and outperforms same-scale VLMs by pp on VSP. A Gaussian-noise control (pp vs. pp for real VTs) and closure of the KL gap confirm that gains come from the semantic content of the generation pathway.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Unified multimodal models, Interleaved visual chain-of-thought, On-policy self-distillation, Efficient multimodal reasoning

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_4g8HOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper finds that, during the inference phase, removing or adding noise to the visual reasoning processes generated by the model itself has a negligible impact on accuracy across nine benchmark tests. Based on this observation, the paper proposes a policy-based automated distillation method called Visual-OPSD. This approach not only outperforms the ThinkMorph model but also reduces the processing time per sample.

Critical strengths
  1. The authors found that removing the visual thoughts automatically generated by ThinkMorph during the inference phase has almost no impact on accuracy.

  2. This method requires no changes to the model architecture, outperforms the text-only mode of the same model, and reduces the output length by approximately half.

Critical weaknesses
  1. The paper states that Visual-OPSD incorporates knowledge distillation into the reasoning process. According to Algorithm 1, the teacher model does not actually utilize the generated paths.

  2. The speed improvement is primarily attributed to the disabling of the diffusion mechanism. The paper does not address the inherent latency issues of the ThinkMorph text-only model itself.

Questions for authors
  1. In the teacher context, do the VT images refer to the reference visual texts from the dataset, or are they images generated by ThinkMorph itself

  2. Given the same dataset and comparable computational resources, could you provide baseline results for SFT, RFT, or RL models initialized using ThinkMorph

Reviewer_F3piOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces Visual On-Policy Self-Distillation (Visual-OPSD) for efficient reasoning in unified multimodal models. Controlled interventions on ThinkMorph show that removing or corrupting intermediate visual thoughts has little effect on accuracy, despite their substantial generation cost. Visual-OPSD uses a JSD-based objective on student-generated trajectories to distill predictions from a teacher with access to privileged visual thoughts into a student conditioned only on the original image and question. Across nine benchmarks, the method improves average accuracy by 3.40pp over interleaved ThinkMorph and 4.13pp over its text-only mode, while achieving a 14.3× speedup relative to the interleaved model.

Critical strengths
  1. The paper provides a useful empirical finding that interleaved image generation is not necessary at inference for multimodal reasoning. Controlled interventions on ThinkMorph show that removing or corrupting visual thoughts largely preserves accuracy across nine benchmarks, motivating more efficient alternatives to interleaved image–text reasoning.
  2. The paper complements benchmark results with noise controls, token-level KL diagnostics, and qualitative case studies. These analyses suggest that the gains depend on meaningful visual thought content and help illustrate how intermediate image generation may interfere with reasoning.
Critical weaknesses
  1. As described in Sec 3.1, the text-only SFT baseline is initialized from BAGEL-7B, whereas Visual-OPSD is initialized from ThinkMorph. The memorization argument given for this choice lacks experimental support. Moreover, this argument applies equally to Visual-OPSD. According to Appendix C, its training data is identical to ThinkMorph's training distribution, so part of its gain may also come from re-fitting data the model has already seen. As a result, the reported +10.28pp gain does not isolate the contribution of Visual-OPSD. Since ThinkMorph already reaches 69.90% in text-only mode, only 4.13pp of the improvement comes from the additional training. Although the noise control supports the role of VT content, matched-initialization baselines are needed to distinguish Visual-OPSD's contribution from further in-domain training on ThinkMorph's data.
  2. The reported 14.3× speedup is measured against interleaved ThinkMorph, although the paper itself notes that simply disabling VT generation already removes the diffusion overhead. Since Table 2 does not report the latency of ThinkMorph's text-only mode, the additional speedup attributable to distillation remains unclear.
  3. Visual-OPSD is evaluated only on ThinkMorph, leaving its generalizability to other UMMs unverified, despite the conclusion suggesting applicability to Chameleon, Emu3, and Janus-Pro.
Questions for authors
  1. Could the authors report a text-only SFT baseline initialized from ThinkMorph, using the same data and a comparable training budget, to distinguish the benefit of VT-conditioned distillation from further in-domain training?
  2. Could the authors report the latency and output length of ThinkMorph in text-only mode to isolate the additional efficiency gain from Visual-OPSD?
  3. Could the authors apply Visual-OPSD to at least one additional UMM to support the claimed generality beyond ThinkMorph?
Reviewer_oogeOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies whether the generated visual thoughts are necessary in solving problems of the unified multimodal models. Pilot experiments show that noising or removing the generated images does not affect the performance a lot. This observation motivates the proposed Visual-OPSD. The method involves distilling a student that infers without generating visual thoughts from the teacher that is able to generate visual thoughts. Extensive experiments show the effectiveness of the proposed method.

Critical strengths
  1. This problem is necessary and practical. The inference cost of generating visual thoughts can be removed by OPSD.
  2. The proposed approach is effective and efficient, removing image generation cost at an affordable training.
  3. The evaluation on nine benchmarks is extensive and comprehensive.
Critical weaknesses
  1. The text-only SFT results should include one starting from the ThinkMorph checkpoint, although there is an overfitting problem.
  2. It seems removing generated images in ThinkMorph affects the text answer probability a lot compared to the one with generated images. It would be interesting to compare the distribution between ThinkMorph (w/o images) and BAGEL or the text-only SFT model.
  3. Does this method work for other UMMs? Including more base models may validate the generalization ability of the proposed method.
  4. This paper does not report the inference latency of the base ThinkMorph model with text-only inference. Including these results may be clear to show the accuracy-latency trade-off.
Questions for authors

See weaknesses.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary for the Area Chair

We have addressed the reviewers’ questions in our point-by-point replies and revised the PDF to incorporate the new experiments and clarify the paper’s claims.

In controlled experiments, ThinkMorph largely retains its benchmark accuracy when we remove its intermediate images at inference or replace them with noise. We find a useful role for visual thoughts in training. Conditioning the same model on reference visual thoughts changes its next-token predictions on identical completions, and distillation with that context yields higher accuracy than a matched noise control. Visual-OPSD turns these predictions into supervision along the student’s own reasoning trajectories, so the student learns to reason in text from the original image and question.

  • Matched training comparisons. With the same ThinkMorph initialization, source data, and training steps, Visual-OPSD outperforms the added SFT, RFT, and GRPO baselines, improving average accuracy by 3.85 percentage points over SFT.
  • Text-only inference efficiency. After visual thought (VT) generation is disabled, distillation delivers a further 1.88× speedup over text-only ThinkMorph, with shorter outputs and higher accuracy.
  • Transfer across backbones. Visual-OPSD improves average accuracy on both Janus-Pro and native BAGEL. The revised results include VSP and report accuracy and latency at each checkpoint.
  • Teacher inputs and token diagnostics. The revision clarifies that the teacher uses dataset reference VTs by default, documents the ablation with cached ThinkMorph-generated VTs, and explains how token predictions are evaluated under reference prefixes.

We thank the reviewers, Area Chair, and Senior Area Chair for their time and constructive feedback. Their questions helped us strengthen the experimental comparisons and make the paper’s claims more precise

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.