acceptodds
Withdrawn from ICLR 2027

LatentOPD: On-Policy Distillation for Multimodal Reasoning with Visual Latents

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Visual latent reasoning lets multimodal language models carry visual evidence through continuous tokens interleaved with textual reasoning, reducing reliance on external tools and explicit image generation. Existing training approaches often supervise these tokens with fixed latent trajectories from image–text reasoning traces. However, uninformative auxiliary views can encourage shared soft prompts, while fixed targets fail to adapt to the student's evolving reasoning context. We propose , which combines informative visual supervision with on-policy distillation over latent sub-trajectories under teacher-forced textual contexts. We train a privileged teacher to encode information-rich auxiliary views into latent trajectories that support subsequent reasoning. After learning to reason from teacher-provided latents, the student learns to generate latent states supervised by the teacher's privileged representations conditioned on student-generated latent prefixes. At inference, the student alternates latent and textual reasoning using only the original query. Experiments with Qwen2.5-VL-7B show improved accuracy over supervised fine-tuning baseline across four benchmarks of visual, mathematical, and geometric reasoning, with a higher weighted average than the compared visual latent methods.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
multimodal large language models, vision Language Models, multimodal reasoning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_wa6uOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies latent visual reasoning. It argues that uninformative auxiliary views cause latent tokens to collapse into flat representations, and that fixed latent targets drift away from the student’s own prefixes. LatentOPD introduces three stages: First it trains a privileged teacher and warm-starts the student on the teacher’s latents, and then distills the teacher’s layer-wise states along the student’s own latent prefix. On Qwen2.5-VL-7B, it improves over the base model and SFT on four benchmarks and achieves the highest average among the latent methods compared.

Critical strengths
  1. The teacher recomputes targets on the student’s own latent prefix, which directly addresses the drift problem. The ablation shows that fixed targets do much worse.
  2. Large gains over the base model (+9.18) without tools at inference.
Critical weaknesses
  1. The comparisons are uncontrolled. The baselines are trained on different data, and external scores are mixed with stricter in-house judge scores. There is also no baseline trained on the same data without latents.
  2. Stage I is essentially Monet, the “informative data” claim is never ablated, and the fixed-latent ablation (28.8) falls below the base model (32.7).
Questions for authors
  1. How do Monet and LVR perform when retrained on the same data?
  2. Can you ablate informative vs. low-information auxiliary views?
  3. Why does the fixed-latent variant underperform the base model?
Reviewer_RyF1OpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes LatentOPD, a 3 stage training pipeline for teaching multimodal models how to use latent tokens for multimodal reasoning. The first stage trains a teacher model T to learn how to represent an auxiliary image with latent tokens using an attention masking approach. The second stage trains a student model S to be able to understand and use these latent tokens by inserting T’s latent tokens into the input context of S and making S condition on these tokens for subsequent generation. The third stage trains S to generate its own latent tokens rather than relying on T’s inserted tokens. The paper makes the third stage “on-policy” by injecting the student generated tokens into the teacher T.

Experiments on Qwen2.5-VL-7B demonstrate that LatentOPD outperforms base Qwen2.5-VL-7B, vanilla SFT and existing latent reasoning methods (Mirage, CoVT, Monet, LVR, SkiLa) on average, across four multimodal reasoning benchmarks - EMMA, Visual Puzzles, MathVision and GeoLaux.

The paper further ran ablations to demonstrate the need for stage 2 and the need for on-policy distillation in stage 3. The paper also compared the latent tokens of LatentOPD with LVR and Monet, demonstrating that the latent tokens of LatentOPD are less similar across different tasks. The paper also showed that replacing later positions of latent tokens with the first latent token showed a sharper drop in accuracy for LatentOPD as compared to LVR, demonstrating a stronger causal dependence on the later latent tokens for LatentOPD.

Critical strengths

S1. The adaptation of on-policy distillation to latent visual reasoning is interesting. The improvements over fixed-latent baseline in Table 2 suggests that this is useful, although the experiment can be better controlled, as discussed in W2 below.

S2. The experiments in Section 5.4 provides useful insights of analyzing the latent tokens, beyond raw VQA numbers. Figure 3 shows existing methods like Monet and LVR producing highly similar latent embeddings across different samples, suggesting that they may be learning generic latent patterns rather than sample-specific representations. Figure 4 shows that in methods like LVR, replacing later tokens with the first does not decrease accuracy by too much, suggesting that not all latents are equally useful for reasoning.

Critical weaknesses

W1. Limited comparability of the main experiments (Table 1).

(a) The existing latent reasoning baselines were trained on different data distributions and compute budgets, and the paper did not demonstrate that they were retrained under matched data and compute. For example, LVR was trained on region/ bounding box based data from Visual-COT, while Monet’s training data was also mostly Visual-COT, with limited (100) geometry-related data from ZebraCOT. LatentOPD instead uses 1058 geometry examples from ZebraCOT. Since the paper evaluated mostly on such geometry/ puzzle based benchmarks, which are also different from the benchmarks that the original papers like Monet or LVR evaluated on, comparing these baselines trained with matched data (or at least similar data distribution) is critical to prove that the gains come from LatentOPD’s method rather than the training data being more in distribution to the test.

(b) The experiment settings of the Vanilla SFT baseline that the authors ran are not fully specified.

W2. While the on-policy latent distillation is one of the paper’s main method contribution, the on-policy ablation may be confounded. More specifically, LatentOPD computes layer-wise loss for the hidden states of the latent tokens (equation 5, page 6), while the fixed-latent ablations only computes loss across latent embeddings rather than all layers (section 5.3, and appendix D2, equation 19). This makes it difficult to establish if the gains of LatentOPD vs fixed latent supervision comes from the on-policy distillation or just stronger, per-layer supervision from LatentOPD.

Questions for authors
  1. Do the gains of LatentOPD vs existing latent visual reasoning methods come from LatentOPD's method design or from the training data being more in distribution to test? (refer to W1)

  2. Do the gains of LatentOPD vs fixed latent supervision come from the on-policy distillation or just stronger per-layer supervision from LatentOPD? (refer to W2)

Reviewer_zSFgOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

LatentOPD studies how to train continuous visual latents that support a VLM’s subsequent textual reasoning. A privileged teacher learns from auxiliary images in 35,920 selected ZebraCoT examples. The student first learns to consume teacher latents, then matches layerwise teacher states computed on its own latent prefixes. Text remains teacher-forced during training; inference uses only the original image–text query. Experiments with Qwen2.5-VL-7B cover four reasoning benchmarks, training ablations, cross-sample similarity, and latent interventions, reporting gains over SFT and the highest weighted average among latent baselines with complete coverage.

Critical strengths

Useful gains across several tasks. Table 1 (p. 8) reports a weighted average of 35.53, versus 28.45 for vanilla SFT and 32.09 for SkiLa, the strongest other latent method with complete coverage. Improvements over SFT span EMMA, VisualPuzzles, and MathVision. This gives the integration practical value, though several latent competitors retain stronger GeoLaux answer and process scores.

The training variants show sizeable differences. In Table 2 (p. 8), fixed-latent training reaches 28.77 on VisualPuzzles and 30.33 GeoLaux ACS, versus 39.55 and 44.92 for LatentOPD. Removing warmup reduces GeoLaux ACS to 26.67. These are substantial differences within the reported pipeline, even though the comparison does not yet isolate their cause.

Critical weaknesses

The main methodological advance is not cleanly isolated. According to the manuscript, Monet supplies the decoding interface and privileged acquisition design (Appendices A and C.2, pp. 14 and 16). Vision-OPD already uses privileged visual evidence on student prefixes, but matches token distributions (§2, p. 3). LatentOPD’s main difference is layerwise state distillation on continuous latent prefixes, including updates to their generator. This could be a valuable integration. However, §5.3 describes the fixed-target comparator as matching teacher latent blocks, whereas Appendix C.3, Eqs. 12–13, aligns processed states at every decoder layer. Data, earlier stages, block length, and text loss are held fixed, but matched layers, loss scaling, and gradient routing are not specified. Table 2 therefore does not establish how much comes from student-prefix conditioning rather than a different supervision objective. That distinction is central to the significance of the proposed extension.

The visual-information explanation remains under-tested. Selection retains whole task categories, and observation annotation receives no images (Appendix C.1, p. 15). This does not itself validate that the selected views expose the intermediate evidence the model needs. Appendix E illustrates the intended supervision, but no matched informative-versus-redundant-view comparison is reported. Appendix D.1 correctly distinguishes accessible evidence from additional statistical information; the missing evidence is empirical, not a requirement that transformed images add new information. Likewise, lower cross-sample similarity can reflect textual context, while repeating the first latent state can disrupt generic sequential computation (§5.4, p. 9). These analyses support example-dependent states and trajectory sensitivity, but do not isolate visual content as the reason for the gains.

Questions for authors

Clarification and decisive control: Does the fixed-target variant preserve the same aligned layers and positions, loss coefficients, gradient paths, and training budget? If not, a comparison changing only whether teacher targets use reference or student prefixes would directly test the central claim and could change my recommendation.

Comparison fairness: Were Monet and vanilla SFT trained on the same 35,920 examples with comparable student-update budgets? Were decoding budgets comparable? For the backbone comparison, do the gains persist when both models are evaluated under the same scoring rule? These details would help separate the proposed training benefit from data exposure and evaluation differences.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.