- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces Uni-LaDiR with 3 pieces: unified encoder built from a single LLM with latent encoding, continuation prediction where tokens are trained to predict later steps and final output, and diffusion reasoner, a flow-matching head generating thought blocks at inference time. Evaluation across VLM reasoning and robotics simulation benchmark demonstrate the effectiveness of proposed method.
Critical strengths
- A general framework across domains and modality:
Uni-LaDiR maps reasoning steps of any modality into a shared latent space learned through continuation prediction, so one formulation covers both VLM reasoning and VLA manipulation. The detailed ablations support the design.
- Broad evaluation with evidence that the latents are used:
Gains hold across 11 VLM benchmarks, two VLA suites, and four backbones. The latent-intervention study shows large drops when the chain is zeroed or shuffled, confirming the model does not bypass its latents.
Critical weaknesses
- Missing controlled baseline and questionable comparisons:
Table 1 has no Qwen2.5-VL baseline fine-tuned on the same Zebra-CoT data without latents. Every baseline uses its own data and recipe, so the gains may largely come from training data close to the evaluation tasks. Some results also need verification. Several baseline scores fall below chance, e.g. Monet at 10.6 and Vision-R1 at 15.2 on VisuLogic. Zebra-cot has subset very similiar to questions VisuLogic while other visual latent baselines may not access during training. I believe a detailed clarification on data usage or comparison under same data budget would strengthen the paper.
- Weak support for the "more modalities, larger gains" claim:
On the VLM side only two modalities (text and image) are unified, so Fig. 3a gives a single comparison, not a trend. On RLBench, the two-modality variants reach 83.5–84.5 and all three reach 87.0 (Fig. 3b). These 1.5–3 point differences fall within the seed std of 3.0–4.0 reported in Table 2, and LIBERO is not included. The abstract's claim of "increasing gains as more teacher modalities are unified" is therefore not well supported.
Questions for authors
See weakness above.
I have some additional questions for authors:
Prior work on visual latent reasoning has shown that latent visual tokens can collapse and be largely ignored by the subsequent reasoning chain [1]. Could the authors provide an explanation or empirical evidence on why the unified formulation avoids this failure mode?
[1] Imagination Helps Visual Reasoning, But Not Yet in Latent Space, ICML 26.
5 comments
Response to Reviewer bnqc (Part 1)
Thank you for the constructive review, and for recognizing the general cross-domain formulation, the breadth of evaluation, and the latent-intervention evidence.
W1(a): Same-data comparison
Thank you for this valuable suggestion. We add two kinds of same-data baselines on Qwen2.5-VL-7B and Zebra-CoT (mean ± SD over 3 seeds): (i) SFT without latents (direct-answer and text-CoT), and (ii) every Table 1 baseline with public code, retrained on Zebra-CoT (§), since several (e.g., Vision-R1, Monet) were originally trained on other data.
| (a) Vision-centric | VisPuz. | ChartQA | V* | BLINK-J | MMVP | SAT | CV-B | Avg. | Table 1 Avg. |
|---|---|---|---|---|---|---|---|---|---|
| Qwen2.5-VL (base) | 34.8 | 78.1 | 76.4 | 59.3 | 77.3 | 51.3 | 75.2 | 64.6 | 64.6 |
| Direct-answer SFT | 38.6±1.1 | 78.8±0.4 | 77.5±1.0 | 61.3±1.1 | 76.0±0.8 | 55.3±1.2 | 77.3±0.5 | 66.4 | – |
| Text-CoT SFT | 41.2±1.2 | 79.6±0.4 | 79.1±1.1 | 63.3±1.2 | 77.7±0.8 | 57.3±1.1 | 78.5±0.5 | 68.1 | – |
| Monet§ | 37.4±1.1 | 79.8±0.5 | 82.9±1.2 | 65.3±1.2 | 61.0±1.0 | 57.3±1.3 | 76.9±0.6 | 65.8 | 62.7 |
| LVR§ | 44.8±1.2 | 80.3±0.4 | 80.5±1.1 | 51.3±1.2 | 70.3±0.9 | 58.7±1.2 | 82.4±0.5 | 66.9 | 68.1 |
| CoVT§ | 42.6±1.2 | 80.3±0.4 | 79.4±1.1 | 67.3±1.1 | 63.0±1.0 | 62.7±1.1 | 81.4±0.5 | 68.1 | 66.7 |
| SLVR§ | 36.9±1.1 | 78.1±0.4 | 82.9±1.1 | 69.3±1.1 | 75.0±0.9 | 64.0±1.2 | 82.4±0.5 | 69.8 | 68.7 |
| Mirage§ | 43.2±1.1 | 79.7±0.4 | 82.2±1.0 | 66.0±1.1 | 72.3±0.9 | 70.0±1.1 | 82.2±0.5 | 70.8 | 72.2 |
| PixelReasoner§ | 45.3±1.1 | 82.0±0.4 | 82.9±1.0 | 71.3±1.2 | 77.0±0.8 | 61.3±1.2 | 83.5±0.5 | 71.9 | 70.7 |
| Mull-Tokens§ | 47.1±1.1 | 80.6±0.4 | 83.2±1.0 | 73.3±1.2 | 75.3±0.8 | 75.3±1.1 | 83.9±0.5 | 74.1 | 74.9 |
| DeepEyes§ | 48.9±1.0 | 83.1±0.3 | 86.0±0.9 | 72.7±1.2 | 79.0±0.8 | 65.3±1.1 | 85.1±0.4 | 74.3 | 73.4 |
| Uni-LaDiR | 57.4±1.0 | 84.2±0.3 | 89.5±0.9 | 75.3±1.1 | 82.0±0.7 | 72.0±1.2 | 88.8±0.4 | 78.5 | 78.5 |
| (b) Math and logic | MathVista | MathVision | VisuLogic | EMMA | Avg. | Table 1 Avg. |
|---|---|---|---|---|---|---|
| Qwen2.5-VL (base) | 68.2 | 25.4 | 26.0 | 26.0 | 36.4 | 36.4 |
| Direct-answer SFT | 67.5±2.2 | 27.8±0.9 | 25.4±1.2 | 27.3±2.2 | 37.0 | – |
| Text-CoT SFT | 69.4±2.1 | 31.6±1.0 | 27.2±1.1 | 28.6±2.1 | 39.2 | – |
| Monet§ | 63.7±2.1 | 33.1±1.1 | 27.2±1.1 | 22.4±2.2 | 36.6 | 32.3 |
| PixelReasoner§ | 68.1±2.1 | 32.0±1.0 | 25.2±1.2 | 21.1±2.3 | 36.6 | 37.5 |
| DeepEyes§ | 70.5±2.1 | 28.9±0.9 | 27.6±1.1 | 30.2±2.2 | 39.3 | 38.4 |
| Vision-R1§ | 72.6±2.2 | 40.1±1.0 | 27.8±1.2 | 25.1±2.1 | 41.4 | 38.5 |
| Mirage§ | 69.5±2.1 | 41.2±0.9 | 27.4±1.1 | 28.3±2.1 | 41.6 | 40.7 |
| ILVR§ | 70.2±2.0 | 47.6±0.8 | 28.4±1.0 | 31.4±2.2 | 44.4 | 45.7 |
| Uni-LaDiR | 73.2±2.0 | 54.3±0.8 | 33.0±1.0 | 35.6±2.2 | 49.0 | 49.0 |
These results show that:
- Same-data SFT gives only modest gains (+0.6 to +3.5 on average), and barely changes VisuLogic (25.4/27.2 vs. 26.0), so the similarity between Zebra-CoT and VisuLogic does not explain our 33.0.
- Retrained baselines stay close to their Table 1 results, and Uni-LaDiR remains best on both groups (+4.2 over DeepEyes§; +4.6 over ILVR§).
W1(b): Below-chance baseline scores
Thank you for checking these numbers carefully. We'd like to clarify that the floor is 0, not 25%. Under our shared protocol (Appendix B.1), models generate free-form text rather than selecting an option, and outputs without a valid option (e.g., truncated or off-format) are scored as incorrect. Retrained on Zebra-CoT, Vision-R1§ and Monet§ reach 27.8 and 27.2 on VisuLogic, and the conclusions are unchanged.
Dear Reviewers,
We would like to kindly follow up on our previous message. We would greatly appreciate it if you could take a moment to review our responses and share any further feedback or remaining concerns.
Thank you for your time and consideration!
Response to Reviewer bnqc (Part 2)
W2: "More modalities, larger gains"
Thank you for this careful analysis. Since Zebra-CoT provides only text and image, the multi-step trend can only be tested on VLA benchmarks. We add the same grouping experiment on LIBERO and test every step with 8 seeds and paired significance tests (mean ± SD):
The SD in Table 2 measures how one method varies across seeds, not the noise of a difference between variants. Since all variants share seeds and initial scenes, we use a paired t-test over the 8 seeds:
These results show that, on both benchmarks, every pair beats separate encoding and all three beat every pair, with every step statistically significant (p < 0.05). We will add these results to Fig. 3 and Section 4.3.
Q: Latent collapse
Thank you for this insightful question. With separate encoding, each modality's latents live in their own space, so the model can lower its loss by relying on the representation it is most familiar with and skipping the rest. In a unified space, all latents are equally familiar, so this shortcut disappears.
To test this, we also apply the interventions of Table 7 to the separate variant:
These results show that the separate variant's latents are largely skipped (it barely changes when they are removed), whereas Uni-LaDiR drops by 22–39 points, so its latents are used.
Thank you for pointing us to this work. The cited ICML 2026 paper's proceedings (PMLR vol. 306) were published on September 29, 2026, after the ICLR full-paper deadline; we will cite and discuss it in the revised PDF.