- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper studies latent visual reasoning. It argues that uninformative auxiliary views cause latent tokens to collapse into flat representations, and that fixed latent targets drift away from the student’s own prefixes. LatentOPD introduces three stages: First it trains a privileged teacher and warm-starts the student on the teacher’s latents, and then distills the teacher’s layer-wise states along the student’s own latent prefix. On Qwen2.5-VL-7B, it improves over the base model and SFT on four benchmarks and achieves the highest average among the latent methods compared.
Critical strengths
- The teacher recomputes targets on the student’s own latent prefix, which directly addresses the drift problem. The ablation shows that fixed targets do much worse.
- Large gains over the base model (+9.18) without tools at inference.
Critical weaknesses
- The comparisons are uncontrolled. The baselines are trained on different data, and external scores are mixed with stricter in-house judge scores. There is also no baseline trained on the same data without latents.
- Stage I is essentially Monet, the “informative data” claim is never ablated, and the fixed-latent ablation (28.8) falls below the base model (32.7).
Questions for authors
- How do Monet and LVR perform when retrained on the same data?
- Can you ablate informative vs. low-information auxiliary views?
- Why does the fixed-latent variant underperform the base model?