- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper proposes TraViS, a framework that learns latent visual trajectories with context-adaptive visual targets for multimodal reasoning. During training, each latent state selects task-relevant visual evidence, which a frozen visual teacher encodes into a continuous target to align with the state and guide subsequent latent computation. At inference, TraViS uses self-feedback latent rollout over a single global image encoding, eliminating the need for the crop teacher and per-step target construction. Across multiple backbones and eight multimodal reasoning benchmarks, TraViS achieves an average improvement of over 2.5% while reducing inference latency.
Critical strengths
- Context-adaptive visual supervision. TraViS allows intermediate latent states to select task-relevant visual targets from the evolving multimodal context, addressing the limited flexibility of predefined supervision in latent visual reasoning.
- Efficient inference. By restricting the visual teacher and target construction to training, TraViS supports self-feedback latent reasoning with a single global image encoding, improving performance while reducing inference latency.
- Improvement of empirical results. Experiments across multiple backbones and benchmarks demonstrate consistent improvements exceeding 2.5% on average, while the controlled fixed-crop ablation supports the effectiveness of context-adaptive targeting.
Critical weaknesses
- Fixed rollout length. Using eight latent steps for every question may limit efficiency and flexibility, as discussed in A.9 Limitations.
- The performance improvement compared with PixelReasoner is marginal.
Questions for authors
Please refer to the weaknesses.