- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
PINT combines a pretrained VLM, a video diffusion model, and a flow-matching Action Expert for robot manipulation. It appends 32 learnable Intent Tokens to the VLM prefix and linearly projects their final-layer states () to condition both the VGM (replacing T5 text embeddings) and the Action Expert. Training runs in two stages: Stage 1 freezes the VGM and updates the VLM, Intent Tokens, projectors, and Action Expert under autoregressive action, video flow, and continuous action losses; Stage 2 freezes the VLM/Intent Tokens and fine-tunes the VGM, projectors, and Action Expert. At test time, default PINT skips future video denoising and caches current-frame VGM K/V features from a single forward pass, while PINT-VA jointly denoises video and actions. Experiments on LIBERO, LIBERO-Plus, RoboTwin 2.0, and 5 real-world ALOHA tasks show strong OOD gains over from-scratch baselines.
Critical strengths
-
Strong OOD results without embodied pretraining.
-
Low inference overhead in the default setup.
-
This paper is clearly written and easy to follow.
Critical weaknesses
-
Missing VGM-free control baseline. The paper credits its OOD gains over Fast-WAM to video-shaped Predictive Intent, yet Fast-WAM has no VLM, and the paper never tests a VGM-free control baseline (Qwen3.5-4B + ActionDiT via with ).
-
Disconnect between the "inference-time world model" claim and default PINT, plus Stage 2 inconsistencies. Unlike prior training-time world models, PINT claims to benefit from retaining the VGM at test time. However, default PINT does not denoise future video at inference. It only extracts K/V features from the current frame , using the 5B VGM as a static visual encoder.
Questions for authors
-
VGM-free VLA baseline: What does a pure VLA baseline (Qwen3.5-4B + ActionDiT conditioned on , trained without the VGM or ) achieve on RoboTwin 2.0 and LIBERO-Plus?
-
Single-pass VGM inference setup: During default PINT's single VGM pass at inference, what is passed for the two future latent time positions and the flow timestep ?
-
Missing comparison with World Tokens and InternVLA-A1.5: Section 2.2 explicitly contrasts PINT against World Tokens (Tang et al., 2026) and InternVLA-A1.5 (Ma et al., 2026), which also use compact foresight tokens from a VLM to condition video and action experts while dropping the VGM at test time.
7 comments
Respond to Reviewer UkhL (2 / 3)
[W2] Role of VGM at Inference
Default PINT performs a single VGM forward pass on the current-frame latent at inference. However, its features are not merely visual: the VGM receives Predictive Intent through its existing cross-attention layers, incorporating task semantics and chunk-level execution intent into its representations. Replacing Predictive Intent with the original instruction for VGM conditioning, while retaining Intent conditioning for the AE, reduces RoboTwin 2.0 success from 68.6%/15.5% to 62.4%/11.1%, and LIBERO/LIBERO-Plus success from 98.0%/79.5% to 95.9%/63.4% (Table 4(a), Appendix D.1).
Additionally, omitting future-video denoising at inference follows a similar principle to standard VLA paradigms: the VLM is trained with next-token prediction (NTP), but the AE directly uses VLM prefix features without costly autoregressive decoding. Similarly, PINT trains the VGM with future-video prediction, while the AE directly uses its Intent-conditioned current-frame features without costly multi-step future-video denoising. NTP shapes VLM prefix representations, whereas future-video prediction shapes Intent-conditioned VGM features with task-relevant physical dynamics. PINT-VA further enables full-video denoising.
To examine the contributions of VGM at different stages, we compare four configurations under matched settings. The training-only VGM configuration corresponds to the w/o VGM-to-AE Features ablation in Table 4(b): the AE does not attend to VGM features, and the VGM is removed at inference.
| VGM Participation | Configuration | RoboTwin 2.0 Easy | RoboTwin 2.0 Hard | Average | Latency (ms) |
|---|---|---|---|---|---|
| No VGM | VLA-only | 51.5 | 12.1 | 31.8 | 261 |
| Training-only VGM | w/o VGM-to-AE Features | 65.9 | 14.6 | 40.3 | 261 |
| Training & Inference VGM (Current-frame) | Default PINT | 68.6 | 15.5 | 42.1 | 331 |
| Training & Inference VGM (Full-video) | PINT-VA | 71.9 | 18.1 | 45.0 | 776 |
All results are success rates (%). Latency is measured under the same hardware and evaluation protocol.
VLA-only and training-only VGM have identical inference latency because neither invokes the VGM at test time. The higher performance of training-only VGM over VLA-only reflects the contribution of VGM during training. Retaining current-frame VGM features provides further gains at modest additional latency, while PINT-VA achieves the highest success through full-video interaction at substantially higher inference cost.
Stage 2. The VLM and Intent Tokens are frozen. The VGM DiT, AE, and both Intent projectors remain trainable. The video VAE is frozen. As explained in [Q2], the current-frame VGM features read by the AE are computed from the same context during Stage 2 training and inference.
(2 / 3)
Respond to Reviewer UkhL (3 / 3)
[Q2] Single-Pass VGM Inference Setup
During default PINT's single VGM pass at inference, what is passed for the two future latent time positions and the flow timestep?
In Default PINT, the VGM receives only the clean VAE-encoded current-frame latent (temporal length 1). The two future latent positions are not instantiated, so no zero or Gaussian-noise placeholders are passed. The VGM flow timestep is fixed to 0 for all current-frame tokens. We perform a single VGM forward pass, cache layer-wise K/V features, and reuse them throughout the AE's 10 action-denoising steps.
The attention mask also prevents current-frame tokens from attending to noisy future latent positions during Stage 2 training. Thus, the current-frame K/V features are computed from the same context during training and Default PINT inference.
[Q3] Comparison with World Tokens and InternVLA-A1.5
Missing comparison with World Tokens and InternVLA-A1.5.
We compare these approaches in terms of Intent construction, training strategy, and inference-time VGM usage:
| Aspect | World Tokens [1] | InternVLA-A1.5 [2] | PINT |
|---|---|---|---|
| Intent Construction | External adapter with 256 world tokens | Foresight tokens in an auxiliary expert | 32 learnable Intent Tokens within the VLM sequence, shared by VGM and AE |
| Training Strategy | Joint VLM–adapter–VGM training | Frozen VGM supervises foresight tokens | Stage 1 freezes VGM to shape Intent; Stage 2 freezes VLM/Intent Tokens and adapts VGM/AE and Intent projectors |
| VGM at Inference | Removed | Removed | Retained for AE, using current-frame features (PINT) or full-video latents (PINT-VA) |
To quantify the effect of retaining the VGM at inference within our framework, the training-only variant (w/o VGM-to-AE Features; see [W2]) achieves 65.9%/14.6% on RoboTwin 2.0 Easy/Hard, compared with 68.6%/15.5% for Default PINT and 71.9%/18.1% for PINT-VA.
References
[1] Q. Tang, B. Zhuang, B. Yuan, X. Yu, L. Guo, and J. Feng, "World Tokens: Enhancing Embodied Policies with Training-Time World Modeling," arXiv preprint arXiv:2608.09730, 2026.
[2] H. Ma, J. Cai, X. Xu, H. Li, Y. Yang, Y. Tian, J. Cao, H. Zhu, Z. Qiu, Zhaxizhuoma, Y. Yang, et al., "InternVLA-A1.5: Unifying Understanding, Latent Foresight, and Action for Compositional Generalization," arXiv preprint arXiv:2607.04988, 2026.
[3] T. Yuan et al., "Fast-WAM: Do World Action Models Need Test-Time Future Imagination?", arXiv preprint arXiv:2603.16666, 2026.
[4] Z. Zhang et al., "Do World Action Models Generalize Better than VLAs? A Robustness Study," arXiv preprint arXiv:2603.22078, 2026.
[5] H. Bi et al., "Motus: A Unified Latent Action World Model," Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2026, pp. 35101–35113.
(3 / 3)
Respond to Reviewer UkhL (1 / 3)
We thank the reviewer for the careful and constructive assessment. We address the weaknesses (W) and questions (Q) below.
[W1, Q1] VGM-Free VLA Baseline
We implement the requested VGM-free VLA baseline using Qwen3.5-4B and ActionDiT, directly conditioning the AE on VLM features zt with the video loss disabled (λvideo=0). It achieves 51.5%/12.1% on RoboTwin 2.0 Easy/Hard and 96.2%/73.6% on LIBERO/LIBERO-Plus.
To further examine whether PINT's gains arise simply from combining VLM and VGM or from how their capabilities are coordinated, we additionally evaluate two unified baselines under matched settings.
Direct Fusion. Following Motus's Tri-model Joint Attention paradigm [5], Direct Fusion projects the VLM's final-layer prefix features into a unified stream (U), which jointly interacts with video (V) and action (A) streams through a 30-layer U-Expert, replacing PINT's dedicated Intent-mediated coordination.
AllPrefix. AllPrefix directly uses the VLM's final-layer prefix features to condition both the VGM and AE, without dedicated Intent Tokens. Both baselines use the same backbones, training data, prediction objectives, two-stage schedule, and optimization budget as PINT. We also include PINT-VA to evaluate the additional benefits of full-video VGM features.
† Our RoboTwin 2.0 reproduction. Fast-WAM's LIBERO and LIBERO-Plus results are from [3] and [4], respectively.
Fast-WAM and VLA-only exhibit complementary performance profiles: Fast-WAM performs strongly on ID settings but degrades substantially under distribution shifts, while VLA-only provides a stronger OOD reference. PINT aims to coordinate the complementary capabilities of the VLM and VGM—semantic reasoning and physical-dynamics modeling—through shared Predictive Intent while preserving their specialized pathways. Direct Fusion does not improve over VLA-only on either OOD benchmark, whereas PINT and PINT-VA outperform VLA-only on both. AllPrefix remains close to VLA-only on RoboTwin Hard.
(1 / 3)