acceptodds
Under review as a conference paper at ICLR 2027

PINT: A Unified Model with Shared Predictive Intent for Robot Manipulation

Abstract

Generalist robot manipulation can benefit from pretrained vision-language models (VLMs) for semantic reasoning and video generation models (VGMs) for world modeling. However, existing unified models that directly fuse them through shared attention at multiple layers can make it harder to draw on their specialized capabilities. We introduce PINT, a unified model that coordinates semantic reasoning, world modeling, and action generation through a shared chunk-level Predictive Intent while preserving expert-specific pathways. PINT inserts learnable Intent Tokens into the VLM sequence to form this representation, conditioning both the VGM and an Action Expert (AE). It replaces only the VGM's prompt conditioning and retains the VGM at inference, where the AE reads its representations through one-way attention. Rather than prescribing what Predictive Intent should encode, PINT learns it jointly from the native prediction objectives of the VLM, VGM, and AE, which provide complementary views of the same upcoming task execution. Training uses two stages: Stage 1 freezes the VGM so that future video prediction shapes Predictive Intent with task-relevant future dynamics, while Stage 2 fixes the learned Intent and adapts the VGM and AE. Without embodied pretraining, PINT achieves strong in-distribution (ID) performance and substantially improves out-of-distribution (OOD) generalization, reaching 15.5% on RoboTwin 2.0-Hard compared with 1.8% for Fast-WAM and 3.2% for StarVLA, and 79.5% on LIBERO-Plus compared with 51.5% and 75.0%, respectively. Across five real-world bimanual tasks, PINT reaches 68% ID and 24% OOD success, compared with 35%/9% for pretrained Motus and 67%/35% for pretrained .

Primary area:
applications to robotics, autonomy, planning
Keywords:
vision-language-action models, world-action models, world models, robot learning, robotic manipulation

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.50 over 4 reviews, above 68% of the others.
Reviewer_UkhLOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

PINT combines a pretrained VLM, a video diffusion model, and a flow-matching Action Expert for robot manipulation. It appends 32 learnable Intent Tokens to the VLM prefix and linearly projects their final-layer states (zt\mathbf{z}_t) to condition both the VGM (replacing T5 text embeddings) and the Action Expert. Training runs in two stages: Stage 1 freezes the VGM and updates the VLM, Intent Tokens, projectors, and Action Expert under autoregressive action, video flow, and continuous action losses; Stage 2 freezes the VLM/Intent Tokens and fine-tunes the VGM, projectors, and Action Expert. At test time, default PINT skips future video denoising and caches current-frame VGM K/V features from a single forward pass, while PINT-VA jointly denoises video and actions. Experiments on LIBERO, LIBERO-Plus, RoboTwin 2.0, and 5 real-world ALOHA tasks show strong OOD gains over from-scratch baselines.

Critical strengths
  1. Strong OOD results without embodied pretraining.

  2. Low inference overhead in the default setup.

  3. This paper is clearly written and easy to follow.

Critical weaknesses
  1. Missing VGM-free control baseline. The paper credits its OOD gains over Fast-WAM to video-shaped Predictive Intent, yet Fast-WAM has no VLM, and the paper never tests a VGM-free control baseline (Qwen3.5-4B + ActionDiT via zt\mathbf{z}_t with λvideo=0\lambda_{\text{video}}=0).

  2. Disconnect between the "inference-time world model" claim and default PINT, plus Stage 2 inconsistencies. Unlike prior training-time world models, PINT claims to benefit from retaining the VGM at test time. However, default PINT does not denoise future video at inference. It only extracts K/V features ht\mathbf{h}_t from the current frame Iˉt\bar{\mathbf{I}}_t, using the 5B VGM as a static visual encoder.

Questions for authors
  1. VGM-free VLA baseline: What does a pure VLA baseline (Qwen3.5-4B + ActionDiT conditioned on zt\mathbf{z}_t, trained without the VGM or Lvideo\mathcal{L}_{\text{video}}) achieve on RoboTwin 2.0 and LIBERO-Plus?

  2. Single-pass VGM inference setup: During default PINT's single VGM pass at inference, what is passed for the two future latent time positions and the flow timestep τ\tau?

  3. Missing comparison with World Tokens and InternVLA-A1.5: Section 2.2 explicitly contrasts PINT against World Tokens (Tang et al., 2026) and InternVLA-A1.5 (Ma et al., 2026), which also use compact foresight tokens from a VLM to condition video and action experts while dropping the VGM at test time.

7 comments
AuthorsOpenReview ↗

Respond to Reviewer UkhL (1 / 3)

We thank the reviewer for the careful and constructive assessment. We address the weaknesses (W) and questions (Q) below.


[W1, Q1] VGM-Free VLA Baseline

We implement the requested VGM-free VLA baseline using Qwen3.5-4B and ActionDiT, directly conditioning the AE on VLM features ztz_t with the video loss disabled (λvideo=0\lambda_{\mathrm{video}}=0). It achieves 51.5%/12.1% on RoboTwin 2.0 Easy/Hard and 96.2%/73.6% on LIBERO/LIBERO-Plus.

To further examine whether PINT's gains arise simply from combining VLM and VGM or from how their capabilities are coordinated, we additionally evaluate two unified baselines under matched settings.

Direct Fusion. Following Motus's Tri-model Joint Attention paradigm [5], Direct Fusion projects the VLM's final-layer prefix features into a unified stream (U), which jointly interacts with video (V) and action (A) streams through a 30-layer U-Expert, replacing PINT's dedicated Intent-mediated coordination.

AllPrefix. AllPrefix directly uses the VLM's final-layer prefix features to condition both the VGM and AE, without dedicated Intent Tokens. Both baselines use the same backbones, training data, prediction objectives, two-stage schedule, and optimization budget as PINT. We also include PINT-VA to evaluate the additional benefits of full-video VGM features.

CategoryMethodExpert ComponentsEmbodied PTRoboTwin 2.0 Easy (ID)RoboTwin 2.0 Hard (OOD)RoboTwin 2.0 AverageLIBERO (ID)LIBERO-Plus (OOD)
VLA-onlyVLA baselineVLM + AENo51.512.131.896.273.6
WAM-onlyFast-WAM†VGM + AENo72.41.837.197.6 [3]51.5 [4]
UnifiedDirect FusionVLM + VGM + AENo66.210.738.598.072.6
UnifiedAllPrefixVLM + VGM + AENo56.312.734.5––
UnifiedPINTVLM + VGM + AENo68.615.542.198.079.5
UnifiedPINT-VAVLM + VGM + AENo71.918.145.098.281.0

† Our RoboTwin 2.0 reproduction. Fast-WAM's LIBERO and LIBERO-Plus results are from [3] and [4], respectively.

Fast-WAM and VLA-only exhibit complementary performance profiles: Fast-WAM performs strongly on ID settings but degrades substantially under distribution shifts, while VLA-only provides a stronger OOD reference. PINT aims to coordinate the complementary capabilities of the VLM and VGM—semantic reasoning and physical-dynamics modeling—through shared Predictive Intent while preserving their specialized pathways. Direct Fusion does not improve over VLA-only on either OOD benchmark, whereas PINT and PINT-VA outperform VLA-only on both. AllPrefix remains close to VLA-only on RoboTwin Hard.


(1 / 3)

AuthorsOpenReview ↗

Respond to Reviewer UkhL (2 / 3)


[W2] Role of VGM at Inference

Default PINT performs a single VGM forward pass on the current-frame latent at inference. However, its features are not merely visual: the VGM receives Predictive Intent through its existing cross-attention layers, incorporating task semantics and chunk-level execution intent into its representations. Replacing Predictive Intent with the original instruction for VGM conditioning, while retaining Intent conditioning for the AE, reduces RoboTwin 2.0 success from 68.6%/15.5% to 62.4%/11.1%, and LIBERO/LIBERO-Plus success from 98.0%/79.5% to 95.9%/63.4% (Table 4(a), Appendix D.1).

Additionally, omitting future-video denoising at inference follows a similar principle to standard VLA paradigms: the VLM is trained with next-token prediction (NTP), but the AE directly uses VLM prefix features without costly autoregressive decoding. Similarly, PINT trains the VGM with future-video prediction, while the AE directly uses its Intent-conditioned current-frame features without costly multi-step future-video denoising. NTP shapes VLM prefix representations, whereas future-video prediction shapes Intent-conditioned VGM features with task-relevant physical dynamics. PINT-VA further enables full-video denoising.

To examine the contributions of VGM at different stages, we compare four configurations under matched settings. The training-only VGM configuration corresponds to the w/o VGM-to-AE Features ablation in Table 4(b): the AE does not attend to VGM features, and the VGM is removed at inference.

VGM ParticipationConfigurationRoboTwin 2.0 EasyRoboTwin 2.0 HardAverageLatency (ms)
No VGMVLA-only51.512.131.8261
Training-only VGMw/o VGM-to-AE Features65.914.640.3261
Training & Inference VGM (Current-frame)Default PINT68.615.542.1331
Training & Inference VGM (Full-video)PINT-VA71.918.145.0776

All results are success rates (%). Latency is measured under the same hardware and evaluation protocol.

VLA-only and training-only VGM have identical inference latency because neither invokes the VGM at test time. The higher performance of training-only VGM over VLA-only reflects the contribution of VGM during training. Retaining current-frame VGM features provides further gains at modest additional latency, while PINT-VA achieves the highest success through full-video interaction at substantially higher inference cost.

Stage 2. The VLM and Intent Tokens are frozen. The VGM DiT, AE, and both Intent projectors remain trainable. The video VAE is frozen. As explained in [Q2], the current-frame VGM features read by the AE are computed from the same context during Stage 2 training and inference.


(2 / 3)

AuthorsOpenReview ↗

Respond to Reviewer UkhL (3 / 3)


[Q2] Single-Pass VGM Inference Setup

During default PINT's single VGM pass at inference, what is passed for the two future latent time positions and the flow timestep?

In Default PINT, the VGM receives only the clean VAE-encoded current-frame latent (temporal length 1). The two future latent positions are not instantiated, so no zero or Gaussian-noise placeholders are passed. The VGM flow timestep is fixed to 0 for all current-frame tokens. We perform a single VGM forward pass, cache layer-wise K/V features, and reuse them throughout the AE's 10 action-denoising steps.

The attention mask also prevents current-frame tokens from attending to noisy future latent positions during Stage 2 training. Thus, the current-frame K/V features are computed from the same context during training and Default PINT inference.


[Q3] Comparison with World Tokens and InternVLA-A1.5

Missing comparison with World Tokens and InternVLA-A1.5.

We compare these approaches in terms of Intent construction, training strategy, and inference-time VGM usage:

AspectWorld Tokens [1]InternVLA-A1.5 [2]PINT
Intent ConstructionExternal adapter with 256 world tokensForesight tokens in an auxiliary expert32 learnable Intent Tokens within the VLM sequence, shared by VGM and AE
Training StrategyJoint VLM–adapter–VGM trainingFrozen VGM supervises foresight tokensStage 1 freezes VGM to shape Intent; Stage 2 freezes VLM/Intent Tokens and adapts VGM/AE and Intent projectors
VGM at InferenceRemovedRemovedRetained for AE, using current-frame features (PINT) or full-video latents (PINT-VA)

To quantify the effect of retaining the VGM at inference within our framework, the training-only variant (w/o VGM-to-AE Features; see [W2]) achieves 65.9%/14.6% on RoboTwin 2.0 Easy/Hard, compared with 68.6%/15.5% for Default PINT and 71.9%/18.1% for PINT-VA.


References

[1] Q. Tang, B. Zhuang, B. Yuan, X. Yu, L. Guo, and J. Feng, "World Tokens: Enhancing Embodied Policies with Training-Time World Modeling," arXiv preprint arXiv:2608.09730, 2026.

[2] H. Ma, J. Cai, X. Xu, H. Li, Y. Yang, Y. Tian, J. Cao, H. Zhu, Z. Qiu, Zhaxizhuoma, Y. Yang, et al., "InternVLA-A1.5: Unifying Understanding, Latent Foresight, and Action for Compositional Generalization," arXiv preprint arXiv:2607.04988, 2026.

[3] T. Yuan et al., "Fast-WAM: Do World Action Models Need Test-Time Future Imagination?", arXiv preprint arXiv:2603.16666, 2026.

[4] Z. Zhang et al., "Do World Action Models Generalize Better than VLAs? A Robustness Study," arXiv preprint arXiv:2603.22078, 2026.

[5] H. Bi et al., "Motus: A Unified Latent Action World Model," Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2026, pp. 35101–35113.


(3 / 3)

Reviewer_FUdbOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses a timely problem: how to combine two complementary pretrained experts - a vision-language model (VLM) for semantic reasoning and a video generation model (VGM) for world modeling - with an action expert (AE) for control. The authors argue that existing unified models fuse these experts through shared attention at multiple layers, which forces each expert's specialized representation to also serve cross-expert coordination and can undermine its pretrained strengths. PINT instead coordinates the three experts through a single shared, chunk-level Predictive Intent: learnable Intent Tokens are appended to the VLM prefix, and their final hidden states condition both the VGM and the AE. Without embodied pretraining, PINT reports strong in-distribution results and clear out-of-distribution (OOD) gains across LIBERO/LIBERO-Plus, RoboTwin 2.0, and five real-world bimanual tasks.

Critical strengths
  • Unifying semantic reasoning, world modeling, and action generation is one of the most active directions in embodied AI right now, and the paper distills the central design question cleanly: what should be shared for coordination versus preserved for specialization.
  • Coordinating all three experts through one chunk-level Predictive Intent, learned jointly from the experts' native objectives and shaped by a two-stage schedule, is a fresh angle relative to prior unified models. The design is also practical.
  • The evaluation spans LIBERO, LIBERO-Plus (seven distribution shifts), RoboTwin 2.0 (Easy/Hard), and five real-world bimanual tasks, with ablations, per-task results, repeated evaluations, and latency measurements. The OOD improvements are substantial.
Critical weaknesses
  • The paper's premise is that multi-layer shared-attention fusion makes it harder for pretrained experts to use their specialized capabilities, but no matched shared-attention fusion baseline is trained under the same data, backbone, and budget, so this claim is never directly tested. Besides, the assumption that Predictive Intent captures task-execution intent lacks quantitative probe (e.g., does the Intent predict the future action chunk, decode the chosen subgoal, or correlate with success?). As a result, it is hard to rule out that the Intent simply encodes task-level semantics rather than the intended execution.
  • Most baseline numbers are taken from third-party papers or leaderboards rather than reproduced, and the compared methods differ in VLM backbone, training data, and embodied pretraining. This makes it difficult to attribute the reported gains to the proposed design rather than to these confounds.
  • The design introduces several free choices like intent capacity, loss weights, the number of projection layers, and the training schedule, but there lacks corresponding ablation study.
Questions for authors
  • Could you train a matched shared-attention fusion baseline under the same backbones, data, and compute, and report both ID and OOD results?
  • Why is Stage-1-only worse than single-stage training, given that Stage 1 is meant to shape the Intent with future dynamics? What does this reveal about the actual role of the frozen-VGM stage?
5 comments
AuthorsOpenReview ↗

Respond to Reviewer FUdb (1 / 3)

We thank the reviewer for the careful and constructive assessment. We address the weaknesses (W) and questions (Q) below.


[W1.1, Q1] Controlled Comparisons of Expert Interaction

We implement the requested matched shared-attention baseline (Direct Fusion) and additionally evaluate AllPrefix as a simpler direct-conditioning baseline.

Direct Fusion. Following Motus's Tri-model Joint Attention paradigm [3], Direct Fusion projects the VLM's final-layer prefix features into a unified stream (U), which jointly interacts with video (V) and action (A) streams through a 30-layer U-Expert, replacing PINT's dedicated Intent-mediated coordination. Apart from the expert-interaction architecture, both methods use the same Qwen3.5-4B and Wan2.2-5B backbones, AE architecture, training data, prediction objectives, and two-stage schedule, with matched training steps, batch size, optimizer, and learning-rate schedule. For RoboTwin 2.0, both use the same 2,500 clean demonstrations (50 per task). For LIBERO, both use the same 50 demonstrations per task across the four standard suites, and each method's own checkpoint is evaluated on LIBERO-Plus without additional training.

MethodRoboTwin 2.0 Easy (ID)RoboTwin 2.0 Hard (OOD)LIBERO (ID)LIBERO-Plus (OOD)
Direct Fusion66.210.798.072.6
PINT68.615.598.079.5

AllPrefix. As a simpler direct-conditioning baseline, AllPrefix bypasses dedicated Intent Tokens and directly uses the VLM's final-layer multimodal prefix features to condition both the VGM and AE. It retains the same expert architectures, projectors, prediction objectives, two-stage training schedule, data, and optimization budget as PINT.

MethodRoboTwin 2.0 Easy (ID)RoboTwin 2.0 Hard (OOD)
AllPrefix56.312.7
PINT68.615.5

PINT matches Direct Fusion on LIBERO while outperforming it on RoboTwin 2.0 and LIBERO-Plus. It also outperforms AllPrefix on both RoboTwin 2.0 settings, supporting the benefits of Intent-mediated coordination over direct expert fusion and full-prefix conditioning.


(1 / 3)

AuthorsOpenReview ↗

Respond to Reviewer FUdb (2 / 3)


[W1.2] Quantitative Analysis of Predictive Intent

Does the Intent predict the future action chunk?

Within-task representational similarity analysis (RSA) on 47,630 valid samples across 50 tasks shows that Predictive Intent is more strongly aligned with future action chunks than the VLM's full multimodal prefix features (Spearman ρ: 0.74 vs. 0.25). For pairs of samples from different episodes of the same task, we compare distances in Predictive Intent space with distances between the corresponding future action chunks, and repeat this with the VLM's prefix features. The difference persists after controlling for current proprioceptive-state distance (partial ρ: 0.64 vs. 0.12).

It is hard to rule out that the Intent simply encodes task-level semantics rather than the intended execution.

Within-trajectory permutation of complete Intent representations reduces closed-loop success from 50/50 to 0/50. We randomly select five tasks and 10 initially successful trajectories per task, permute Intent across timesteps while keeping observations, instructions, and proprioceptive states unchanged, and recompute the corresponding VGM features before closed-loop evaluation. If Intent encoded only task-level semantics, such a within-trajectory permutation would be expected to have little effect because task identity and instructions remain unchanged.

Together, these results indicate that Predictive Intent captures chunk-level, execution-related temporal structure beyond task-level semantics.


[W2] Baseline Comparability

External baselines provide broader performance context, while our conclusions about expert interaction rely primarily on controlled comparisons within the same training framework. All configurations except Fast-WAM are implemented and trained using our PINT training framework, with expert components and interaction designs varied as indicated. Fast-WAM's LIBERO and LIBERO-Plus results are taken from the original paper [1] and a separate published evaluation [2], respectively.

CategoryMethodExpert ComponentsEmbodied PTRoboTwin 2.0 EasyRoboTwin 2.0 HardRoboTwin 2.0 AverageLIBEROLIBERO-Plus
VLA-onlyVLA baselineVLM + AENo51.512.131.896.273.6
WAM-onlyFast-WAM†VGM + AENo72.41.837.197.6 [1]51.5 [2]
UnifiedDirect FusionVLM + VGM + AENo66.210.738.598.072.6
UnifiedAllPrefixVLM + VGM + AENo56.312.734.5––
UnifiedPINTVLM + VGM + AENo68.615.542.198.079.5
UnifiedPINT-VAVLM + VGM + AENo71.918.145.098.281.0

† Our RoboTwin 2.0 reproduction.

All unified models use the same Qwen3.5-4B and Wan2.2-5B backbones, training data, prediction objectives, and optimization budget, without embodied pretraining.

Among the models trained within our framework, PINT outperforms VLA-only, Direct Fusion, and AllPrefix on RoboTwin 2.0 average success; PINT-VA further improves performance. These controlled comparisons support the benefits of unified expert modeling and Intent-mediated coordination.


(2 / 3)

AuthorsOpenReview ↗

Respond to Reviewer FUdb (3 / 3)


[W3] Ablation of Design Choices

We conduct controlled ablations on RoboTwin 2.0. All variants use the same Qwen3.5-4B and Wan2.2-5B backbones and the same 2,500 clean demonstrations. Each comparison changes only the specified design choice, while keeping the remaining configurations and evaluation protocol unchanged. The original paper reports four groups of ablations (Table 4(a)–(d)); we additionally examine video loss weights and projector depth.

Design ChoiceSettingEasyHardDesign ChoiceSettingEasyHard
Intent DesignPrompt-conditioned VGM62.411.1Supervision & Interactionw/o Stage 1 Video Loss63.211.5
Split Intent65.111.8w/o VGM-to-AE Features65.914.6
Shared Intent (PINT)68.615.5w/o AR Loss67.312.2
Training ScheduleStage 1 Only58.513.1PINT68.615.5
Single-Stage65.514.4Video Loss Weight0.0562.013.5
Two-Stage (PINT)68.615.50.1 (PINT)68.615.5
Intent Capacity16 Tokens67.915.00.266.612.1
32 Tokens (PINT)68.615.5Projector Depth2-layer MLP67.514.9
64 Tokens67.114.7Linear (PINT)68.615.5

All results are success rates (%).

Performance remains relatively stable across the tested Intent capacities and projector depths, while the two-stage schedule and Stage 1 video supervision provide clear benefits. Among the tested video-loss weights, 0.1 achieves the best performance. Removing autoregressive supervision or VGM-to-AE features also reduces performance, supporting their contributions to PINT.


[Q2] Role of the Frozen-VGM Stage

Stage 1 Only is the intermediate checkpoint after 5 epochs, whereas Single-Stage jointly trains the VLM, Intent Tokens, VGM, and AE for 10 epochs. The lower performance of Stage 1 Only is expected because it uses only half the training budget and the VGM remains frozen, preventing it from adapting its generation capability to robot data.

The purpose of freezing the VGM in Stage 1 is to ensure that future-video supervision shapes Predictive Intent through the Intent-conditioning pathway, rather than being absorbed by updates to the VGM itself. In Single-Stage, the VGM may absorb part of the video-prediction objective through its own parameter updates, reducing the pressure on Intent to encode future dynamics. PINT separates these processes by first learning Intent with a frozen VGM and then freezing the VLM and Intent Tokens while adapting the VGM, AE, and Intent projectors. With the same 10-epoch budget, PINT outperforms Single-Stage (68.6%/15.5% vs. 65.5%/14.4%). As shown in Fig. 3, Stage 1 already captures coarse task-directed dynamics, while Stage 2 further refines motion generation.


References

[1] T. Yuan et al., "Fast-WAM: Do World Action Models Need Test-Time Future Imagination?", arXiv:2603.16666, 2026.

[2] Z. Zhang et al., "Do World Action Models Generalize Better than VLAs? A Robustness Study," arXiv:2603.22078, 2026.

[3] H. Bi et al., "Motus: A Unified Latent Action World Model," Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2026, pp. 35101–35113.


(3 / 3)

Reviewer_cW4XOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes PINT, a unified robot manipulation model that uses learnable Intent Tokens within a VLM to construct a shared Predictive Intent for conditioning a video generation model (VGM) and an action expert. A two-stage training strategy first learns this representation with the VGM frozen, then adapts the VGM and action expert while freezing the VLM and Intent Tokens. Experiments cover LIBERO, LIBERO-Plus, RoboTwin 2.0, and real-world bimanual manipulation. Without embodied pretraining, PINT achieves 98.0% success on LIBERO and 79.5% on LIBERO-Plus, and improves RoboTwin 2.0 Hard success from the reproduced Fast-WAM baseline’s 1.8% to 15.5%. On real-world tasks, it achieves 68% ID and 24% OOD success, outperforming Motus while remaining below π0.5 under OOD evaluation.

Critical strengths
  1. The paper addresses an important problem: how to coordinate the semantic capabilities of pretrained VLMs with the dynamics priors of VGMs for robot manipulation. The architecture is clearly presented, and shared conditioning, one-way expert interaction, and staged optimization form a coherent approach.
  2. The improvements under environmental distribution shifts are encouraging. Without embodied pretraining, PINT achieves 15.5% success on RoboTwin 2.0 Hard, compared with 1.8% for the authors’ reproduction of Fast-WAM. It also achieves 79.5% on LIBERO-Plus, outperforming several baselines without embodied pretraining.
  3. The experimental evaluation is broad and well documented. The paper includes simulation and real-world experiments, ablations of the main design choices, per-task results, baseline result provenance, and inference latency comparisons under a common hardware configuration.
Critical weaknesses
  1. The evidence supporting the paper’s central motivation remains indirect. The authors argue that direct fusion may interfere with expert specialization, motivating coordination through shared Predictive Intent. However, comparisons with Motus involve differences in pretraining data, model configurations, and training strategies, making it difficult to attribute the performance gap specifically to the coordination mechanism. While the ablations support individual design choices within PINT, a controlled comparison against a representative direct-fusion architecture with matched backbones, training data, and optimization budgets would more convincingly establish the proposed mechanism’s advantage.
  2. The reported improvements would be more convincing with confidence intervals or multi-seed experiments.
  3. Could the authors explain why the following experiment was not run: identical backbones (Qwen3.5-4B + Wan2.2-5B) and identical training data (the 2,500 RoboTwin 2.0 clean demonstrations), with multi-layer shared attention as the only variable against the Intent bottleneck? This appears to be the one comparison that would directly isolate the paper's central claim.
  4. Relatedly, I find the Motus numbers in Table 3 hard to reconcile. Motus uses embodied pretraining yet reaches only 35% ID — 33 points below PINT, which uses none — and far below π0.5's 67%. A gap of this magnitude seems anomalous, and I suspect it reflects insufficiently tuned fine-tuning of the Motus baseline rather than an intrinsic limitation of direct fusion.
Questions for authors

Please see the weakness section.

3 comments
AuthorsOpenReview ↗

Respond to Reviewer cW4X (1 / 2)

We thank the reviewer for the careful and constructive assessment. We address the weaknesses (W) below.


[W1, W3] Controlled Comparisons of Expert Interaction

We ran the requested matched Direct Fusion comparison and additionally evaluated AllPrefix as a simpler direct-conditioning baseline.

Direct Fusion. Following Motus's Tri-model Joint Attention paradigm [2], Direct Fusion projects the VLM's final-layer prefix features into a unified stream (U), which jointly interacts with video (V) and action (A) streams through a 30-layer U-Expert, replacing PINT's dedicated Intent-mediated coordination. Apart from the expert-interaction architecture, both methods use the same Qwen3.5-4B and Wan2.2-5B backbones, AE architecture, training data, two-stage schedule, prediction objectives, and optimization budget. For RoboTwin 2.0, both are trained on the same 2,500 clean demonstrations (50 per task). For LIBERO, both use 50 demonstrations per task across the four standard suites (Spatial, Object, Goal, and Long). Each method's own LIBERO-trained checkpoint is then evaluated on LIBERO-Plus without additional training.

MethodRoboTwin 2.0 EasyRoboTwin 2.0 HardLIBEROLIBERO-Plus
Direct Fusion66.210.798.072.6
PINT68.615.598.079.5

AllPrefix. Whereas Direct Fusion tests multi-layer shared attention, AllPrefix provides a simpler direct-conditioning alternative. It bypasses dedicated Intent Tokens and directly uses the VLM's final-layer multimodal prefix features to condition both the VGM and AE. AllPrefix retains the same expert architectures, projectors, prediction objectives, two-stage training schedule, training data, and optimization budget as PINT.

MethodRoboTwin 2.0 EasyRoboTwin 2.0 Hard
AllPrefix56.312.7
PINT68.615.5

PINT matches Direct Fusion on LIBERO and outperforms it on RoboTwin 2.0 and LIBERO-Plus, with the largest margins under distribution shift (+4.8 points on RoboTwin 2.0 Hard, +6.9 on LIBERO-Plus). It also outperforms AllPrefix on both RoboTwin 2.0 settings, supporting the benefits of Intent-mediated coordination over direct expert fusion and full-prefix conditioning.


(1 / 2)

AuthorsOpenReview ↗

Respond to Reviewer cW4X (2 / 2)


[W2] Statistical Reliability

We evaluate three independently trained two-stage PINT models on LIBERO and LIBERO-Plus, reporting mean ± sample standard deviation. For RoboTwin 2.0, we additionally report repeated evaluations of the same checkpoint using three evaluation seeds. All values are success rates (%).

BenchmarkProtocolRun 1Run 2Run 3Mean ± Std.
LIBEROTraining seeds98.497.698.298.07 ± 0.42
LIBERO-PlusTraining seeds79.779.479.579.53 ± 0.15
RoboTwin 2.0 EasyEvaluation seeds68.3468.6668.9068.63 ± 0.28
RoboTwin 2.0 HardEvaluation seeds15.0815.5215.8615.49 ± 0.39

These results show consistent performance across independent training runs and repeated evaluations.


[W4] Motus Comparison

The Motus results reported in Table 3 are from our own real-world evaluation, where Motus achieves 35% ID and 9% OOD success. We fine-tuned Motus from its official pretrained checkpoint using the released ALOHA-Agilex-2 configuration [1], retaining its default optimization hyperparameters, including a learning rate of 5e-5, weight decay of 0.01, and a linear scheduler. The fine-tuning steps were matched across Motus, PINT, and π0.5. All three methods were fine-tuned on the same 700 real-world demonstrations and evaluated under the same ID/OOD protocol (Appendices B.3 and C.2).

Independent real-world evaluations also report lower Motus performance than π0.5, although with smaller gaps than in our setting. FACT reports 88% vs. 64% on seen tasks and 85% vs. 62% on unseen variants [3]. WLA-0 reports π0.5/Motus mean successes of 7.0/6.0, 4.5/3.75, and 4.75/4.0 out of 10 across standard, OOD-object, and OOD-scenario settings, respectively [4].

We do not attribute the real-world performance gap between Motus and PINT solely to the expert-interaction mechanism, given their different pretrained architectures and training backgrounds. Instead, we use the controlled Direct Fusion comparison in [W1] to evaluate this mechanism. Under matched settings, PINT outperforms Direct Fusion on RoboTwin 2.0 Hard (15.5% vs. 10.7%) and LIBERO-Plus (79.5% vs. 72.6%), supporting the advantage of Intent-mediated coordination over direct multi-expert fusion.


References

[1] THU-ML, "Motus: A Unified Latent Action World Model," Official GitHub Repository, configs/aloha_agilex_2.yaml.

[2] H. Bi et al., "Motus: A Unified Latent Action World Model," Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2026, pp. 35101–35113.

[3] Q. Peng, Y. Liang, R. Yan, N. Hansen, and X. Wang, "FACT: Failure-Aware Causal Training for World-Action Models," arXiv preprint arXiv:2608.10232, 2026.

[4] Y. Yang et al., "World-Language-Action Model for Unified World Modeling, Language Reasoning, and Action Synthesis," arXiv preprint arXiv:2606.05979, 2026.


(2 / 2)

Area_Chair_mDRAOpenReview ↗

Please take some time to review the authors' rebuttal and actively participate in the discussion

Dear Reviewers,

This is a friendly reminder that the rebuttal discussion period is approaching its deadline.

Please take some time to review the authors' rebuttal and actively participate in the discussion. If you have any remaining questions or concerns, please raise them and share your feedback with the authors and other reviewers.

As the discussion period is about to close, we kindly ask you to complete your discussion and provide your updated assessment before the deadline.

Thank you for your time and effort.

Best regards, Area Chair

Reviewer_p2HxOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes PINT, a unified robot manipulation framework that combines a pretrained VLM, video generation model (VGM), and action expert through a shared Predictive Intent representation. Rather than directly fusing the experts, PINT uses learnable tokens within the VLM to produce a representation that conditions both video and action generation. A two-stage training strategy first learns this representation with the VGM frozen, then adapts the VGM and action expert while keeping the learned representation fixed. At inference, the action expert also attends to features from the VGM. The method is evaluated on LIBERO, LIBERO-Plus, RoboTwin 2.0, and five real-world bimanual manipulation tasks. The results show competitive in-distribution performance and improved OOD generalization without embodied pretraining. Ablations further examine the contributions of Predictive Intent, the training objectives, and the two-stage training strategy.

Critical strengths
  1. Strong OOD generalization across multiple benchmarks. PINT achieves substantial improvements over Fast-WAM on LIBERO-Plus (79.5% vs. 51.5%) and RoboTwin 2.0-Hard (15.5% vs. 1.8%), while maintaining competitive in-distribution performance. The real-world experiments also show encouraging results across five bimanual manipulation tasks. These results suggest that the proposed architecture is more effective in handling distribution shifts compared with the reported baselines.

  2. Reasonable architectural design with supporting ablations. Coordinating pretrained VLM and VGM experts through a shared intermediate representation, rather than directly fusing their internal representations, is a sensible design choice. The ablations cover the Intent design, supervision objectives, expert interaction, and two-stage training strategy. The consistent performance drops when removing these components provide useful evidence for the effectiveness of the overall design.

Critical weaknesses
  1. Limited methodological novelty over closely related work. The central idea of learning intermediate tokens from a VLM to condition both future prediction and action generation is closely related to existing approaches such as World Tokens and InternVLA-A1.5. PINT mainly differs in how these tokens are constructed, the two-stage optimization schedule, and the interaction between the VGM and action expert. While these design choices are reasonable, it is unclear whether they constitute a sufficiently substantial methodological contribution beyond existing predictive-token approaches.

  2. Insufficient evidence for the central motivation and proposed mechanism. A major motivation of PINT is that direct attention fusion may interfere with the specialized capabilities of pretrained experts. However, the paper does not provide a controlled comparison against direct fusion using the same backbones, data, and training budget. Moreover, the prompt-conditioned VGM ablation changes both the conditioning source and the learned representation, making it difficult to isolate the benefit of shared Predictive Intent. Removing Stage 1 video supervision only reduces LIBERO-Plus success from 79.5% to 75.2%, suggesting that much of the improvement may come from stronger VLM conditioning rather than the proposed predictive representation itself. These limitations weaken the evidence supporting the paper's main technical claims.

Questions for authors
  1. Can you provide a controlled comparison of the three simulation benchmarks with an appropriate direct-fusion architecture using the same pretrained VLM/VGM backbones, training data, and optimization budget? This would help establish whether preserving expert-specific pathways is indeed responsible for the improved generalization.

  2. The existing ablations demonstrate the benefits of Stage 1 video supervision and shared Intent Tokens. However, it remains unclear whether introducing dedicated Predictive Intent Tokens is necessary, or whether similar improvements could be achieved by directly conditioning the VGM and action expert on standard VLM features. Could you provide a controlled baseline using conventional VLM feature conditioning, while keeping the same prediction objectives, expert architectures, and training budget? This would help isolate the contribution of the proposed Intent representation from the benefits of VLM-derived conditioning itself.

  3. The ablation without VGM-to-AE features achieves 14.6% on RoboTwin Hard, compared with 15.5% for full PINT. Could you report the performance and inference latency when the VGM is completely removed at inference? This seems important for justifying the decision to retain the relatively expensive VGM during deployment.

9 comments
AuthorsOpenReview ↗

Respond to Reviewer p2Hx (1 / 3)

We thank the reviewer for the careful and constructive assessment. We address the weaknesses (W) and questions (Q) below.


[W1] Methodological Differences from Existing Predictive-Token Approaches

PINT shares the predictive-token paradigm with recent approaches such as World Tokens [1] and InternVLA-A1.5 [2]. Its contribution, however, lies in how a shared, chunk-level Predictive Intent is constructed, learned, and used to coordinate the three experts.

AspectWorld Tokens [1]InternVLA-A1.5 [2]PINT
Intent ConstructionExternal adapter with 256 world tokensForesight tokens in an auxiliary expert32 learnable Intent Tokens within the VLM sequence, shared by VGM and AE
Training StrategyJoint VLM–adapter–VGM trainingFrozen VGM supervises foresight tokensStage 1 freezes VGM to shape Intent; Stage 2 freezes VLM/Intent Tokens and adapts VGM/AE and Intent projectors
VGM at InferenceRemovedRemovedRetained for AE, using current-frame features (PINT) or full-video latents (PINT-VA)

Our ablations support the contributions of these designs. Relative to PINT, Split Intent, single-stage training, removal of Stage 1 video supervision, and removal of VGM-to-AE features reduce RoboTwin 2.0 Easy/Hard success by 3.5/3.7, 3.1/1.1, 5.4/4.0, and 2.7/0.9 points, respectively (Table 4(a)–(c)).


[W2.1, Q1] Controlled Comparison with Direct Fusion

Can you provide a controlled comparison of the three simulation benchmarks with an appropriate direct-fusion architecture using the same pretrained VLM/VGM backbones, training data, and optimization budget?

We ran the requested matched Direct Fusion comparison. Following Motus's Tri-model Joint Attention paradigm [3], Direct Fusion projects the VLM's final-layer prefix features into a unified stream (U), which jointly interacts with video (V) and action (A) streams through a 30-layer U-Expert, replacing PINT's dedicated Intent-mediated coordination. Both methods use the same Qwen3.5-4B and Wan2.2-5B backbones, training data, prediction objectives, two-stage schedule, and optimization budget.

MethodRoboTwin 2.0 EasyRoboTwin 2.0 HardLIBEROLIBERO-Plus
Direct Fusion66.210.798.072.6
PINT68.615.598.079.5

PINT is comparable to Direct Fusion on ID settings (+2.4 points on RoboTwin 2.0 Easy and identical performance on LIBERO), but achieves larger gains under distribution shifts (+4.8 points on RoboTwin 2.0 Hard and +6.9 points on LIBERO-Plus). This supports the benefit of Intent-mediated coordination for OOD generalization.


(1 / 3)

AuthorsOpenReview ↗

Respond to Reviewer p2Hx (2 / 3)


[W2.2, Q2] Necessity of Dedicated Predictive Intent

We add the requested AllPrefix baseline and summarize it alongside the VLA-only baseline and relevant ablations from Table 4 and Appendix D.1.

ConfigurationConditioning / SupervisionRoboTwin 2.0 EasyRoboTwin 2.0 HardLIBERO-Plus
VLA-onlyVLM + AE, without VGM51.512.173.6
AllPrefixFull VLM prefix conditions VGM and AE56.312.7–
Prompt-conditioned VGMOriginal instruction conditions VGM; AE retains Intent62.411.163.4
w/o Stage 1 Video LossIntent conditioning retained; Stage 1 video loss removed63.211.575.2
PINTShared Intent with Stage 1 video supervision68.615.579.5

AllPrefix was evaluated on RoboTwin 2.0 only.

Could you provide a controlled baseline using conventional VLM feature conditioning, while keeping the same prediction objectives, expert architectures, and training budget?

AllPrefix does not reproduce PINT's gains: it replaces dedicated Intent Tokens with the VLM's final-layer prefix features to condition both the VGM and AE, while retaining the same expert architectures, projectors, prediction objectives, two-stage schedule, training data, and optimization budget. Its Hard performance (12.7%) remains close to VLA-only (12.1%) and below PINT (15.5%); PINT also achieves higher Easy success (68.6% vs. 56.3%).

Removing Stage 1 video supervision only reduces LIBERO-Plus success from 79.5% to 75.2%, suggesting that much of the improvement may come from stronger VLM conditioning rather than the proposed predictive representation itself.

VLM conditioning provides a strong baseline, but it does not explain PINT's full OOD gain: relative to VLA-only (73.6%), PINT gains 5.9 points on LIBERO-Plus, whereas the gain falls to 1.6 points without Stage 1 video supervision (75.2%). The w/o Stage 1 Video Loss ablation retains Intent conditioning for both experts and removes only Stage 1 video supervision. We use this ablation to assess its contribution because the prompt-conditioned VGM ablation changes both the VGM's conditioning source and the pathway through which video supervision shapes Intent. Neither Direct Fusion ([W2.1]) nor prompt-conditioned VGM outperforms VLA-only on either OOD benchmark, whereas PINT does.

Representation analysis. Within-task representational similarity analysis (RSA) on 47,630 valid samples across 50 tasks shows that Predictive Intent is more strongly aligned with future action chunks than the VLM's full multimodal prefix features are (Spearman ρ: 0.74 vs. 0.25). This difference persists after controlling for current proprioceptive-state distance (partial ρ: 0.64 vs. 0.12). Since instructions are shared within each task, this alignment supports the interpretation that Intent captures execution-related information beyond task identity.


(2 / 3)

AuthorsOpenReview ↗

Respond to Reviewer p2Hx (3 / 3)


[Q3] VGM-Free Inference and Efficiency

Could you report the performance and inference latency when the VGM is completely removed at inference?

The w/o VGM-to-AE Features variant in Table 4(b) is the training-only VGM configuration: the AE never attends to VGM features, so the VGM can be completely removed at inference. It achieves 65.9%/14.6% Easy/Hard success at 261 ms, compared with 68.6%/15.5% at 331 ms for Default PINT. We compare four configurations to distinguish the contributions of VGM during training and inference:

VGM ParticipationConfigurationRoboTwin 2.0 Easy ↑RoboTwin 2.0 Hard ↑Average ↑Latency (ms) ↓
No VGMVLA-only51.512.131.8261
Training-only VGMw/o VGM-to-AE Features65.914.640.3261
Training & Inference VGM (Current-frame)Default PINT68.615.542.1331
Training & Inference VGM (Full-video)PINT-VA71.918.145.0776

All results are success rates (%). Latency follows the measurement protocol in Appendix D.3.

VLA-only and training-only VGM have identical inference latency because neither invokes the VGM at test time. The higher performance of training-only VGM over VLA-only shows the contribution of VGM during training; retaining current-frame VGM features provides further gains at modest additional latency, while PINT-VA further improves performance through full-video denoising at higher cost.


References

[1] Q. Tang, B. Zhuang, B. Yuan, X. Yu, L. Guo, and J. Feng, "World Tokens: Enhancing Embodied Policies with Training-Time World Modeling," arXiv preprint arXiv:2608.09730, 2026.

[2] H. Ma, J. Cai, X. Xu, H. Li, Y. Yang, Y. Tian, J. Cao, H. Zhu, Z. Qiu, Zhaxizhuoma, Y. Yang, et al., "InternVLA-A1.5: Unifying Understanding, Latent Foresight, and Action for Compositional Generalization," arXiv preprint arXiv:2607.04988, 2026.

[3] H. Bi et al., "Motus: A Unified Latent Action World Model," Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2026, pp. 35101–35113.


(3 / 3)

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.