acceptodds
Under review as a conference paper at ICLR 2027

IronMan: Information-Constrained Video-Action Learning for Robot Manipulation

Abstract

Video Action Models (VAMs) couple visual dynamics modeling with action generation for robot manipulation. However, video representations are not naturally suited to action generation, as exposing the action policy to excessive visual detail can impair its generalization ability. Therefore, we introduce **IronMan** (**I**nformation-const**R**ained vide**O**-actio**N** learning for robot **MAN**ipulation), a robust video-action learning framework built on information bottleneck principle. The core principle of this framework is to impose information constraints that suppress irrelevant visual information while preserving action-relevant dynamics cues. IronMan employs a dynamics-aware bottleneck that distills noisy, entangled one-step video features into compact world representations. Extensive simulation and real-world experiments demonstrate strong in-distribution (ID) performance and out-of-distribution (OOD) robustness while maintaining efficient inference. IronMan achieves success rates of 99.0% on LIBERO and 79.4% on RoboTwin clean2clean, outperforming all the evaluated baselines. Under OOD shifts, IronMan achieves a success rate of 79.1% on LIBERO-Plus, exceeding the strongest baseline by 10.4 percentage points.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Robot Manipulation, Video Action Models

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_QQT7OpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a method that uses an information bottleneck to extract action relevant features from a pretrained video diffusion model. The method uses spatiotemporal aggregation followed by a Gaussian bottleneck to compress one-step video features into world tokens, which condition a separate action diffusion transformer. The model is jointly trained with video and action flow matching losses and KL regularization. The authors evaluate on LIBERO, LIBERO-Plus, RoboTwin, and real world manipulation tasks, reporting improved in-domain performance and improved robustness to visual perturbations.

Critical strengths
  1. The paper addresses a well-motivated problem. Features learned for video reconstruction may preserve visual details that are unnecessary for action prediction and potentially harmful under distribution shifts. Introducing an information constraint at the video action interface is a reasonable approach, and extracting features in a single video model forward pass avoids the overhead of full video generation.
  2. The ablations are well designed and informative. The authors separately evaluate the spatiotemporal aggregation module and the information bottleneck objective, showing that the two are most effective when combined. The experiments varying bottleneck capacity and regularization strength further support the proposed trade-off between information compression and action prediction.
  3. The experimental evaluation is fairly extensive, covering multiple simulation benchmarks and real world manipulation. The results on LIBERO-Plus are encouraging, particularly under camera and sensor-noise perturbations. I also appreciate the intervention experiments in Appendix A.4.1, which provide additional evidence that the resulting policy is less sensitive to visual changes.
Critical weaknesses
  1. The contribution of video supervision and the learned representations could be better isolated. While the ablations convincingly demonstrate the benefit of the bottleneck, the action flow matching loss already provides direct supervision to the world tokens and video backbone. It remains unclear how much the video prediction loss contributes during joint training. A useful ablation would be to train with and without this loss, starting from the same video-pretrained checkpoint and keeping the action loss and KL regularization unchanged. The paper could also examine the world tokens directly under the visual perturbations used in Appendix A.4.1. Comparing their sensitivity with the original video features would provide more direct evidence that the bottleneck suppresses irrelevant visual information, rather than only showing reduced variation in the predicted actions.
  2. The related work discussion overlooks much of the latent action and motion-centric representation literature. Works such as LAPA [1], Moto [2], UniVLA [3], villa-X [4], ViPRA [5], and AdaWorld [6] have explored learning compact dynamics-related representations from video for downstream control or world modeling. These works address closely related questions about learning control-relevant representations from video, and the paper would benefit from better positioning its contribution relative to them.
  3. The robustness improvements are somewhat uneven. Most of the improvement on LIBERO-Plus comes from camera and sensor noise perturbations, while gains under background and layout shifts are comparatively modest, despite their relevance to the paper's motivation. Performance also drops substantially on RoboTwin clean2random. These results do not undermine the demonstrated improvements, but suggest that the robustness benefits may be more limited than the general claims imply.
Questions for authors
  1. Have the authors examined how the learned world tokens change under the controlled visual perturbations in Appendix A.4.1? Comparing their sensitivity with the original video features could provide more direct evidence for the information filtering mechanism.
  2. How much does the video flow matching objective contribute during joint training? Would performance remain similar if the Video DiT were updated only through the action loss and KL objective, starting from the same pretrained checkpoint?
  3. How does performance change when the Video DiT is frozen during joint training? This would help establish whether adapting the video backbone is necessary or whether its pretrained representations are already sufficient.

References

[1] Ye et al. Latent Action Pretraining from Videos. ICLR 2025.

[2] Chen et al. Moto: Latent Motion Token as the Bridging Language for Learning Robot Manipulation from Videos. ICCV 2025.

[3] Bu et al. Learning to Act Anywhere with Task-centric Latent Actions (UniVLA). RSS 2025.

[4] Chen et al. villa-X: Enhancing Latent Action Modeling in Vision-Language-Action Models. ICLR 2026.

[5] Routray et al. ViPRA: Video Prediction for Robot Actions. ICLR 2026.

[6] Gao et al. AdaWorld: Learning Adaptable World Models with Latent Actions. ICML 2025.

Reviewer_LcUNOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

IronMan connects a pretrained Wan2.2-5B video backbone to an action diffusion model through learned query tokens and a Gaussian bottleneck. Video/action supervision and KL regularization aim to reduce sensitivity to irrelevant visual detail. Separate policies are trained on LIBERO and RoboTwin, with additional real-robot experiments.

Critical strengths
  1. The interface achieves 79.1% on LIBERO-Plus with 200 ms reported latency, alongside 99.0% on LIBERO. Real-robot results also improve substantially over FastWAM. This gives the method practical value, although RoboTwin clean2random remains difficult at 12.1% and pi0.5 scores better under real-world background shifts.
  2. Table 3 makes a useful comparison: aggregation alone reaches 74.3%, token-wise IB 73.2%, and their proposed combination 79.1%. The paired visual perturbations in Appendix A.4.1 further support reduced sensitivity to appearance changes.
Critical weaknesses
  1. The technical contribution is modest as currently established: Q-Former-style aggregation and a standard variational Gaussian bottleneck are combined at the video-action interface (§3.2–3.3). The combination improves performance, but Table 3 does not separate its temporal processing from generic learned pooling at a matched token budget and KL penalty. The attention maps and paired perturbations support reduced visual sensitivity, rather than showing what dynamics-specific information the proposed structure contributes. This leaves the distinctive value of the design less clear than its empirical gain.
  2. The cross-method comparison needs clearer training provenance. IronMan receives video-only finetuning before joint training (Appendix A.1.2), while equivalent details are not given for Table 2 baselines. The no-bottleneck baseline already scores 73.4%, compared with FastWAM's 49.2%, so much of the whole-system advantage precedes the proposed interface.
Questions for authors
  1. Which Table 2 results were reproduced, and with what backbone initialization, video-only finetuning, and joint-training budget? What accounts for the 73.4% no-bottleneck baseline versus 49.2% FastWAM?
  2. What benefit comes specifically from the temporal processing in Eq. (6), beyond generic Q-Former-style pooling with the same token budget and KL penalty? A matched comparison or existing representation analysis would help establish the distinctive contribution.
  3. What data were used to select Q, D, beta, the feature layer, and the reported checkpoint? Were the LIBERO-Plus evaluation outcomes used in those choices, or was selection done on separate validation data?
Reviewer_iCr7OpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

To address the irrelated information coupled in action generation using Video Action Models (VAMs), this paper proposes IronMan which employs a dynamics-aware bottleneck that distills noisy, entangled one-step video features into compact representations. Experiments shows the strong performance of IronMan on both in-distribution tasks and out-of-distribution tasks. Ablation study also shows that a well settled bottleneck size and information constraint strength β\beta are critical for action learning in VAMs.

Critical strengths
  1. The problem is well-motivated and clearly presented. The statement that 'single-step video features suffer from high noise and the entanglement of visual details and dynamics, while unconstrained interfaces allow the policy to learn shortcut features' provides a clear characterization of a common issue shared by a class of VAMs. Introducing IB into the video-action interface (rather than general visuomotor representations) is indeed an under-explored point."

  2. Strong performance on both in-distribution and out-of-distribution tasks.

Critical weaknesses
  1. Action learning with information bottleneck is not well explained and lacks in-depth analysis. Using equation (9) as a surrogate of equation (5) lacks clear therotical analysis. Variation of the bottleneck tokens is not analyzed during learning with/without the proposed IB and DB losses. How the bottleneck token learning align with the designed mechanism is unknown. It cannot be verified that the bottleneck is genuinely compressing (rather than β\beta being too small, which renders the regularizer ineffective, or posterior collapse). At inference, the operation zt = µt is not well studied and how it affects the action generation is unknown.

  2. Fairness of the comparison: This paper highlights that they achieve the best performance against the strongest baseline. However, nor some stronger baselines are compared such as 4D-WAM[a] on RoboTwin c2c nor they achieved the best performance on RoboTwin c2r and real-world background task. Insightful analysis on why IronMan performs well on some OOD tasks (Camera, light, noise in LIBERO-Plus) while weak on other OOD tasks (background) is absent. Comparison that without embodied pretraining is also questionable as the IronMan use Wan2.2-5B which is pretrained on a large-scale dataset.

[a] Yang, Lishan, Wenxuan Song, Xi Wang, Pingyue Sheng, Zheng Fang, Ziyang Zhou, Junjie He et al. "4D-WAM: Infusing spatiotemporal awareness into world action models through trajectory fields." arXiv preprint arXiv:2608.08023 (2026).

  1. Some details are missing such as how L_feat is selected in a one-step forward process.
Questions for authors
  1. What are the ablation results for sampling (z) at inference time, rather than using (\mu)?
  2. What accounts for the uniformly low C2R scores? Are bottleneck-based methods less sensitive to physical randomization than to visual perturbations?
  3. What is the rationale behind the choice of (LfeatL_{\text{feat}})?

Comments on the paper

1 comment
Area_Chair_EVU7OpenReview ↗

Discussion with the authors

Dear reviewers,

the authors have provided responses for your reviews. Can you please let them if you comments have been addressed?

Thank you, Your AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.