- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes PHI, a framework for humanoid bimanual manipulation that combines task-oriented future representation prediction with temporal force feedback. Its three-stage training procedure first learns future representations from privileged future observations using action supervision, then trains the WAM to predict these representations from the current context, and finally jointly optimizes future prediction and action generation. A temporal force encoder processes recent force measurements to provide additional contact information to a flow-matching action expert. Experiments on five real-world Unitree G1 tasks report an average success rate of 82.0%, compared with 58.7% for the strongest baseline, along with evaluations on unseen objects and visual distractors.
Critical strengths
-
The method is well motivated. Learning future representations through action supervision offers a reasonable way to focus on information useful for control.
-
Combining future visual predictions with force history is sensible for contact-rich manipulation.
-
The paper evaluates the method on real-world humanoid manipulation tasks and includes tests with unseen objects and visual distractors. The demo is convincing.
Critical weaknesses
-
The architecture allows the action expert to rely mainly on the current context and ignore the future branch. The training objectives do not rule out this shortcut. Specifically, (Z^+) could become redundant, nearly constant, or predictable but unused by the action expert. It is unclear what information (Z^+) contains. For the “w/o Future Learning Module” variant in Table 3, does removing this module reduce the total number of parameters?
-
The fairness and coverage of the baseline comparisons are unclear. Are the baselines fine-tuned from pretrained checkpoints or trained from scratch on the collected data? Do the baselines receive force inputs? Could the authors compare against additional relevant methods, such as [1] and [2]?
[1] Zheng, Ruijie, et al. "Flare: Robot learning with implicit world modeling." arXiv preprint arXiv:2505.15659 (2025). [2] https://huggingface.co/nvidia/GR00T-N1.5-3B
Questions for authors
- What happens if the force encoding is also fed directly into the WAM?
- The authors can describe this cascaded structure as hierarchical. Many VLAs, including the baseline π₀, already use a vision-language backbone followed by an action expert. PHI mainly adds future latent predictions supervised by future observations and passes them to the action expert. Why do the authors need the emphasize it?
- Despite feature alignment, the paper doesn't demonstrate or evaluate the ability of the "WAM" to predict the future. How is the term "world action model" justified?
8 comments
Response to Reviewer jts4 (2/2)
Q3: What happens if force encoding is also fed directly into the WAM?
Feeding the encoded force history into the WAM could make its future predictions more sensitive to contact evolution and potentially provide a modest improvement on contact-rich tasks. For example, recent force changes could help distinguish stable contact from emerging slip when predicting subsequent interaction. We would expect the additional benefit to be incremental, since the WAM already receives current force through the robot state and the action expert directly uses the encoded force history. This is an architectural expectation; we have not evaluated this additional conditioning route.
Our choice balances this potential benefit against the additional training and inference computation needed to integrate force-history conditioning into the large WAM backbone. PHI gives the two pathways complementary roles: the WAM predicts future visual interaction, while the action expert combines that prediction with recent force evolution to generate contact-sensitive actions. Routing force history directly to the action expert preserves its contribution to control without requiring that information to pass through the large backbone. We therefore favor this design for real-robot deployment, where task performance, computational efficiency, and timely action generation must be considered together.
Q4: Why emphasize a hierarchical structure when many VLAs already use a vision-language backbone followed by an action expert?
As discussed in the Introduction (Section 1 and Figure 1), we distinguish three architectural paradigms for connecting future modeling with action generation in WAMs: cascaded approaches, joint autoregressive approaches, and joint diffusion approaches. Our emphasis on a hierarchical structure explains PHI's architectural position within this WAM landscape and how it connects future prediction to control.
In cascaded WAMs, future visual observations are first reconstructed, and representations for action generation are then derived from them. Joint autoregressive and joint diffusion WAMs instead model future and action representations together. PHI follows a hierarchical cascaded design, with an explicit predicted future representation serving as an intermediate input to the downstream action expert. Its key distinction within this design is what the upper pathway predicts: a compact future visual latent shaped by downstream action supervision, rather than a reconstructed future image or video. The Exploit–Predict–Align procedure first learns this action-relevant target from privileged future observations, then learns to predict it from the current context, and finally trains action generation with the predicted representation.
This organization combines an explicit prediction-to-action cascade with selective latent prediction. The upper pathway anticipates task-relevant future visual interaction, while the lower action expert combines that prediction with current state and temporal force feedback to produce actions. Thus, “hierarchical” communicates an architectural choice among WAM designs: PHI separates future representation prediction from action generation while connecting them through an action-shaped latent interface. This distinction is why we emphasize the term in the Introduction.
Q5: How is the term “world action model” justified if the paper does not show future prediction?
PHI predicts future visual information in latent space rather than generating future video pixels. During training, privileged future observations provide the target representation. At deployment, the WAM predicts that representation from the current interaction context, and the action expert uses it together with force feedback. Appendix A.5 evaluates this prediction over 20 Open Book trajectories, reporting a mean latent prediction MSE of 0.069287 and cross-sample statistics for predicted and target latents. Thus, “world action model” refers to PHI's explicit prediction of a future interaction representation for action generation; it does not imply a pixel-level simulator or full future-video reconstruction.
We appreciate the reviewer's questions, which help clarify the role of PHI's future pathway, force input, and hierarchical design.
-
I noticed that the other two reviewers also highlighted the lack of comparisons with alternative designs for the future prediction module. Comparisons with additional baselines are helpful. However, a more convincing evaluation would be an ablation study comparing the proposed method with other variants, as suggested by reviewer VwGx, while keeping all other design choices fixed. It's not clear to me why the authors choose not to conduct the ablation experiments specified by reviewer VwGx.
-
The experiment result in the comment shows that the features have variance and randomly perturbing it causes the performance. It's still not clear what information (Z^+) contains. Can it be used to generate (partial) future states or images?
-
"All policies receive current force measurements through the robot-state input. PHI additionally encodes recent force history." Does it mean the baselines do not take in recent force history as input?
-
Not critical, but the neuroscience framing in Figure 1 is redundant and decorative.
Response to Reviewer jts4 (1/2)
We thank the reviewer for recognizing the motivation, the combination of future visual prediction and force feedback, and the real-robot evaluation. We clarify the role of the future representation, the baseline comparisons, and PHI's hierarchical design below.
Q1: The action expert could ignore the fture branch, and Z+ could be redundant or nearly constant. What information does it contain? Does removing the Future Learning Module reduce the parameter count?
We use Z+ as a compact representation of future visual interaction, shaped to retain information useful for action generation; it is not intended to reconstruct every detail of a future frame. Appendix A.5 reports statistics over 20 Open Book trajectories.
The predicted latent retains 45.6% of the target variance and multiple effective dimensions, which is inconsistent with a constant representation in these samples. With the action predictor fixed, shuffling latent tokens increases joint MSE by 23.18%; replacing each latent with its global mean or zeros increases it by about 802%.
The component ablation provides complementary behavioral evidence: removing the Future Learning Module reduces Place Book Straight success from 76.7% to 50.0%. The Future Learning Module contains approximately 0.6B parameters. Removing it reduces the model from 4.7B to about 4.1B. In the real-robot ablation in Table 3, including this module improves Place Book Straight success from 50.0% (15/30) to 76.7% (23/30), with the other PHI components retained.
Q2: Are the baselines fine-tuned from pretrained checkpoints or trained from scratch? Do they receive force inputs? Could the authors compare with FLARE and GR00T?
The evaluated backbones use publicly released pretrained parameters and are fine-tuned on our collected demonstrations; PHI's newly introduced modules use random initialization. All policies receive current force measurements through the robot-state input. PHI additionally encodes recent force history. Appendix A.7 reports model sizes, training resources and steps, total batch sizes, RTC availability, and inference settings: π0 and π0.5 use 500k steps/GPU with total batch 32, VLA-JEPA uses 400k steps/GPU with total batch 96, Fast-WAM uses 400k steps/GPU with total batch 32, and PHI uses 500k steps/GPU with Stage I/II/III total batches 32/32/48. PHI's single-GPU setting uses batch size 6. The methods use the same robot platform, task definitions, action space, low-level controller, and evaluation initial-condition sequence, with 30 real-robot trials per task.
We added VLA-JEPA and Fast-WAM to the same-protocol comparison because they are more directly aligned with PHI's future-representation and WAM design. PHI reaches 82.0% average success, compared with 45.3% for Fast-WAM and 1.3% for the evaluated VLA-JEPA configuration. The full per-task results are in Table 2.
We did not include FLARE as a quantitative baseline because no public implementation or checkpoint was available to us for a same-protocol reproduction; NVIDIA's project page lists its GitHub release as forthcoming. GR00T-N1.5 is described by NVIDIA as a general-purpose humanoid foundation model, rather than a future-predictive WAM (project page), so it is less directly matched to PHI's central question. We therefore report same-protocol results for VLA-JEPA and Fast-WAM, the closer alternatives for this comparison.