- Rating
- 4(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes GeoWRA, which aims to let the predicted future in a VLA directly guide action generation, rather than serving only as an auxiliary training signal. A lightweight predictor takes features of the current observation from a frozen V-JEPA encoder and, conditioned on the VLA's action representations, predicts future V-JEPA features at several horizons. These predictions are fed back into the action tokens through a bounded residual, so the action expert is conditioned on the predicted future at every denoising step. A distinctive design choice is to embed part of the prediction in hyperbolic space, which the authors use to model relations among nested image regions and across time. The method is built on π0.5 and a Qwen-based VLA, and is evaluated on LIBERO, LIBERO-Plus, SimplerEnv and six real-world tasks.
Critical strengths
- Predicted futures directly guide action generation. Prior latent world action models use future prediction only as an auxiliary loss. Here, the predictor is conditioned on the current noisy action, and its prediction is fed back into the action tokens at every denoising step. The feedback is bounded and zero-initialized, so the module can be added to an existing VLA without disturbing its original behavior. It is also lightweight and has been applied to several different VLA backbones.
- Structural supervision on the predicted future. Beyond aligning individual future features, the paper also supervises the relations among predicted regions and across time, using a hyperbolic embedding. This brings hyperbolic representation learning into world action models, and gives the predictor a structural constraint that plain feature alignment does not provide.
- Evidence that the policy relies on the predicted content. The real-world tasks are chosen to require anticipating the consequences of actions, such as intercepting a moving object or keeping an unstable stack intact, and the method improves on all of them. The intervention experiments go a step further: replacing the predicted future with current features, or with futures from other trajectories, clearly lowers success. This suggests that the policy actually uses the predicted content, rather than only benefiting from extra parameters.
Critical weaknesses
- The motivation for hyperbolic geometry remains unclear. The spatial hierarchy consists of shallow, fixed grids of image regions. Although the Euclidean–Euclidean comparison supports a benefit from hyperbolic supervision, it is reported only on SimplerEnv. The paper also does not examine whether the embeddings capture the intended hierarchy, leaving the proposed geometric explanation insufficiently validated
- The benefit of future prediction is not fully separated from that of additional V-JEPA features. The StarVLA baseline lacks this feature pathway. Although inference-time substitutions show that the trained policy uses its predicted inputs, they may introduce a distribution mismatch. A baseline trained to use current V-JEPA features through the same adapter, without future prediction, would better isolate the contribution of predicting the future.
Questions for authors
- Could you elaborate on why hyperbolic geometry is particularly suitable for the shallow, fixed hierarchy of image regions used in this method? The Euclidean–Euclidean comparison provides evidence of an empirical benefit, but the connection between this benefit and the proposed hierarchical inductive bias remains unclear. Is there evidence that the hyperbolic representation better captures the intended nesting relationships, rather than primarily changing how feature prediction errors are weighted during training?
- Could you clarify how the contribution of future prediction can be separated from the benefit of introducing additional V-JEPA features? The feedback-content interventions show that the trained policy is sensitive to its predicted inputs, but inference-time substitutions may introduce a feature-distribution mismatch. Have you considered training a baseline that uses current V-JEPA features through the same residual adapter, with comparable capacity and training budget? Such a comparison would help establish what future prediction adds beyond the additional visual representation.
- Could you test whether the model predicts different futures for different actions from the same starting state? For example, in simulation, you could execute a few different action sequences from the same scene and compare the predicted features with the features of the actual outcomes. It would be useful to check whether providing the correct action gives a more accurate prediction than providing a different action. This would build on the existing ablations and help show whether the predictor has learned how actions change the scene, beyond what it can infer from the observation and task alone.
3 comments
Thank you for the detailed response and the additional experiments. The trained current-feature control directly addresses my concern about separating the contribution of future prediction from that of additional V-JEPA features. The same-state, different-action experiment also provides convincing evidence that the predictor uses action information to distinguish future outcomes in the evaluated setting.
The expanded Euclidean controls and relation-distance analysis strengthen the empirical justification for the hyperbolic formulation. While the specific explanation in terms of a hierarchical inductive bias remains suggestive rather than conclusive, this is now a limited concern rather than a major weakness.
Overall, the additional experiments substantially address my original concerns.
Thank you for recognizing our work and raising the score to 4
Thank you for raising your score to 4 and for the careful follow-up. Your suggestions on the trained current-feature control, the same-state action test, and the expanded geometry controls directly shaped the new experiments and substantially improved the paper. We sincerely appreciate your time and constructive feedback throughout the review process.
Motivation for hyperbolic geometry, requested current-feature control, and same-state action test
Thank you for the precise questions and for recognizing that "predicted futures directly guide action generation" and that "the policy actually uses the predicted content." We have revised the paper accordingly, with all changes highlighted in blue.
For the motivation, we do not rely on hyperbolic capacity, which three nested grids (21 regions) do not require. The fixed nesting defines parent–child and cross-horizon pairs that must be matched jointly, and a tree-like distance suits the nested grids while the same metric also measures the cross-horizon edges (Sec. 3.3 of our revised paper). The learned representations capture this structure. On SimplerEnv, the Spearman correlation between predicted and teacher relation distances (rank-based, as training metrics differ) reaches 0.82/0.79/0.72 for GeoWRA on spatial/temporal/nonadjacent pairs versus 0.73/0.68/0.60 for E–E, trained with relation losses on the same pairs (Table 5 in our revised main paper). The gain also holds on nonadjacent pairs, which the relation loss does not directly supervise, suggesting that it extends beyond these edges. GeoWRA without relation matching (0.76) still exceeds E–E with it (0.73), so the gain is not due to relation matching alone. Figure 4 in our revised main paper visualizes the learned nesting.
For the reweighting question, we add a conformal-weighted E–E that scales its flat losses by the conformal factor λc, a position-dependent reweighting without curvature. We also extend E–E (the zero-curvature limit with identical banks, pairs, and budget) beyond SimplerEnv to all four settings. As shown in Table R4, both Euclidean variants remain below GeoWRA in all four settings, so this reweighting recovers part, but not all, of the gain and does not fully explain it.
Table R4 (Table 14 in our revised appendix). Geometry controls: success rates (%).
We train the suggested current-feature control, which uses current V-JEPA features through the same residual adapter without future prediction, with comparable capacity, data, and training budget. As shown in Table R5, this additional visual representation improves StarVLA by 7.2 points. With the same adapter, future prediction adds a further 6.9 points. Because the control is trained and evaluated with current features, it also avoids the distribution mismatch you raise. The 6.9-point gap is thus free of it, and the control reaches 63.6% versus 61.8% for the inference-time substitution of Table 11 in our revised appendix (Table 9 in the original submission).
Table R5 (from Table 4 in our revised main paper). Contribution of each component on SimplerEnv.
Thank you for this constructive suggestion. From each of 120 initial scenes (30 per task) with a fixed instruction, we execute four distinct seven-step action sequences, so only the action differs. We compute the NRMSE between predicted and actual-outcome features at τ=1, conditioning on the generating action (correct) or, averaged, on the other three (incorrect). Identification accuracy is the fraction of the 480 outcomes whose generating action gives the lowest NRMSE, and a predictor that ignores the action stays at chance (25%). As shown in Table R6, on all four tasks, the correct action gives a more accurate prediction than a different action, with 22.3% lower NRMSE and 89.0% accuracy. In this setting, the predictor has thus learned how actions change the scene, beyond what it can infer from the observation and task alone.
Table R6 (Table 16 in our revised appendix). Same-state action-conditioned prediction on SimplerEnv–Google (last-row reduction from the mean NRMSE).
We hope these responses address your concerns and are glad to answer further questions.