acceptodds
Under review as a conference paper at ICLR 2027

GWM-VLA: Geometry-Aware Latent World Modeling for Vision-Language-Action Learning

18 views

Abstract

Vision-Language-Action (VLA) models achieve strong robotic manipulation performance but often degrade under visual and environmental shifts. Latent world modeling offers a promising approach to improving robustness, yet existing methods commonly encode camera views independently and predict holistic scene dynamics without explicitly modeling their geometric relationships. We propose GWM-VLA, a geometry-aware latent world modeling framework for VLA learning. GWM-VLA combines geometry-aware multi-view state encoding, global context-conditioned target-view prediction, and shared latent-action representations grounded in robot-action supervision. Specifically, VGGT- jointly aggregates multi-view observations at each timestep to construct geometry-aware multi-view states. The latent world model predicts the next-step patch tokens of a selected target view using patch and register tokens extracted after multi-view aggregation, thereby retaining multi-view geometric information without predicting the complete multi-view state. We use the wrist view as the target in our experiments, placing greater emphasis on end-effector motion and local gripper-object interactions. Finally, the shared latent-action representations condition both the latent world model and the flow-matching action head, allowing latent-prediction supervision and ground-truth robot-action supervision to jointly shape the same latent-action representations. Experiments across both simulation and real-world environments demonstrate the effectiveness and robustness of GWM-VLA.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Vision-Language-Action Models, Robot Learning, Imitation Learning, World Models, Representation Learning, Embodied AI

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_HT78OpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents GWM-VLA, a geometry-aware latent world modeling framework for Vision-Language-Action (VLA) robot learning, targeting the well-known robustness degradation of VLA policies under visual and environmental distribution shifts. Departing from the common practice of using geometric foundation models purely as perception feature extractors, this work adopts jointly aggregated multi-view geometric representations from VGGT-Ω as the predictive state space of the latent world model. The framework features three core designs: per-timestep geometry-aware multi-view state encoding, global context-conditioned target-view (wrist-view) latent prediction, and shared latent-action representations that condition both the world model and the flow-matching action head. The method is validated on LIBERO and LIBERO-Plus simulation benchmarks as well as real-world SO-101 robot experiments, with extensive ablation studies verifying the contribution of each component. The authors explicitly acknowledge key limitations, including the requirement of synchronized multi-view inputs.

Critical strengths

The work proposes a fundamentally novel perspective for VLA world modeling by repurposing geometric foundation model outputs as the dynamical state space of latent world models, rather than treating them as passive perceptual frontends. This breaks the prevailing paradigm of using geometric representations only as input features, and opens a new avenue for integrating pre-trained geometric priors into embodied world dynamics learning. This conceptual innovation is the core contribution of the paper and provides strong justification for acceptance.

The evaluation is rigorous, comprehensive, and well-scoped. It spans both in-distribution simulation (LIBERO) and out-of-distribution robustness benchmarks (LIBERO-Plus across seven perturbation dimensions), complemented by real-robot deployment on the SO-101 platform. A full suite of targeted ablation studies systematically validates every core design choice. Critically, the paper clearly delineates the method’s operating boundaries and limitations instead of overstating its generalizability, which strengthens the credibility of its conclusions.

Critical weaknesses

The prediction target view is fixed as the wrist camera throughout training, with no mechanism for adaptive view selection tailored to different task types. The paper provides no empirical evidence that the wrist view remains the optimal choice for other operation modes (e.g., tool use, long-horizon assembly tasks), leaving the generalizability of this design rule largely unsubstantiated. This limits the broader applicability of the proposed framework to diverse robotic manipulation scenarios.

As previously noted in the VLA-JEPA work, the shared latent-action design carries an inherent risk of shortcut learning (information leakage): latent action tokens may directly encode future target-view patch token information rather than learning genuine action-conditioned state transition dynamics. The authors posit that the geometric latent space can mitigate this risk, but provide no diagnostic experiments to verify whether shortcut learning occurs, nor to quantify its magnitude. This methodological concern calls into question whether the observed performance gains truly stem from learned world dynamics.

Questions for authors

How do you verify that the shared latent-action representations learn genuine action-conditioned transition dynamics, rather than directly encoding the patch token information of future target views? Have you conducted diagnostic experiments (e.g., shuffling the order of action tokens, corrupting action inputs, or probing information content) to quantify the degree of potential information leakage?

In the paper, comparison is only made against the VLA-JEPA variant without human video pretraining. What is the rationale for not including the full VLA-JEPA baseline with human video pretraining? Would the robustness improvements of GWM-VLA remain statistically significant when compared against the full VLA-JEPA baseline?

The paper acknowledges the limitation that the method requires synchronized multi-view observations, while most existing large-scale robot datasets are single-view. Do the authors have a concrete research plan to extend this framework to single-view datasets to enable large-scale pretraining, and are there any preliminary results to suggest this extension is feasible?

3 comments

We sincerely thank the reviewer for the constructive comments and the appreciation of our work. Our response to your concerns/questions:

W1: Wrist Prediction as a Task-Aligned Target

We select the wrist view because it moves with the robot arm and captures local gripper–object interactions. Predicting this changing viewpoint encourages the world model to use cross-view geometric information to understand how end-effector motion changes the observed scene. Cross-view aggregation provides the global scene context needed to support this prediction, while the wrist target emphasizes manipulation-relevant state changes. The value of focusing on wrist-local interactions is also supported by World-to-Wrist [1], which uses future wrist prediction to improve fine-grained manipulation.

Our target-view ablation on LIBERO-Spatial supports this choice: wrist prediction achieves 92.8%, compared with 87.8% for third-person prediction and 92.4% for mixed target sampling (80% wrist and 20% third-person). These results support wrist prediction as an effective target for learning manipulation-relevant geometric dynamics in the evaluated setting.

W2: Information-Flow Isolation of Latent Actions

GWM-VLA prevents direct future-observation leakage into latent actions by design. Latent actions are computed exclusively from current observations and the task instruction; future observations do not enter this computation.

The leakage mechanism discussed in VLA-JEPA’s Introduction concerns methods that extract latent actions using both current and future observations. In our framework, VGGT-Ω encodes each timestep independently, and the world predictor uses temporally causal attention, preserving the separation between prediction inputs and future targets.

We verified this design by replacing future images while keeping the model parameters and current inputs fixed. Across 1,200 replacements, the latent actions and first-step predictions remained exactly unchanged, even though the future target features changed. This confirms that future images provide prediction-loss targets without entering or influencing the latent-action computation in the tested implementation.

Q1: Latent-Action Perturbation Diagnostic

Following the reviewer’s suggestion, we replaced latent actions with those from other trajectories of the same task, while keeping current-state patch tokens, register tokens, and prediction targets fixed. We selected 400 timesteps from 160 newly collected trajectories for next-state prediction. For each sample, we used latent actions from three other trajectories separately and averaged the resulting prediction errors.

Table 1. Effect of latent-action replacement on next-state prediction.

Latent-action conditionFeature L1 ↓Error increase
Original1.246567—
Replaced1.2869293.24%

As shown in Table 1, replacement increased prediction error by 3.24%, corresponding to a mean L1 increase of 0.0404 (95% CI: [0.0349, 0.0460]). This supports the predictor’s use of sample-specific latent actions for state-transition prediction, complementing the future-input isolation verified in W2.

Q2: Fair Comparison with VLA-JEPA

Robot-only VLA-JEPA and GWM-VLA share the same policy-training settings: DROID pretraining, LIBERO fine-tuning, the Qwen3-VL-2B backbone, 50K pretraining steps, 30K fine-tuning steps, and an effective batch size of 256. Their comparison is therefore fair.

Full VLA-JEPA additionally uses Something-Something V2 (SSv2), a single-view human-video dataset without synchronized multi-view observations, for policy pretraining, which is separate from our core contribution: using multi-view geometric representations for latent world modeling to shape shared latent actions.

For completeness, we also report full VLA-JEPA below. It achieves higher average success rates under the additional human-video pretraining setting. Incorporating human videos is a meaningful direction for future work.

Table 2. Average success rates (%) under different policy-pretraining settings.

MethodPolicy-pretraining dataLIBEROLIBERO-Plus
VLA-JEPA, robot-onlyDROID96.162.9
GWM-VLADROID97.175.4
Full VLA-JEPADROID + human videos97.279.5

Q3: Applicability to Multi-View and Single-View Data

An expanding collection of large-scale robot datasets supports multi-view learning, including DROID[2], RH20T[3], and AgiBot World[4]. These resources provide a practical foundation for scaling our geometry-aware world-modeling framework.

For single-view data, a straightforward adaptation is to use the frozen VGGT-Ω as a single-view geometric encoder: encode the available image at each timestep and predict its next-step latent features, retaining the shared latent-action representation and joint action/prediction objectives.

Our wrist-only encoding ablation provides preliminary evidence for this approach. Under the same DROID pretraining and LIBERO fine-tuning procedure, it achieves 96.2% on LIBERO and 68.2% on LIBERO-Plus, as shown in Table 3. This experiment uses a single view for world-state encoding, supporting the feasibility of single-view geometric prediction supervision.

We plan to incorporate single-view robot datasets into large-scale pretraining by supplying the available view to both the policy and the geometric encoder, and jointly training on single-view and multi-view trajectories. This would broaden the usable training data while exploiting cross-view geometric information whenever multiple views are available.

Table 3. Success Rates (%) with Single-View and Multi-View World-State Encoding.

MethodWorld-state encoding viewsLIBEROLIBERO-Plus
GWM-VLAMain + wrist97.175.4
Wrist-only encodingWrist96.268.2

References

[1] Pan Y, Peng H, Zhang Z, et al. World-to-Wrist: Task-conditioned future wrist modeling for fine-grained robot manipulation[J]. arXiv preprint arXiv:2608.05369, 2026.

[2] Khazatsky A, Pertsch K, Nair S, et al. Droid: A large-scale in-the-wild robot manipulation dataset[J]. arXiv preprint arXiv:2403.12945, 2024.

[3] Fang H S, Fang H, Tang Z, et al. Rh20t: A comprehensive robotic dataset for learning diverse skills in one-shot[C]//2024 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2024: 653-660.

[4] World A B, Bu Q, Cai J, et al. Agibot world colosseo: A large-scale manipulation platform for scalable and intelligent embodied systems[J]. arXiv preprint arXiv:2503.06669, 2025.

We greatly appreciate it if you could take a look at our first-round responses. Please let us know if you have any further comments or suggestions. We would be happy to make additional improvements and engage in further discussion.

Reviewer_LqDvOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a geometry-aware latent world model for VLA learning. The method is building upon VLA-JEPA. It replaces the V-JEPA2 encoder with VGGT-Omega encoder for cross-view geometry information in latent world model. The latent world model predicts only wrist-view patch token instead of full multi-view latent states. And the latent world model and action head use shared latent-action representation. The proposed method is evaluated on LIBERO, LIBERO-Plus, and real-world robotic manipulation tasks, which shows competitive performance and improved robustness compared with VLA-JEPA baseline.

Critical strengths

The experimental results includes several benchmarks, including LIBERO, LIBERO-Plus, and real world tasks. The proposed method is tested on OOD setting and demonstrates improved robustness under several settings. The ablation study includes target-view selection, which supports the design of using wrist-view only for latent world model's future prediction.

Critical weaknesses

The proposed method replaces V-JEPA-2 encoder with VGGT-Omega, while VLA-JEPA has two different versions, one including human-video data pretraining and the other one using only robot data. However, since VGGT-Omega has also been pretrained on large scale multiview data, is this a fair comparison? Also, can the improvement be mainly attributed to the geometric information contained in VGGT-Omega?

This also extend to my second concern. The proposed method actually only use wrist view as latent world model's prediction target, which is single-view prediction. And the main reason for leveraging VGGT-Omega is its multiview geometry awareness. So this makes me confused: does the model really leverage the multiview geometry information, or is this just a replacement of the encoder, and could DINO, a monocular depth model, or other types of vision encoders can also be helpful?

Questions for authors

It would be helpful if the authors could further analyze and explain why the proposed method does not outperform some baselines on LIBERO-Plus under certain settings, especially under camera-viewpoint changes, given that this is already a geometry-aware learning framework.

3 comments

We thank the reviewer for the constructive comments. We address the concerns about comparison fairness, multi-view geometric information, and viewpoint robustness below.

W1. Comparison Fairness and the Contribution of Geometric World Modeling

Regarding the fairness of the comparison between GWM-VLA and VLA-JEPA, both methods use pretrained and frozen world-state encoders: VLA-JEPA uses V-JEPA-2, while GWM-VLA uses VGGT-Ω. “Robot-only” refers to the data used during policy pretraining; the robot-only VLA-JEPA baseline also retains the pretrained V-JEPA-2 encoder. Our comparison aligns DROID pretraining, LIBERO fine-tuning, the Qwen3-VL-2B backbone, training steps, and effective batch size, providing a fair setting for comparing how different world-modeling approaches shape the policy. To examine whether the improvement comes from VGGT-Ω’s geometric representations or from shaping latent actions through geometric world modeling, we introduce a Spatial-Forcing-style baseline [1] using the same frozen VGGT-Ω teacher. Specifically, Qwen’s layer-21 visual tokens are mapped through a learnable projection head and aligned with VGGT-Ω’s final-layer, multi-view-aggregated features using a cosine loss, replacing the latent world model and future-state prediction objective. Under the same DROID pretraining and LIBERO fine-tuning pipeline, this baseline achieves 96.1% on LIBERO and 65.5% on LIBERO-Plus, compared with 97.1% and 75.4% for GWM-VLA (Tables 1 and 2). These results suggest that shaping latent actions through geometric world modeling enables the policy to learn multi-view geometric dynamics and use the pretrained geometric information more effectively, improving VLA robustness under distribution shifts.

W2. Multi-View Geometric Information in Wrist-View Prediction

Regarding whether the prediction is single-view and whether multi-view geometric information is used, our prediction targets are the wrist-view tokens extracted after cross-view aggregation by VGGT-Ω, as described in Sections 3.2–3.3. These tokens already incorporate geometric information from the main view. To assess the contribution of this information, we add a wrist-only encoding ablation under the full DROID-pretraining and LIBERO-fine-tuning pipeline. This variant retains the same VGGT-Ω encoder but uses it as a single-view encoder, constructing the state from the wrist image alone. Its LIBERO-Plus success rate decreases from 75.4% to 68.2%, including a decrease from 57.9% to 42.4% under camera perturbations (Table 2). These results support the contribution of multi-view geometric state construction to policy robustness. We further introduce a DINOv2 ViT-L/14 baseline. The frozen encoder processes the main and wrist views independently, and the resulting tokens are concatenated as input to the world model. We retain the next-step wrist-view prediction target and the same DROID pretraining and LIBERO fine-tuning pipeline. This baseline achieves 96.0% on LIBERO and 63.6% on LIBERO-Plus, compared with 97.1% and 75.4% for GWM-VLA (Tables 1 and 2). By retaining both input views, this baseline evaluates independent visual encoding with simple concatenation. Together with the wrist-only ablation, these results show that latent world modeling over representations that integrate multi-view geometry provides greater benefits than the tested single-view encoding and DINO-based alternatives.

Q1. Viewpoint Robustness and Transfer Across Policy Architectures

We thank the reviewer for highlighting viewpoint robustness. Under camera-viewpoint perturbations in LIBERO-Plus, GWM-VLA outperforms robot-only VLA-JEPA under the aligned comparison setting, while π₀-FAST achieves a higher score. Viewpoint robustness is also influenced by the pretrained backbone and policy architecture. Our geometry-aware learning framework can be adapted to other VLAs. To examine its contribution across architectures, we integrate GWM into π₀-FAST, which provides a stronger reference under camera perturbations. We retain π₀-FAST’s pretrained vision-language backbone and native FAST action output, inserting 10 learnable latent-action tokens before the action sequence. These tokens participate in action generation and condition the world model to predict next-step wrist features extracted after multi-view aggregation by frozen VGGT-Ω. Causal attention prevents the latent tokens from accessing ground-truth action tokens. Training jointly optimizes FAST action cross-entropy and a geometric feature-prediction loss weighted by 0.1. The geometric teacher and world-model predictor are removed at inference. Starting from the official pretrained π₀-FAST weights, we train on LIBERO with a global batch size of 32. The 10K-step EMA checkpoint achieves 74.5% under camera perturbations, exceeding the reported π₀-FAST baseline of 65.1% by 9.4 percentage points. Overall LIBERO-Plus success reaches 69.6%, compared with 61.6%, an improvement of 8.0 percentage points, with higher scores in six of seven perturbation categories (Table 2). This extends GWM from Qwen-based continuous action generation to PaliGemma-based autoregressive FAST action generation, supporting the transferability of geometry-aware predictive supervision and its value for viewpoint robustness. We will include this experiment in the revised manuscript.

Table 1. LIBERO Results for Ablations and Cross-Architecture Evaluation

MethodSpatialObjectGoalLIBERO-10Overall
GWM-VLA96.899.098.094.497.1
Wrist-only encoding96.298.296.893.696.2
DINOv295.898.496.693.296.0
VGGT-Ω Spatial Forcing96.697.896.893.296.1
π₀-FAST96.496.888.660.285.5
π₀-FAST + GWM96.698.692.676.691.1

Table 2. LIBERO-Plus Robustness Results for Ablations and Cross-Architecture Evaluation

MethodCameraRobotLanguageLightBackgroundNoiseLayoutOverall
GWM-VLA57.954.789.895.490.872.577.175.4
Wrist-only encoding42.447.484.792.987.862.773.368.2
DINOv234.745.488.691.190.245.566.563.6
VGGT-Ω Spatial Forcing37.342.980.688.187.759.676.565.5
π₀-FAST65.121.661.073.273.274.468.861.6
π₀-FAST + GWM74.526.174.590.388.768.875.769.6
All ablation experiments conducted using 8 NVIDIA H200 GPUs.

References

[1] Li F, Song W, Zhao H, et al. Spatial forcing: Implicit spatial representation alignment for vision-language-action model[C]//International Conference on Learning Representations. 2026, 2026: 132324-132345.

We greatly appreciate it if you could take a look at our first-round responses. Please let us know if you have any further comments or suggestions. We would be happy to make additional improvements and engage in further discussion.

Reviewer_ZxKXOpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes GWM-VLA, a geometry-aware latent world modeling framework for vision-language-action learning. The proposed method integrates a pretrained VGGT-omega encoder for multi-view geometric feature extraction, a latent world model for predicting future target-view patch tokens, and a flow-matching action head conditioned on shared latent-action representations. The model is pretrained on DROID and evaluated on LIBERO, LIBERO-Plus, and real-world SO-101 manipulation tasks. GWM-VLA achieves competitive performance on LIBERO and improves the overall LIBERO-Plus success rate from 62.9% to 75.4% compared with the robot-only VLA-JEPA baseline.

Critical strengths
  • Well-motivated problem and promising insight. Latent world modeling is a promising direction for improving VLA robustness. The paper identifies a meaningful limitation of VLA-JEPA in modeling cross-view geometric relationships and proposes leveraging VGGT-omega to address it, which is a reasonable and interesting insight.
  • Reasonable architecture and encouraging results. The overall architecture is well designed and the integration of geometry-aware representations, latent world modeling, and action prediction makes sense. The experimental results are encouraging, and the paper is generally clear and well presented.
Critical weaknesses
  • Insufficient benchmark evaluation and comparisons. The reported performance is not particularly compelling compared with stronger existing methods, and several competitive baselines on LIBERO-Plus are missing. Moreover, the comparison with VLA-JEPA is not entirely fair given the differences in pretraining configurations, and evaluation on only LIBERO and LIBERO-Plus is insufficient to demonstrate broader generalization.
  • Insufficient evidence for the benefits of cross-view geometric modeling. A central motivation of the paper is to explicitly capture cross-view geometric relationships, yet the experiments provide little analysis of whether and how such information improves world modeling and policy learning. In particular, the necessity of VGGT-omega over alternative visual encoders or simpler multi-view fusion methods is not convincingly demonstrated.
  • Limited ablations and unclear attribution of improvements. The ablations of multi-view aggregation and register-token conditioning show only marginal gains (1.0–1.2 percentage points), without variance estimates. Moreover, the contribution of world-model supervision is only evaluated under a restricted setting, making it difficult to disentangle the benefits of the proposed world modeling objective from those of stronger pretrained geometric features.
Questions for authors

As shown in the weaknesses.

3 comments

We thank the reviewer for the constructive feedback and recognition of our motivation and architectural design. We address the three concerns below.

W1: Broader Baseline Comparisons and Fair Evaluation

Additional baselines. We expand the LIBERO-Plus comparison to include Fast-WAM [1], 4D-WAM [2], and Cosmos Policy [3]. As shown in Table 1, GWM-VLA outperforms Fast-WAM in all seven perturbation categories and achieves a higher category-average success rate than 4D-WAM. Cosmos Policy achieves a higher overall score using a pretrained video-generation backbone and a larger LIBERO training budget: 40K steps with a global batch size of 1,920, compared with 30K steps and 256 for GWM-VLA. Its standard inference procedure jointly generates actions, future states, and values, whereas GWM-VLA removes the geometric teacher and world-model predictor at inference time.

Table 1. LIBERO-Plus comparison with additional baselines.

MethodCameraRobotLanguageLightBackgroundNoiseLayoutOverall
Fast-WAM [1]16.444.568.978.253.737.760.751.5
4D-WAM [2]45.264.390.694.357.769.179.271.0
Cosmos Policy [3]75.863.381.796.588.992.782.282.2
GWM-VLA57.954.789.895.490.872.577.175.4

Fairness of the VLA-JEPA comparison. Our comparison with robot-only VLA-JEPA is fair under matched policy-training conditions: both use DROID pretraining, LIBERO fine-tuning, the Qwen3-VL-2B backbone, 50K pretraining steps, 30K fine-tuning steps, and global batch size 256.

Evaluation scope. Beyond LIBERO and LIBERO-Plus, we also include SO-101 real-robot experiments covering in-distribution tasks, novel tasks, and changed object layouts. These provide complementary evidence of practical applicability. We also plan to extend evaluation to additional simulation benchmarks.

W2: Evidence for Cross-View Geometric Modeling

Alternative encoder with simple multi-view fusion. We replace VGGT-Ω with a frozen DINOv2 ViT-L/14, which independently encodes the main and wrist images. Their tokens are concatenated and supplied to the world model, retaining the next-step wrist prediction objective and the same DROID pretraining and LIBERO fine-tuning procedure. As shown in Tables 2–3, this baseline achieves 96.0% on LIBERO and 63.6% on LIBERO-Plus, compared with 97.1% and 75.4% for GWM-VLA. These results show that VGGT-Ω-based cross-view geometric modeling improves policy robustness compared with the alternative of DINOv2 encoding and simple multi-view token concatenation.

Cross-view information with the same geometric encoder. Our wrist-only encoding ablation retains the same frozen VGGT-Ω but supplies only the wrist image to the world-state encoder; the policy still receives both camera views. This ablation achieves 96.2% on LIBERO and 68.2% on LIBERO-Plus, compared with 97.1% and 75.4% for GWM-VLA. These results provide evidence that cross-view geometric modeling improves policy robustness beyond the benefit of the pretrained geometric encoder itself.

Table 2. Full-training ablations on LIBERO.

MethodSpatialObjectGoalLongOverall
GWM-VLA96.899.098.094.497.1
Wrist-only encoding96.298.296.893.696.2
Without register-token conditioning96.498.697.894.496.8
DINOv2 encoding + concatenation95.898.496.693.296.0
VGGT-Ω Spatial Forcing96.697.896.893.296.1

Table 3. Full-training ablations on LIBERO-Plus.

MethodCameraRobotLanguageLightBackgroundNoiseLayoutOverall
GWM-VLA57.954.789.895.490.872.577.175.4
Wrist-only encoding42.447.484.792.987.862.773.368.2
Without register-token conditioning55.954.089.293.789.970.476.474.1
DINOv2 encoding + concatenation34.745.488.691.190.245.566.563.6
VGGT-Ω Spatial Forcing37.342.980.688.187.759.676.565.5

W3: Full-Training Ablations and Attribution of the Gains

Full-training ablations. We extend the multi-view aggregation and register-token conditioning ablations to the complete DROID pretraining and LIBERO fine-tuning pipeline, as reported in Tables 2–3 (“Wrist-only encoding” and “Without register-token conditioning”). Cross-view aggregation contributes the larger improvement, with 7.2 percentage points on LIBERO-Plus, while retaining register-token conditioning improves success by 1.3 points compared with removing it. These results highlight cross-view aggregation as the more influential component and register conditioning as a complementary contribution.

Geometric features versus latent world modeling. To disentangle the benefit of pretrained geometric features from that of shaping latent actions through latent world modeling, we add a Spatial-Forcing-style baseline [4] using the same frozen VGGT-Ω. This baseline replaces the world predictor and future-state prediction objective with current-frame geometric alignment: VGGT-Ω jointly encodes the main and wrist images, and its final-layer cross-view features supervise the corresponding Qwen layer-21 visual tokens through a learnable projection and cosine-alignment loss with weight 0.5. Action supervision and the complete DROID pretraining and LIBERO fine-tuning pipeline are retained. As shown in Tables 2–3, this baseline achieves 96.1% on LIBERO and 65.5% on LIBERO-Plus, compared with 97.1% and 75.4% for GWM-VLA. Both methods use the same pretrained geometric teacher, but GWM-VLA uses its representations to supervise future-state prediction and thereby shape shared latent actions. The stronger results support the benefit of shaping latent actions through geometry-aware latent world modeling beyond directly aligning visual features with the same pretrained geometric representations.

References

[1] Yuan T, Dong Z, Liu Y, et al. Fast-wam: Do world action models need test-time future imagination?[J]. arXiv preprint arXiv:2603.16666, 2026.

[2] Yang L, Song W, Wang X, et al. 4d-wam: Infusing spatiotemporal awareness into world action models through trajectory fields[J]. arXiv preprint arXiv:2608.08023, 2026.

[3] Kim M J, Gao Y, Lin T Y, et al. Cosmos policy: Fine-tuning video models for visuomotor control and planning[J]. arXiv preprint arXiv:2601.16163, 2026.

[4] Li F, Song W, Zhao H, et al. Spatial forcing: Implicit spatial representation alignment for vision-language-action model[C]//International Conference on Learning Representations. 2026, 2026: 132324-132345.

We greatly appreciate it if you could take a look at our first-round responses. Please let us know if you have any further comments or suggestions. We would be happy to make additional improvements and engage in further discussion.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.