acceptodds
Under review as a conference paper at ICLR 2027

G2P-WAM: From Geometry Distillation to Preference Alignment for World-Action Models

Abstract

World-action models (WAMs) jointly predict future observations and robot actions, yet visually plausible predictions do not guarantee successful execution. Geometric distillation improves representation grounding, but using geometry for subsequent policy improvement requires relating geometric scores to task outcomes. We propose G2P-WAM, a post-training framework that carries geometric supervision from representation learning into preference alignment. GeoSFT first aligns WAM features with static 3D representations and dynamic 4D trajectories. Reward diagnosis then evaluates candidate geometric scores against execution outcomes and compares their behavior on imagined and observed videos. GeoDPO uses outcomes to order preferences and reconstruction confidence to select trajectories for reference-anchored optimization. Experiments on simulation benchmarks and two physical-robot platforms show that preference alignment further improves the geometrically distilled policy. Joint static and dynamic supervision outperforms either branch alone, while confidence filtering improves on outcome-only preference training with matched pair counts and optimization settings. These results support using geometry for both representation learning and preference selection, without adding geometric teachers or reward scorers at deployment. Project page: https://anonymous.4open.science/w/G2P-WAM-F7BF/.

Primary area:
applications to robotics, autonomy, planning
Keywords:
World-Action Models, Geometry Distillation, Direct Preference Optimization

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_cG5eOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes G2P-WAM, a post-training framework connecting geometric representation learning with preference alignment for world-action models. GeoSFT supervises WAM video representations using static VGGT features and dynamic motion targets from SpatialTrackerV2. A diagnostic stage evaluates geometric scores against execution outcomes on imagined and observed videos, selecting 4RC point confidence for trajectory filtering. GeoDPO then uses outcome-ordered, confidence-filtered pairs for reference-anchored preference training. Experiments report modest aggregate improvements on simulation benchmarks and a larger average gain over GeoSFT on three physical-robot tasks, without adding inference-time modules.

Critical strengths
  1. The framework explicitly separates geometric supervision, score diagnosis, and preference construction. Using execution outcomes to determine preference order and reconstruction confidence to filter candidates avoids treating geometric plausibility alone as evidence of task success.

  2. The ablations compare static-only, dynamic-only, and joint geometric supervision, and include a count-matched outcome-only DPO control. These comparisons provide targeted evidence for the contributions of joint supervision and confidence-based selection.

  3. Evaluation covers simulation benchmarks, video-quality benchmarks, and physical-robot tasks. On the three PiPER X tasks, preference alignment improves average success from 77.8% to 85.6%, while deployment requires no geometric teachers, reconstruction scorer, or reference model.

Critical weaknesses
  1. The main GeoSFT calibration uses 122 rollouts from a single task, while the same confidence-filtering direction is applied across tasks. Given the task- and policy-dependent score behavior reported in the diagnostics, the reliability of this selection rule across the target rollout distributions remains insufficiently established.

  2. The gains attributable to the key components are modest on RoboTwin. Joint GeoSFT exceeds dynamic-only supervision by 0.06 percentage points, and confidence-filtered GeoDPO exceeds count-matched outcome-only DPO by 0.31 points. Although bootstrap intervals are reported for the full GeoDPO-stage improvement, the stability of these component-level gains across training runs is not established.

  3. The preference-data construction is not fully quantified. Per-task rollout counts, success/failure pool sizes, and training-pair counts are not clearly reported, making it difficult to assess data coverage, trajectory reuse, and the execution cost required to reproduce the preference-training stage.

Questions for authors
  1. The count-matched comparison supports the usefulness of confidence filtering. However, Appendix A.13 also shows that filtering changes the estimated pose-motion distribution. Does reconstruction confidence provide additional benefits over selection based on this motion proxy? A comparison under matched pair counts and training settings would help clarify the contribution of the geometric score.

  2. How does GeoSFT compare with continued SFT using only the original flow-matching objective under the same data and training budget? Since A0 is the official checkpoint without additional training, this comparison would help separate the benefit of geometric supervision from that of further fine-tuning.

Reviewer_ECbnOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

World action models can generate realistic videos and actions yet still fail at physical manipulation. The proposed method, G2P-WAM, explores whether geometric information can help improve a policy after its initial training. It improves a world-action model in three stages. First, GeoSFT trains it with static 3D features from VGGT and dynamic 4D trajectories from SpatialTrackerV2. Next, reward diagnosis tests geometric scores on the model’s own rollouts and selects reconstruction confidence to filter them. Finally, GeoDPO uses success–failure pairs to update the model with reference-anchored DPO. The paper evaluates the method on simulation benchmarks and two robot platforms. The authors report higher success rates after preference training, and ablations show gains from combining static and dynamic supervision and from confidence-based trajectory filtering.

Critical strengths
  1. The paper uses geometry not only to improve the model’s representations, but also to choose trajectories for later policy training. This gives the method a clear connection between geometric distillation and preference learning.

  2. GeoSFT uses both static 3D scene features and dynamic 4D motion information. This is a good fit for manipulation, where the model needs to understand both the scene and how objects move. The ablation results support using both types of supervision.

Critical weaknesses

None

Questions for authors

Clarification Question: How much do the results change with different confidence-filtering ratios? For example, would using ratios close to the reported 50% keep similar gains?

Reviewer_bsRSOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper post-trains a world-action model (LingBot-VA) in three stages. GeoSFT adds two training-only losses: video features are aligned with frozen VGGT features, and a readout head predicts 3D point motion from SpatialTrackerV2. A diagnosis stage then scores imagined rollouts with a frozen 4D reconstructor (4RC) to find which geometric score separates successful from failed executions, and selects point confidence. GeoDPO runs diffusion-style DPO on success/failure pairs from the same task, where outcomes set the preference and confidence decides which rollouts enter the pairs (the most confident half of successes and the least confident half of failures). Results are reported on RoboTwin 2.0, LIBERO, LIBERO-Plus and two real robots.

Critical strengths
  1. All geometry modules are used only in training and removed at deployment, so the deployed policy keeps LingBot-VA's structure and inference cost (Fig. 1, p. 4). The method is tested on three simulation benchmarks and two real robots, with a per-task table for all 50 RoboTwin tasks (Table 6). The RoboTwin gain and the real-robot success rates also come with confidence intervals (p. 8, Table 8).

  2. The paper's search for a geometric reward is useful mainly for figuring out what did not work. Reprojection consistency barely separates successes from failures (AUC 0.553 / 0.510, Table 13), whether a higher geometric score means success or failure changes from task to task (A.9), and in exploratory runs, using VGGT reprojection directly as a reward makes the policy collapse (Table 14).

Critical weaknesses
  1. In GeoDPO, the preference comes only from task success, and geometry only decides which rollouts are kept (p. 5). The paper does not explain why the reconstruction confidence of predicted frames should tell good actions from bad ones. The rule that higher confidence is better was fixed on one task (Turn Switch) and was not checked on GeoSFT rollouts of any other task. On the official policy's rollouts, higher confidence does not go with success on five of the six RoboTwin tasks analysed (Fig. 13c). Fig. 2 lists "motion leakage" as one of the checks, yet confidence is clearly related to estimated motion: on Turn Switch, low estimated motion separates successes from failures at least as well (A.12), and the filter tends to keep successes with less estimated motion and failures with more (A.13). The comparison with random pairs in Table 3 cannot show what confidence adds beyond this.

  2. I am not convinced that GeoDPO learns which actions are better. In DPO, the two samples of a pair share the same input, so their difference reflects only the model's output. Here a pair is a whole successful episode and a whole failed episode of the same task, and the two may start from different scenes (A.13). The paper does not say whether Eq. 9 is applied to every action of an episode or only to some of them. A failed episode can contain many reasonable actions, and its failure may come from a harder starting scene rather than worse actions. The pairs may then mostly teach the policy to repeat its successes, which fine-tuning on the successful episodes alone would also do, but this comparison is missing. On RoboTwin, the only benchmark where the DPO stage is ablated (Table 3), GeoDPO adds 0.49 points, with an uncertainty of about ±0.3 points (p. 8). The larger gains on LIBERO-Plus (2.7 points) and the real robots come without such comparisons.

Questions for authors
  1. Confidence was checked on GeoSFT's own rollouts only for Turn Switch. How well does it separate successes from failures on GeoSFT rollouts of the other tasks? Given that Fig. 2 lists motion leakage as a check, what does confidence add beyond sorting rollouts by estimated motion?

  2. The success or failure label belongs to a whole episode. When a failed episode is used as the loser, are all of its actions treated as bad in Eq. 9, or only some of them, for example those near the failure? How does GeoDPO compare with fine-tuning GeoSFT on its own successful rollouts with the same data and budget?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.