acceptodds
Withdrawn from ICLR 2027

CoGeo-WAM: Controllable Geometry Adaptation for World Action Models

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

World action models (WAMs) have demonstrated impressive robotic manipulation capabilities by jointly modeling future visual dynamics and actions. However, their visual representations are largely learned from RGB videos without explicit geometric perception, creating a mismatch with the inherently 3D nature of physical interaction. Existing approaches can strengthen spatial awareness through explicit geometry modeling, but often introduce additional geometric inputs. Moreover, geometry-oriented adaptation can alter useful pretrained representations, while its utility varies across manipulation contexts, making uniform geometry adaptation potentially suboptimal. To this end, we propose CoGeo-WAM, a framework for controllable geometry adaptation in world action models that combines implicit geometry alignment with adaptive geometry control. We align intermediate WAM video representations with 3D-aware features from a frozen pretrained 3D vision model and capture the resulting geometry-aligned adaptation in a LoRA residual branch, requiring no explicit geometric inputs or additional geometry model at inference time. A context-guiden controller then dynamically modulates this branch, allowing the model to selectively exploit geometric adaptation and preserve its pretrained video-action prior in different manipulation contents. Extensive experiments across multiple simulation benchmarks, real-world manipulation tasks, and different WAM backbones demonstrate that CoGeo-WAM consistently improves manipulation performance and outperforms both the original world action models and static geometry adaptation.

Primary area:
applications to robotics, autonomy, planning
Keywords:
World Action Models, 3D Representation Learning, Robot Manipulation

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_K22hOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes CoGeo-WAM, a framework for incorporating and controlling geometry-related adaptation in pretrained world action models. After calibrating an alignment projector, the first stage trains LoRA residuals using video and action objectives together with feature alignment to a frozen VGGT teacher. The second stage freezes these residuals and learns a controller that modulates their contributions using intermediate hidden states, block identity, and flow time. Evaluation covers LingBot-VA on RoboTwin 2.0 and FastWAM-Joint on LIBERO and four real-world Franka manipulation tasks.

Critical strengths
  1. The problem is well motivated, and the adaptation interface is practical. Table 4 shows that uniform geometry adaptation can help some tasks while hurting others, motivating control over the learned residual. The framework preserves the original input modalities and does not require VGGT or explicit geometric inputs at inference. The measured memory increase is below 1% for the LingBot-VA configuration (Tab.6), supporting its practical appeal.

  2. The evaluation includes meaningful manipulation settings. The experiments cover two WAM implementations, two simulation benchmarks, and physical execution on a Franka robot. The real-world tasks include cup nesting, sequential block stacking, and pouring, providing useful evidence of applicability to different interaction requirements.

Critical weaknesses
  1. The paper does not sufficiently establish what geometric capabilities are learned or how they explain the policy improvements. The appendix gives plausible descriptions of spatial localization, gripper clearance, and contact requirements, but these descriptions are not independently tested. Specifically

    • Fig.4 measures how well features from the frozen original WAM can be mapped to VGGT features; it supports layer selection rather than demonstrating improved geometry after adaptation.
    • Fig.5 illustrates gate variation during one task, without establishing that the variation responds to geometric demands or causes better execution.
    • Tab.5 measures RGB prediction quality rather than specific 3D capabilities. Moreover, the Stage-I residual is jointly optimized with video, action, and alignment losses, so its benefits cannot be attributed specifically to geometry without a matched no-alignment control. These gaps leave the connection between task requirements, acquired geometry, and policy advantages unclear, limiting the insight provided by the central geometry-aware framing.
  2. The incremental value of the more involved training pipeline over related approaches is not sufficiently demonstrated. For example, the aggregate number of real-world successes increases only from 62/80 to 67/80. With just 20 trials per task, this gain is too small to be meaningful.

Questions for authors
  1. What specific geometric capability improves after adaptation? Could you provide a focused probe, controlled spatial variation, or failure analysis that links this improvement to a relevant manipulation requirement, beyond VGGT feature similarity and overall success rate?

  2. How does the full method compare with the same LoRA/controller procedure without geometric alignment, and with static learnable block-wise gates trained under a matched budget? Is there a real-world Uniform Geometry comparison, particularly on Block Stacking, that clarifies the contribution of dynamic control?

  3. Please specify the LoRA rank, scaling and insertion modules, controller architecture, and data and optimization budgets for projector calibration and both training stages.

Reviewer_3fDGOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper asks whether world action models (WAMs) can gain 3D awareness without geometric inputs at inference, and whether the strength of that adaptation should depend on context. CoGeo-WAM trains in two stages. In Stage I, a LoRA residual is learned on a frozen WAM with the WAM's video and action losses plus a cosine alignment term. This term pulls block-20 video tokens, through an MLP projector, toward frozen VGGT features. In Stage II, the LoRA is frozen, and a controller predicts one positive gate per DiT block from pooled hidden states, a block embedding and the flow time. The gate scales that block's LoRA residual. VGGT and the projector are discarded at inference. The paper reports 91.06 → 91.51 on 50 RoboTwin 2.0 tasks (LingBot-VA backbone), 97.7 → 98.7 on LIBERO (FastWAM-Joint backbone), and 77.5 → 83.75 on four Franka tasks with 20 trials each, together with a controller-granularity ablation on four RoboTwin tasks.

Critical strengths
  1. A cheap, backbone-agnostic recipe with a sensible question behind it. Distilling geometry into a WAM without paying for it at inference is a practical goal, and asking whether geometry should be applied uniformly is a fair question that prior alignment methods do not address. The recipe is simple and adds under 1% memory at inference. It is applied to two different WAMs, evaluated in two simulators and on a real robot, and the alignment layer is chosen by a probe rather than by hand. The full per-task RoboTwin table (Table 7) is appreciated.

  2. A transparent and well-polished paper. I thank the authors for their candour. They report per-task results for all 50 RoboTwin tasks in both settings (Table 7), including the many tasks on which CoGeo-WAM does not help. App. C states plainly which numbers were re-evaluated and which were taken from prior work, and Fig. 5 shows the gate deviations on their actual scale. This openness made the analysis below possible, and it is the reason my concerns can be stated precisely. The paper is also clearly written and presented: the two-stage design is easy to follow from Fig. 2, the method section is concise, and the controller variants in Table 4 are defined precisely enough to reproduce.

Critical weaknesses
  1. The main RoboTwin gain is within evaluation noise and comes from the tasks used to choose the controller. The average gain is 0.45 points, and the randomized setting decreases (90.76 → 90.58). Per task (Table 7), CoGeo-WAM beats the re-run LingBot-VA on 18 clean tasks and loses on 23. The four tasks of the controller ablation (Table 4) carry identical numbers in Table 7, so the same runs serve for selection and reporting. On the other 46 tasks, the averages are 92.85 vs. 92.87 (clean) and 91.17 vs. 91.74 (randomized), so there is no gain outside the selection set. Moreover, the comparison mixes protocols. App. C states that LingBot-VA was re-evaluated by the authors, while the other methods use published results. The re-run is lower than the published LingBot-VA results (92.93 / 91.55; Li et al., "Causal World Modeling for Robot Control", RSS 2026), which are above CoGeo-WAM in both settings; for example, Open Microwave clean is 64 here and 82 in the original paper. The gap between two evaluations of the same backbone is therefore larger than the claimed improvement, and against published numbers the "state of the art" claim does not hold. No seeds or confidence intervals are reported. The real-robot result (67/80 vs. 62/80, Fisher's p ≈ 0.4) does not resolve this. As a result, the claim of "consistent improvement" is not yet supported (plz refer to Q1, Q3, Q4).

  2. Neither the geometry nor the adaptivity is isolated. Stage I is close to Spatial Forcing (Li et al., ICLR 2026) and Geometry Forcing (Wu et al., ICLR 2026), which the paper cites, so the contribution rests mainly on the controller and on the claim that geometry is the source of the gain. Neither is tested directly. There is no LoRA trained with λ_align = 0 under the same rank, data and steps. With λ_align = 0.005, most of the LoRA gradient comes from the task losses, so the gain may come from extra fine-tuning rather than geometry. Uniform Geometry, the natural static control, is reported only on the four selection tasks, not on the full RoboTwin suite, LIBERO or the real robot. The gates also vary little as shown in Fig. 5. Finally, the motivating example points the other way: on Pick Diverse Bottles, CoGeo-WAM is below LingBot-VA (93/76 vs. 95/87), 11 points lower in the randomized setting (Q1, Q2).

Questions for authors

A. Requests that could change my rating:

  1. Please report three experiments on the full RoboTwin suite and on LIBERO: (a) the same LoRA trained with λ_align = 0 and the same rank, data and steps; (b) Uniform Geometry, i.e., the Stage I LoRA without a controller; and (c) CoGeo-WAM. (b) is in effect Spatial Forcing applied to a WAM with LoRA, so it also serves as the natural comparison with prior alignment methods. If (b) beats (a), the geometry claim holds; if (c) beats (b) beyond the seed spread, the adaptivity claim holds. If only the first holds, the paper could be reframed around geometry alignment for WAMs.

  2. Please show whether the controller is adaptive in practice. What are the across-task, across-episode and across-flow-time variances of g_l? What happens if the gates are frozen at their dataset means at inference, or replaced by learned per-block static scalars trained with the same Stage II budget? If these match CoGeo-WAM, the gain comes from block-wise rescaling rather than context.

B. Clarifications:

  1. Why does your LingBot-VA evaluation differ from the published one (e.g., Open Microwave 64 vs. 82)? Did you use the released checkpoint, and what are the differences in sampling, execution horizon and protocol? How were the four Table 4 tasks chosen?

  2. In Table 1, only LingBot-VA is re-run, while the other rows are published results (App. C). Since CoGeo-WAM is compared with a re-run backbone but with published competitors, please either re-run the competitors under your protocol or also report CoGeo-WAM against the published LingBot-VA numbers. π0.5 appears at 42.98 / 43.84, which seems to follow a single-task protocol, whereas the LingBot-VA paper reports a multi-task π0.5 at 82.7 / 76.8. In Table 2, was FastWAM-Joint on LIBERO re-run under the same protocol as CoGeo-WAM?

  3. Please give the LoRA rank and placement (in particular, whether adapters exist after block 20, which receive no alignment gradient), the controller architecture, the optimizer and steps for the calibration, Stage I and Stage II, the training data per benchmark, and the training compute. Does the block-20 probe cosine (Fig. 4) actually increase after Stage I? Will code be released?

  4. Are the Table 5 videos generated open-loop or closed-loop? Treating VBench Dynamic Degree as "higher is better" is questionable for robot videos, since more motion is not necessarily better, and a 0.2 dB PSNR difference is small. Since geometry is the property the method targets, a geometric-consistency metric (e.g., multi-view or depth consistency of predicted frames) would be more informative.

Reviewer_p9frOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

CoGeo-WAM introduces a two-stage method for geometry adaptation in world action models. It first trains a LoRA branch by aligning intermediate video features with a frozen VGGT teacher, alongside the original video and action losses. It then freezes the LoRA weights and trains block-level gates conditioned on pooled hidden states, block identity, and flow time. The teacher and alignment projector are removed at inference. The authors evaluate the method on RoboTwin 2.0 with LingBot-VA, and on LIBERO and four real-world Franka tasks with FastWAM-Joint.

Critical strengths

A useful and practical formulation of selective geometry adaptation. The observation that uniform geometry adaptation can help some manipulation tasks while hurting others motivates controlling its contribution. The two-stage design provides a straightforward interface for doing this through a frozen residual and learned gates.

Promising gains on difficult manipulation tasks, supported by evaluation across multiple settings. The evaluations cover two WAM backbones, simulation benchmarks, and physical manipulation. Table 3 reports an improvement from 60% to 75% on real-world Block Stacking. These results suggest useful potential for articulated and contact-sensitive manipulation, and motivate further investigation of the proposed adaptation mechanism.

Critical weaknesses

My main concern is the comparison with static adaptation. Section 4.3 clearly defines a gate that depends on the current hidden states and flow time. What is missing is evidence that this dependence improves performance over a learned constant gate.

Since Eq. (4) also trains the residual with video and action losses, i think a plain LoRA baseline without geometry alignment is needed to judge the contribution of geometric supervision.Without this control, it is difficult to attribute the improvement specifically to the proposed geometry adaptation

The overall gains are small, and the task-level results are mixed. RoboTwin improves by 0.45 percentage points on average, while randomized performance drops from 90.76% to 90.58%.Under randomization, Handover Block falls from 94% to 78%, and Pick Diverse Bottles from 87% to 76%. These declines deserve attention because preserving useful pretrained behavior is part of the motivation.

The paper describes projector calibration and two adaptation stages, but does not provide enough detail about the data and optimization budget for each step or how these compare with the backbone baselines. Important settings, including the LoRA rank, adapted layers, learning rates, and training steps, are also missing from the provided manuscript.

Questions for authors

Could you compare the full controller with learned per-block constant gates, using the same Stage-I checkpoint and Stage-II training budget? This would help determine whether conditioning on hidden states and flow time provides an advantage beyond learning suitable residual scales.

Could you include a plain LoRA baseline trained without geometry alignment, using the same data and a matched training budget? Ideally, reporting its performance both before and after controller training would show whether geometric supervision contributes beyond ordinary adaptation and gating.

What failures account for the declines on randomized Handover Block and Pick Diverse Bottles? Do the gates suppress the geometry branch in these cases, and does this differ between successful and failed rollouts?

Please clarify the data, starting checkpoints, learning rates, and training steps used for projector calibration, Stage I, Stage II, and the backbone baselines.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.