acceptodds
Withdrawn from ICLR 2027

GuardWAM: Mode-Conditioned Structural Priors for Contact-Rich Video World Models

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Video world action models (WAMs) predict future video conditioned on robot actions, yet visual plausibility does not ensure action faithfulness: when contact changes, imagined motion may violate what the action can physically cause. Using physics-grounded paired probes, we find that a frozen LingBot-VA checkpoint localizes action effects significantly worse at contact onset than in matched stable controls, despite unchanged global visual quality. Neither a mode token nor global identity/inverse/composition regularization repairs this failure; the latter improves reversible free-space behavior while inducing false restoration at irreversible transitions. This reveals a missing principle: structural relations should hold only within interaction-dependent validity domains. We instantiate this principle in GuardWAM. An interaction guard predicts mode, transition events, and executability; a deterministic validity compiler activates only valid structural constraints; and mode-flow plus boundary jump/reset adapters model within-mode evolution and discontinuous transitions. On held-out RoboTwin probes, Oracle GuardWAM reduces contact-boundary false restoration and false success to roughly one third of the frozen WAM while preserving free-space consistency. A learned guard retains about 90% of this repair, identifying guard accuracy as the remaining bottleneck.

Primary area:
applications to robotics, autonomy, planning
Keywords:
video world models, world action models, contact-rich manipulation, hybrid dynamical systems, action faithfulness

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_1DSYOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

GuardWAM addresses action-faithful video prediction in contact-rich robot manipulation. Its central hypothesis is that structural constraints such as identity, inverse consistency, and composition should be applied only within interaction-dependent validity domains. The method predicts interaction modes, transition events, and action executability, then uses a deterministic compiler to select applicable structural losses. Mode-specific residual adapters model within-mode dynamics, while event-conditioned adapters handle boundary transitions. Additional objectives supervise spatial structure, counterfactual effects, and failure outcomes.

Critical strengths
  1. The proposed validity-domain perspective is physically meaningful. Reversing an end-effector command need not reverse the resulting object motion after unilateral contact, slip, or release. Distinguishing these cases from reversible free-space motion is a useful extension of group-action structural regularization. Component-local constraints and stable-grasp relative-transform consistency provide concrete ways to express this distinction.

  2. The reported experimental design targets relevant alternative explanations. The paper includes generic adapters, spatial supervision, mode conditioning, global structural regularization, and variants removing validity gating or boundary adapters. It also reports three-initialization statistics and a second backbone. If the evaluation and supervision controls are clarified, this design could provide useful evidence that the benefit extends beyond additional capacity or a mode label alone.

Critical weaknesses
  1. The empirical evidence is insufficiently specified and contains unresolved inconsistencies affecting the motivating claim. The introduction reports a contact-onset IoU decrease of 0.0821 with a 95% confidence interval of [−0.1023, −0.0628], whereas Figure 1 and Figure A1 report approximately −0.0418 for the fifteen matched probes. Figure A1 additionally gives an interval of [−0.067, +0.011] at the 8/255 threshold. The direction is consistent, but statistical significance is not established at every threshold. The manuscript does not explain whether these numbers describe different analyses. More importantly, the main evaluation is not operationally defined sufficiently to assess the reported gains. The paper does not specify the eight task identities, dataset sizes and split construction, evaluation counts per mode, decision thresholds and denominators for false restoration and false success, or the normalization of ranking regret. The candidate-action generator, future-scoring function, planning horizon, and closed-loop evaluation budget are also missing. It is unclear whether generated outcomes are assessed by an independent evaluator or the method’s trainable structure readout. These omissions directly affect the interpretation of the central results, including the reported closed-loop improvement.

  2. The experiments do not yet isolate the claimed structural-validity mechanism from interface changes and additional supervision. Section 3.1 states that the Phase 0 public LingBot-VA inference path does not accept a fixed external candidate action. Consequently, its contact-onset reconstruction errors do not by themselves establish failure to follow a specified action. The subsequent action-swap test establishes sensitivity—two candidate actions produce sufficiently different motion masks—but does not establish that either predicted effect matches its corresponding action. Without shared generation noise and an action-aligned ground-truth comparison, stochastic variation or incorrect sensitivity remains a possible explanation. Furthermore, the full method adds counterfactual-effect, background-invariance, and failure-ranking objectives alongside guarded structural losses. Although parameter counts, data, and budgets are described as matched, the manuscript does not provide a per-baseline accounting of these objectives and supervision signals. The composition and coupling ablations are useful, but do not resolve this broader attribution issue. Establishing that the validity compiler drives the repair requires comparisons holding these auxiliary objectives and oracle information fixed.

Questions for authors
  1. Clarification of the motivating evidence: Which analysis produces the 0.0821 IoU decrease and its confidence interval, and how does it relate to the approximately 0.0418 decrease in Figures 1 and A1? Please provide the corresponding probe definitions and clarify the significance claim across thresholds.

  2. Evaluation protocol: Please specify task identities, train/test split units, evaluation sample counts, operational definitions of false restoration/false success and ranking regret, and the complete closed-loop planning procedure. Are outcomes measured independently from decoded videos, or using a method-specific trainable readout?

  3. Action-interface validation: Were action-swap futures generated using identical noise and sampling settings? Can you show that each candidate action’s predicted effect matches its simulator outcome, rather than merely demonstrating that different candidates change the generated video? Was the Phase 0 boundary diagnosis repeated through this validated action-conditioned interface?

  4. Mechanistic attribution: Which baselines receive each auxiliary loss and each mode/event/executability label? Does the full method retain its advantage when all non-structural losses and supervision are held fixed? Also, what does “Oracle” mean in Table A3, where oracle mode/event/executability accuracy is 0.942 rather than perfect?

Reviewer_ecmAOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper addresses the lack of physical consistency in action-conditioned video world models, particularly at contact transitions where globally enforced structural constraints may produce physically implausible predictions. The authors propose GuardWAM, which predicts interaction modes and events, selectively applies valid structural constraints, and uses mode-specific and jump/reset adapters to model contact dynamics. Experiments on RoboTwin 2.0 with two backbones demonstrate improvements in action-effect prediction, physical consistency, and closed-loop task success.

Critical strengths
  • The paper identifies an important limitation of video world models: visual plausibility does not necessarily imply physically faithful action-conditioned predictions. The distinction between reversible free-space motion and potentially irreversible contact interactions is well motivated. In particular, the observation that globally enforced structural consistency can improve free-space behavior while worsening predictions at irreversible transitions provides a compelling motivation for considering the validity domains of physical constraints.

  • The paper includes architecture-matched baselines, ablations of mode conditioning, validity gating, and jump/reset mechanisms, as well as comparisons between oracle and learned guards. The reported improvements across two different backbones are encouraging. These experiments help distinguish the proposed approach from simply adding interaction-mode tokens or increasing model capacity.

Critical weaknesses

False Restoration and False Success are central to the paper's empirical claims, yet their operational definitions and computation procedures remain unclear. It is not specified how these events are detected from predicted video rollouts, which physical states or thresholds determine a violation, or how evaluation samples are aggregated.

The action proposal and closed-loop planning protocols are insufficiently described, making it difficult to attribute the reported control improvements. Although the paper reports substantial improvements in closed-loop task success, the source of candidate action sequences and the downstream planning procedure are not adequately specified. It is unclear whether candidate actions are generated by a pretrained policy, sampled from demonstrations, or obtained through another mechanism. The action-scoring objective, candidate set size, execution horizon, and replanning procedure are also not clearly described.

Lastly, the proposed system is evaluated primarily through an action-conditioned future-prediction interface rather than direct action generation. Its positioning as a World Action Model, rather than an action-conditioned video world model built upon pretrained WAM backbones, would benefit from clarification.

Questions for authors
  • Could the authors provide formal definitions and detailed computation procedures for False Restoration and False Success? Specifically, how are these events identified from generated videos?

  • Where do the candidate action sequences originate, and how are they evaluated during closed-loop planning? Please clarify the proposal mechanism, number of candidates, scoring function, action execution horizon, replanning frequency, and final success evaluation protocol. Are the same candidate-generation and planning procedures used for all baselines? How is action-ranking regret computed?

Reviewer_MzZAOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies video prediction when robot contact changes. GuardWAM predicts interaction modes, events, and executability. Fixed rules select valid structural constraints, while adapters model motion and boundary changes. Tests with two backbones in RoboTwin report fewer impossible outcomes and better closed-loop success. The learned model needs no simulator labels at inference.

Critical strengths
  • Using interaction states to select constraints is a useful idea. Boundary adapters also model what happens when contact changes, instead of only removing wrong constraints.

  • The tests go beyond video quality. They include action errors, several ablations, two backbones, and closed-loop tasks. Learned LingBot-VA improves reported success from 62.5% to 82.3% (Table 1). This supports the combined design in simulation.

Critical weaknesses
  • The authors report matched parameters, data, and budgets, but the ablations do not clearly isolate validity gating. No Structure removes structural losses, while No Jump removes the boundary adapter. Global Structure is useful, but it is unclear whether local supervision, coupling, and auxiliary losses stay the same. The combined design has stronger support than the claim that the compiler carries the repair.

  • The main evaluation is hard to check. Test counts, split rules, error-counting details, and the planner setup are not clear. Three-seed statistics are helpful but do not replace these details. Also, the Phase 0 IoU drop differs between the text and figures.

Questions for authors
  • Does Global Structure keep the same inputs, adapters, local supervision, coupling, and auxiliary losses, changing only validity gating? If not, is such a matched comparison available?
  • Please give test counts and split rules, explain how false restoration and false success are counted, and specify candidate generation and scoring. Is the planner and rollout budget shared?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.