- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
The paper introduces an event-aligned visual action reasoning framework for World Action Models (WAMs), robot policies that imagine future frames before generating actions. It addresses 1 main concern from previous WAMs: visual imagination follows fixed time intervals, so long transitions (reaching, carrying) take most of the prediction while the short interactions that actually change the object state (grasp, release, press) get little attention. The main idea is to reorganize training trajectories around interaction events: transitions are compressed into a single chunk, while the moment the interaction happens is kept at full temporal resolution. This way the model learns to focus its imagination on the events that change object state and guide the following motions. The backbone is LingBot-VA, plus a small head that skips the padded actions created by this reorganization. The method improves DOMINO success rate by 10.26 points over LingBot-VA and is on par on RoboTwin 2.0.
Critical strengths
- Intuitive, well-motivated idea by shifting the imagination budget from long transitions to the short interactions that change object state.
- Specific, concrete allocation scheme with transitions compressed to one chunk, the moment the state change is visible kept at full resolution, nearby motion compressed into the remaining slots, and no chunk crossing an event boundary. This is a finer-grained design than prior event- or keyframe-based WAMs.
- Strong controlled ablation (Table 4) that isolate the idea itself rather than the amount of data.
- Large gain on dynamic manipulation with +10.3 success rate over LingBot-VA on DOMINO, and better transfer to unseen difficulty levels.
Critical weaknesses
- The novelty relative to event- and keyframe-based world models is not established. WALL-WM already organizes WAM training around semantic events, with variable-length execution chunks. SKIP keeps event frames (approach, contact, grasp, release) and fills the gaps by interpolation. Both are mentioned in a single sentence, with no discussion of how the proposed scheme differs and no experimental comparison. The contribution is effectively the within-event budget allocation and the validity head, but the paper presents event-based organization itself as new. Can the authors clarify?
Compressing transitions discards information that the policy may need, and failure cases are not analyzed. Each transition is resampled into a single chunk, so a long reach or carry is represented by only a few frames, and only the actions paired with those frames are kept as targets (Fig. 2, App. A). The paper treats transitions as carrying no task-relevant information. This assumption is questionable in several common situations:
-
Moving objects. objects can move, so intercepting them depends on approach timing. Because compressed chunks cover variable amounts of real time, the imagined frames no longer have a fixed time step. The model must therefore infer how much time each predicted chunk covers, which matters most precisely when timing is critical.
-
Approach geometry and pre-grasp alignment. For hanging, insertion, or placing with a specific orientation, success is largely decided by the trajectory and orientation during the approach, before contact happens.
-
Choosing the target object. In scenes with several candidate objects, the direction of the approach reveals which object the policy intends to manipulate.
-
Safety and collisions. Obstacle avoidance and collision-free paths are properties of the transition.
Questions for authors
See above, critical weaknesses section. Also, there are misplaced citations in the paper. For examples, in the introduction, one citation appears as (?).
5 comments
W2-1: Does Compressing Transitions Discard Task-Critical Information?
Thank you for raising these practical manipulation scenarios. In this work, our goal is to demonstrate that a World-Action Model can and should imagine future at adaptive temporal rates. Specifically, viewing and predicting the video at variable speeds (e.g., native speed during critical physical interactions vs. or fast-forward during free-space transitions). Since task-critical state changes are predominantly concentrated in interaction phases for general manipulation, resampling each transition into a single chunk serves as a hyperparameter-free instantiation to prove the effectiveness of multi-rate visual imagination.
Before discussing each scenario, we clarify a key design property of our Interaction Taxonomy: Transitions are compressed, not discarded, and pre-contact alignment is preserved in Support Phases. Each transition event is still represented by a full chunk of visual-action slots. More importantly, local motions immediately preceding and following contact, such as pre-grasp alignment, gripper closing before contact, and lifting off after contact, are categorized as the Interaction Event's leading and trailing Support Phases. By Rule 3 (Page 5, Lines 256–257), the start and end frames of support phases are strictly anchored, and our Reduced Context ablation in Tab.4 confirms that retaining these surrounding support/transition slots boosts SR from 37.83% to 42.83%.
Below, we address how our framework handles each of the four scenarios with empirical evidence from the paper:
- Moving objects and interception timing: All 35 tasks in DOMINO are dynamic manipulation tasks with continuously moving objects. Rather than suffering from variable chunk durations, our method improves DOMINO Level 1 SR from 32.57% to 42.83% (+10.26%) (Tab. 1) and achieves state-of-the-art zero-shot transfer to faster, unseen object dynamics in Level 2 (29.4% vs. 16.9% SR) and Level 3 (9.7% vs. 6.8% SR) (Tab. 2). Two mechanisms enable this: (i) Spatial grounding via visual imagination: the model first predicts the visual state showing where the moving object and gripper will meet at the end of the chunk, which grounds the action generation (when we mask access to imagination under the exact same event sampling, SR drops from 42.83% to 30.69%); and (ii) Boundary-aligned closed-loop correction: since a transition chunk never crosses into an interaction event (Rule 1, Page 5, Line 250-251) and real observations replace predicted states after each chunk, any approach timing drift is corrected by a fresh visual rollout right at the interaction boundary before grasping begins (Page 6, Lines 270–273).
- Approach geometry and pre-grasp/placement alignment:
Since pre-contact approach alignment is preserved across the transition-to-interaction boundary and within the leading Support Phase, our method excels on tasks requiring strict geometric and orientation alignment. As shown in Tab. C2 and Tab. D3, on tasks governed by oriented placement (S: placed_oriented), precise insertion, or small-target alignment, our method substantially outperforms LingBot-VA: such as Place Fan (50% vs. 14%), Place Mouse Pad (64% vs. 19%), Move Stapler Pad (20% vs. 10%), Rotate QRcode (34% vs. 26%), and real-world Tennis Ball Insertion (58.67% vs. 42.67%, Tab. C1). - Choosing the target object in multi-object scenes:
Even when an approach is compressed into a single initial chunk, the predicted visual frames explicitly resolve target selection. In Fig. 3, two objects are present: at Chunk 0 (Object approach), LingBot-VA diffuses its attention across both objects and fails, whereas our model immediately concentrates both video attention (Fig. 3a) and action attention (Fig. 3c, red box) onto the intended moving target (22% vs. 11% SR, Tab. C2). - Safety and collision avoidance:
As defined in Sec. 4.1 (Page 7, Lines 355–356), the DOMINO Manipulation Score (MS) explicitly penalizes workspace violations and clutter collisions. Our method improves average MS from 45.55 to 55.92 (+10.37) across all 35 Level 1 tasks (Tab. 1) and from 30.37 to 45.42 (+15.05) on Level 2 transfer (Tab. 2).
W2-2: Failure Case Analysis and Limitations:
Thank you for the suggestion to include a failure case analysis. We have added a detailed comparison of failure episodes against LingBot-VA (across Tab. C2) with qualitative visualizations in Sec. C.7 (page 21), as well as a new subsection on limitations and future work in Sec. 6 (page 10) of the revised manuscript. While our method improves upon or matches the baseline on 26 of the 35 tasks, the primary performance drops occur on Adjust Bottle (48% vs. 81% SR), Move Can Pot (32% vs. 48% SR), and Place Can Basket (24% vs. 37% SR).
-
Analysis of Failure Cases: These tasks require the robot arm to follow a precise trajectory during mid-air transport. Using a single chunk for the entire carry motion may miss important intermediate waypoints. As a result, the arm may move outside the target height range or collide with the basket rim, as visualized in Fig. C2(page 21).
-
Limitation & Future Work: We agree with the reviewer that different task categories carry different information demands across events. We have incorporated this discussion into the revision, noting that extending our framework from a fixed 1-chunk transition rule to task-adaptive transition budgets (e.g., allocating additional chunks when transport displacement or path curvature is large) is an exciting direction for future work.
Q1: Misplaced Citation
Yes — the Fast-WAM citation was broken when reorganizing model names, which is why it renders as (?) . This is fixed in the revision.
Thank you for the detailed feedback and constructive comments! We are delighted that you highlighted our "intuitive, well-motivated idea" with its "finer-grained design", as well as the "strong controlled ablation" and "large gain on dynamic manipulation". In the following, we clarify each of your concerns in detail.
W1: Novelty of Our Event-Aligned Organization and Distinction from WALL-WM & SKIP
Thank you for raising this point and we have added the discussion of the difference between our method and WALL-WM and SKIP. We want to clarify that our contribution is not simply organizing trajectories by event itself. As stated in our first contribution (Sec. 1, Lines 99–102), we propose "an event-aligned visual action reasoning framework that organizes future prediction with task-critical interactions as focus to reason more carefully over crucial state changes. Instead of inheriting a predefined temporal structure from sampled trajectories, our formulation enables the reasoning process to adapt its granularity to the interaction dynamics." Our event-aligned organization structure trajectories into Interaction vs. Transition events and further into native-resolution Evidence vs. anchored Support phases so that a closed-loop WAM adapts its visual reasoning granularity autonomously. This is fundamentally new and distinct from both WALL-WM and SKIP in problem formulation, temporal structure, and inference mechanism:
Difference from WALL-WM (Uniform event clips with external test-time oracles vs. Adaptive-granularity autonomous reasoning):
Difference from SKIP (Offline action-conditioned video interpolator vs. Closed-loop WAM policy):
Regarding Experimental Comparison: