acceptodds
Under review as a conference paper at ICLR 2027

Verifiable 4D World-Action Model for Robust Robot Control

Abstract

World-action models couple imagined visual futures with robot actions, but RGB-only imagination requires recovering geometry and motion from appearance that can change across scenes. We propose S4NDBOX, a 4D world-action model that jointly predicts RGB, depth, view-specific geometric motion fields, and robot actions, explicitly connecting visual evolution to 3D robot motion. The explicit 4D future supports action generation and candidate search based on geometric agreement between predicted futures and paired actions before execution. Across both simulated and real-world distribution shifts, S4NDBOX improves robustness over RGB-only control. On RoboTwin, after training only on clean scenes, S4NDBOX improves success in the hard (domain-randomized) setting by 13.3 percentage points over the RGB-only baseline. We further evaluate zero-shot simulation-to-real transfer on Stack Cups, training on simulated demonstrations and deploying on a physical robot without real-world task demonstrations or fine-tuning. Under additional shifts in object pose, lighting, and clutter, with camera-pose shifts additionally evaluated in the single-view setting, our 4D model improves the mean task score over RGB-only by 41.7 and 40.0 points in three-view and single-view settings, respectively. Website: https://demo1-indol-gamma.vercel.app/.

Primary area:
applications to robotics, autonomy, planning
Keywords:
World Action Model

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_NNRsOpenReview ↗
Rating
2(+1)
Confidence
1(−3)
Informativeness
3(+2)
Presentation
3
Soundness
3(+2)
Read review
Paper summary

This paper propose S4NDBOX, a 4D world-action model that jointly predicts RGB, depth, view-specific geometric motion fields, and robot actions

Critical strengths

The experiments looks good.

Critical weaknesses
  1. Fig 1 and Fig 2 are obviously generated by AI, at least are AI format. And there is even not any conclusion section in the paper. I do not want to waste my time on such a paper. And I think so do the other readers.

  2. the novelty is also very limite, just a multi-task training.

Questions for authors

see weakness

Reviewer_qHNdOpenReview ↗
Rating
2(−1)
Confidence
2
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes S4NDBOX, a 4D World-Action Model that jointly predicts future RGB, depth, fixed-view optical flow, and wrist-camera world-frame rays in a shared VAE latent space before causally generating robot actions. It also introduces a pre-execution verification mechanism to rank candidate actions based on geometric agreement between the decoded 4D future and action-implied kinematics. Experiments show strong OOD generalization on RoboTwin Hard and zero-shot sim-to-real transfer on a Franka robot.

Critical strengths
  • Decoupling fixed-view optical flow for 3D translation and moving wrist-camera rays for 3D rotation is geometrically sound for multi-view manipulation and clearly validated by the modality ablations in Table 3.
  • The model achieves a +13.3% gain on RoboTwin Hard over the LingBot-VA baseline and transfers well zero-shot to the real world, while Table 4 confirms that action decoding actively relies on the predicted 4D tokens.
Critical weaknesses
  • The methodological novelty is somewhat incremental, as the framework directly combines the LingBot-VA backbone with existing depth normalization and polar-RGB flow codecs, closely resembling recent 4D WAMs like RynnWorld-4D and 4D-WAM. Moreover, feeding non-RGB geometric modalities into a frozen RGB VAE without adaptation may distort metric geometry, and computing ground-truth flow requires privileged 3D entity poses from simulation.
  • The proposed verifier only checks robot-arm kinematic self-consistency while ignoring object contact physics and grasping dynamics. Empirically, despite featuring "Verifiable" in the title, it is only evaluated on 1 task for the first action chunk, with a 95% confidence interval of -0.49 to 16.05 in Appendix E.1 that crosses zero, and relies on an impractical multi-attempt setup with environment resets rather than standard closed-loop rollouts.
  • No inference latency is reported for generating 3 visual modalities and verifying 16 candidates. Furthermore, S4NDBOX receives ground-truth depth at test time unlike RGB-only baselines, and dropping observed depth in Table 4 reduces Hard success to 31.2%, falling below the 34.6% RGB-only baseline.
Questions for authors
  1. Beyond combining existing depth and flow codecs into LingBot-VA, could the authors clarify the core technical novelty over prior 4D WAMs and quantify the geometric error introduced by the frozen RGB VAE?
  2. How can the verifier filter out object grasping failures if it only checks robot-arm kinematics, and what is the closed-loop success gain on RoboTwin Hard if the verifier selects the top action at every step without environment resets?
  3. What is the per-step inference latency for S4NDBOX and the extra overhead for verifying 16 candidates, and why does the single-view policy at 68.8 drastically outperform the three-view policy at 45.8 in Table 2?
Reviewer_bCkdOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes S4NDBOX, a world-action model that jointly generates future RGB, metric depth, and view-specific motion representations, namely fixed-camera flow and wrist-camera world-frame rays, before generating an action chunk. The paper further proposes a geometric world-action agreement score for ranking sampled future-action pairs before execution, and evaluates robustness on RoboTwin domain randomization and zero-shot simulation-to-real cup stacking.

Critical strengths
  • The paper introduces a unified framework that jointly predicts RGB, depth, motion fields, and robot actions.
  • S4NDBOX achieves 47.9% success on RoboTwin Hard, outperforming the RGB-only LingBot-VA baseline by 13.3 percentage points. The experiments cover 50 manipulation tasks, and the modality ablations provide useful evidence that geometric information improves robustness.
  • The paper demonstrates zero-shot Sim2Real transfer on a physical Franka robot without real-world task demonstrations.
Critical weaknesses
  • The RGB-only baseline receives RGB observations, whereas S4NDBOX receives RGB-D observations. Therefore, the reported improvements may partly result from access to observed depth rather than explicit future geometry prediction.
  • Geometric agreement between predicted futures and paired actions does not necessarily imply task success. The verifier is evaluated on only one RoboTwin task, and the reported improvement of 7.9 percentage points has a 95% confidence interval that includes zero. Further evidence is needed to establish its reliability for action selection.
Questions for authors
  • Could the authors provide a controlled baseline using the same RGB-D observations but without future depth and motion prediction? This would help isolate the contribution of explicit 4D imagination from that of observed geometric information.
  • How strongly does the proposed world-action agreement score correlate with actual execution success?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.