acceptodds
Under review as a conference paper at ICLR 2027

DELE-w0.5: Inferring Action from Future Latent State for Robotic Manipulation

Abstract

Video-generation-based World-Action Models (WAMs) jointly predict dense future visual trajectories and robot actions. For robotic manipulation, we investigate whether action learning can benefit from predicting physical outcomes without reconstructing every intermediate frame. In this paper, we propose DELE-w0.5, which learns robot actions jointly with future latent-state prediction, without video generation. The future latent state models action-relevant physical outcomes and connects world modeling with action learning through shared representations. The principle of DELE-w0.5 is to model how the real world changes under robot actions, rather than how its visual appearance evolves frame by frame. Across 640 real-robot trials on four long-horizon manipulation tasks, DELE-w0.5 achieves the best performance among all compared policies, attaining 62.5% overall full-task success and 81.3% macro ordered-stage progress. It outperforms the strongest baseline by 32.5 percentage points in full-task success and 20.1 percentage points in macro progress.

Primary area:
applications to robotics, autonomy, planning
Keywords:
World Model, Robotic Manipulation

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_aWJ8OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces DELE-w0.5, a robot policy jointly trained to predict action chunks and future visual features using flow matching. A structured attention mask prevents action tokens from attending to future-state tokens, allowing the future branch to be removed at inference. Evaluated against seven baselines on four real-world manipulation tasks, the method achieves 62.5% overall task success and 81.3% average ordered-stage progress.

Critical strengths
  1. The method achieves 62.5% full-task success across four multi-stage manipulation tasks, outperforming the strongest evaluated baseline by 32.5 percentage points.
  2. The attention mask allows future-state tokens to be removed at inference while retaining their supervision during training.
  3. Reporting both ordered-stage progress and full-task success distinguishes partial execution from completion. The tasks include challenging bimanual coordination and articulated-object manipulation.
Critical weaknesses
  1. Prior work such as LaDi-WM and FLARE already explores future latent prediction for robot control. The paper does not clearly establish what its joint flow-matching objective and attention mask add beyond these approaches.

  2. There is no controlled comparison against the same policy trained without future-state supervision. The reported gains therefore cannot be attributed to the proposed mechanism rather than pretraining data or other design choices.

  3. Action tokens cannot attend to future-state tokens, which are removed at inference. This supports future prediction as auxiliary supervision, but does not directly demonstrate “inferring action from future latent state.”

Questions for authors
  1. How much does performance drop when future-state supervision is removed under otherwise identical training conditions?

  2. Does flow matching for future features outperform a simple feature-regression loss? What benefit does the proposed attention mask provide?

  3. What distinguishes the method from FLARE-style future representation alignment, and can this difference be tested under matched conditions?

  4. Given that action tokens cannot access future-state tokens, what precisely does “inferring action from future latent state” mean in this architecture?

  5. Does the method retain its advantage under randomized object positions or systematic execution-time perturbations?

Reviewer_vhqvOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

DELE-w0.5 jointly trains action generation and future-state prediction in frozen DINOv3 feature space using a dual-stream flow-matching architecture. A structured attention mask prevents action tokens from attending to future-state tokens, enabling action-only inference. Evaluation covers four real-world manipulation tasks, reporting 62.5% full-task success and 81.3% macro ordered-stage progress.

Critical strengths
  1. The training–inference design is coherent. The attention mask permits future-state supervision through shared parameters while allowing future tokens to be removed at deployment without changing the action branch’s information access.
  2. The real-world results are encouraging. Table 2 reports 50/80 full successes versus 24/80 for the strongest baseline by this metric. The evaluation uses matched task-specific sample exposure and shared reset rules, with explicit physical success criteria and stage-wise progress analysis.
Critical weaknesses
  1. The methodological contribution is insufficiently distinguished. Fast-WAM already removes future generation at deployment, while VLA-JEPA uses future-latent supervision. DELE’s specific advance over these approaches is not clearly established. Moreover, predicting frozen DINOv3 features does not by itself demonstrate the claimed focus on action-relevant physical outcomes rather than appearance.
  2. The central training objective lacks controlled validation. Neither the main paper nor the appendix reports an action-only control with the future-state loss removed, or a matched comparison of latent versus video supervision. Consequently, the baseline gains do not isolate the benefit of the proposed prediction target from pretraining and other architectural differences.
  3. Validation on standard simulation benchmarks is missing. Evaluation is limited to four fixed-scene real-world tasks, with no results on established suites such as LIBERO, CALVIN, or RoboTwin. While the real-world gains are encouraging, the narrow task coverage limits assessment of generalization across tasks and scenes, as well as comparison with existing predictive policies under standardized protocols.
Questions for authors
  1. What specifically distinguishes the method from Fast-WAM and latent predictive policies, and what evidence shows that the DINOv3 prediction target captures the claimed physical outcomes?
  2. Does the improvement persist against an action-only variant trained with the same data and training budget? How does latent supervision compare with video supervision under a controlled setup?
  3. Can you provide results on a standard simulation benchmark, such as LIBERO, CALVIN, or RoboTwin, with comparable data access and training budgets, to assess performance across a broader range of tasks and scenes?
Reviewer_B1nZOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper explores whether predicting future visual features can improve robot action learning without generating full future videos. DELE-w0.5 jointly learns to predict action chunks and frozen DINOv3 features of the observations at the end of each chunk. A structured attention mask prevents actions from attending to future-state tokens, allowing the future-prediction branch to be removed at inference. On four real-robot manipulation tasks, the method achieves higher success rates than seven baseline policies and reports a median core-model latency of 87.5 ms per action chunk.

Critical strengths
  1. Using future prediction as a training objective without requiring it at inference is a practical design for efficient robot policies.

  2. The reported success-rate gains are substantial across four real-robot tasks involving multi-stage and coordinated manipulation.

  3. The stage-wise evaluation provides useful detail on where policies fail, beyond overall task success.

Critical weaknesses
  1. There is no matched action-only ablation, so the gains cannot be attributed to future-state supervision rather than architecture or pretraining. The claim that frozen DINOv3 targets capture action-relevant physical outcomes also lacks supporting analysis or comparisons with alternative targets.

  2. Evaluation is limited to four fixed-scene tasks on one robot, with no simulation evaluation or public benchmark results, making the reported gains difficult for third parties to reproduce and independently verify. Robustness beyond these settings is also unclear, as scene changes are explored only through qualitative examples.

  3. The 20,000-hour pretraining dataset, task-specific training budgets, and baseline adaptation procedures are insufficiently described. Matching downstream sample exposure alone does not resolve these differences. Missing sampling and action-chunk configurations also make the latency comparison difficult to reproduce and interpret.

Questions for authors
  1. How does an action-only variant perform with the same architecture, pretraining data, and training budget? This is needed to isolate the benefit of future-state supervision.

  2. Why use frozen DINOv3 features as the prediction target rather than alternatives such as SigLIP, V-JEPA, or action-supervised condition representations, for example, as in [1]? A controlled comparison would help establish which properties of the target representation matter for action learning.

  3. What does the pretraining set contain, and does it overlap with the evaluation tasks or scenes? Please provide per-task data sizes, baseline checkpoints, adaptation details, and training budgets, particularly for baselines with near-zero success.

  4. Please specify the auxiliary loss weight, action-chunk length, future prediction horizon, and sampling steps. Were latency results obtained under comparable chunk lengths and execution/replanning settings across methods?

[1] Su Y, Chen S, Shi H, et al. World guidance: World modeling in condition space for action generation[J]. arXiv preprint arXiv:2602.22010, 2026.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.