acceptodds
Withdrawn from ICLR 2027

DIDO: Distilling Interaction-Centric Dynamics into One-Step Denoising for World Action Models

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

World Action Models (WAMs) use video generation models to predict future visual dynamics for robotic manipulation, but iterative denoising introduces additional latency for closed-loop control. We empirically find that visual content converges at different rates during denoising. Static background structure forms early, whereas the gripper and manipulated object remain blurry after the first step, with their interaction dynamics emerging only through subsequent denoising. Consequently, naively truncating a multi-step video model to one step preserves scene structure but loses the interaction-centric dynamics most critical for manipulation. To address this issue, we propose DIDO, which distills the converged dynamics of a multi-step video model into a single denoising step. DIDO combines distribution matching distillation with interaction-centric representation guidance. Beyond compressing multi-step generation into one forward pass, DIDO explicitly models the gripper, manipulated object, and their interaction using supervised bounding-box visual reasoning tokens. Additionally, DIDO aligns the target object's representations across multiple model layers with features from a pretrained DINOv3 encoder. This interaction-centric guidance helps the distilled model preserve both the relevant entities and their future dynamics in a single step, while substantially reducing inference latency. DIDO achieves an average success rate of 99.0% on LIBERO, 76.6% on LIBERO-Plus, and 92.0% on RoboTwin, while also demonstrating effective transfer to long-horizon and generalization tasks in real-world robotic manipulation.

Primary area:
applications to robotics, autonomy, planning
Keywords:
robotics, world action model

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_yNJbOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper targets the inference latency of World Action Models (WAMs). The authors observe that in multi-step video denoising, background structure forms early while gripper–object dynamics emerge only in later steps, so naive one-step truncation loses interaction-relevant information. DIDO distills a robot-adapted four-step Wan2.2 teacher into a one-step generator via DMD, and adds interaction-centric tokens. These tokens are supervised by future gripper and object bounding-box trajectories and by multi-layer alignment to DINOv3 object features. A mixture-of-transformers action expert then predicts actions from the one-step world model’s representations. DIDO reports 99.0% on LIBERO, 76.6% on LIBERO-Plus and 92.0% on RoboTwin, plus real-robot results. Latency is 384 ms on an H100, versus 562 ms for the four-step teacher.

Critical strengths
  • The motivation is clear and reasonably supported. The finding that gripper–object dynamics emerge later than the background during denoising (Fig. 1, Fig. 5) gives a concrete reason why naive one-step truncation is insufficient, and the method design follows directly from it.

  • The evaluation goes beyond the near-saturated LIBERO and includes LIBERO-Plus and real-robot tests with visual clutter. There, the gains over Fast-WAM and π0.5 are much larger (e.g., 76.6 vs. 51.5 on LIBERO-Plus, and 66.0 vs. 33.0 / 53.3 in the real world). This is the main evidence that the interaction-centric design is useful in practice.

Critical weaknesses
  • The baselines do not isolate the paper's central claim. DIDO generates a future representation and then predicts actions from it, i.e., the IDM-style paradigm. Yet Table 1 leaves out Fast-WAM-IDM (98.0 on LIBERO / 91.3 on RoboTwin) and Fast-WAM-Joint (98.5 / 90.6), while reporting the "No co-video" row from the same Fast-WAM table. Against these variants, the margin is only 0.5–1.0 points, with no variance reported. Results on LIBERO-Plus are also needed. The same applies to efficiency. The latency gain is measured only against the authors' own four-step teacher, while DIDO is in fact slower than Fast-WAM, which skips video prediction at inference.

  • Novelty is limited. One-step distillation for WAMs already exists. Auxiliary supervision through object/region dynamics and DINO-feature prediction appears in prior work such as DreamVLA. Table 3 shows that most of the gain comes from this auxiliary supervision (+3.6 on LIBERO-Plus) rather than from distillation (+0.9). The contribution therefore reads mainly as a combination of existing components.

Questions for authors
  • Which method do the authors regard as the main baseline, Fast-WAM or one of its variants? If it is Fast-WAM-IDM or Fast-WAM-Joint, the comparison should mainly be on success rate. Please report both on LIBERO, LIBERO-Plus and RoboTwin under the same setting, along with their latency on the same hardware. If it is Fast-WAM itself, DIDO is slower, so the efficiency contribution does not seem to hold, and the gain would need to be justified by success rate alone. Clarifying this would substantially affect my assessment.

  • Once the interaction-centric supervision is added, is test-time future prediction still needed? Have the authors considered a Fast-WAM variant trained with the same interaction tokens and auxiliary losses, but without video prediction at inference?

Reviewer_9dZuOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper targets the inference latency of world action models (WAMs) caused by multi-step denoising. It observes that static scene structure forms early in denoising, while gripper–object dynamics emerge only in later steps. DIDO builds on Wan2.2-TI2V-5B and adds three components:

  • One-step distillation: DMD distills a robot-adapted 4-step teacher into a one-step generator.
  • Interaction tokens: 29 tokens are inserted into the video transformer. They are supervised by future bounding-box trajectories and by DINOv3 features of the target object.
  • Action expert: a mixture-of-transformers action expert reads the video model's intermediate features, with dynamics-based pruning of future-frame tokens.

DIDO reports 99.0% on LIBERO, 92.0% on RoboTwin, and 76.6% on LIBERO-Plus, and is also evaluated on four real-robot tasks on a Galbot G1. Its latency is 384 ms per action chunk, versus 562 ms for the 4-step teacher.

Critical strengths
  • Future prediction at modest cost. DIDO adds only 28 ms over Fast-WAM yet improves LIBERO-Plus by 25.1 points, and it is 32% faster than the 4-step teacher with no accuracy loss. This is a useful engineering result.
  • Grounded supervision improves robustness. Supervising dedicated tokens with future bounding boxes and object-level DINOv3 features raises LIBERO-Plus from 72.3 to 75.9, the largest gain in the ablation. These tokens make up 5.74% of what the action expert sees but receive 33.9% of the attribution.
Critical weaknesses

1. Limited novelty, and the central claim is weakly supported.

  • The one-step distillation adds little.
    • The 4-step model is a public Wan2.2 checkpoint, which the authors distill to one step using standard DMD2.
    • Latency drops only from 562 ms to 384 ms (−32%), and about 30 ms of that comes from token refinement rather than distillation. Most of the cost does not depend on the step count at all: Fast-WAM, which never generates the future, still takes about 356 ms.
    • At matched compute, distillation beats one-step truncation by only 0.6 points on LIBERO and 0.9 on LIBERO-Plus. Both gains fall within single-run evaluation noise, which a binomial estimate puts at roughly ±0.9 and ±1.25.
    • Fig. 5 shows that full 4-step denoising beats truncation by just 0.7 and 1.2 points. The "missing interaction dynamics" the paper emphasizes are therefore worth about one point.
  • The other components are mostly existing techniques or heuristics.
    • The multi-layer alignment closely resembles REPA, which the paper does not cite.
    • The MoT action expert and the co-video loss borrowed from Fast-WAM are both standard.
    • Token refinement is a hand-set rule that keeps the top 3 of 20 tokens. It adds +0.1 / +0.7 and saves about 30 ms.
  • Only the interaction tokens give a gain clearly beyond noise (+3.6 on LIBERO-Plus). However, they are independent of one-step generation, and the paper has no control that uses the tokens without distillation.

2. Missing comparisons and weak statistical evidence.

  • The closest work is not compared. The authors themselves name S-VAM as one of the most related methods, since it also performs one-step self-distillation with structured foresight. Yet neither S-VAM nor object-centric WAMs such as OA-WAM and MaskWAM appear in the experiments.
  • The margins in the main tables are small. DIDO leads by only 0.5 points on LIBERO and actually trails LingBot-VA by 0.2 on RoboTwin.
Questions for authors

Q1. In Fig. 4, DIDO scores 91 on the Corn task under the basic protocol. Under the scoring rule in Appendix C.3 (10 trials, each scored 0/3/6/10), a total of 91 does not seem achievable. Is this a typo, or is something else misstated, such as the number of trials or the scoring rule? Could the authors clarify?

Q2. In Stage II, DMD is replaced by a video loss with τ=0. This amounts to directly regressing the mean of future frames, and Stage II is trained much longer than Stage I. Could the authors report the prediction quality of the one-step generator at the end of Stage I and at the end of Stage II, to confirm that the final model still retains the effect of distillation?

Reviewer_sKLaOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper observes that when a video diffusion model is truncated to a single denoising step, static scene structure is already formed, but the gripper–object interaction dynamics are still missing. To address this, DIDO distills a four-step Wan2.2-based teacher into a one-step generator with DMD, and adds interaction-centric tokens supervised by gripper and object bounding-box trajectories and by multi-layer DINOv3 feature alignment. The one-step world model is then coupled with an action expert through a mixture-of-transformers design, and the method reports 99.0% on LIBERO, 76.6% on LIBERO-Plus, and 92.0% on RoboTwin, together with real-robot experiments on Galbot G1.

Critical strengths
  1. The analysis of the denoising trajectory gives an intuitive view of what one-step truncation loses in a WAM, and the paper tries to address this problem directly.

  2. The experimental coverage is broad, including two standard benchmarks, a robustness benchmark, and a real robot, where the baselines are retrained with the same data and training budget.

  3. The paper provides useful supporting analyses, such as the audit of the automatic bounding-box annotations and the token-level attribution study.

Critical weaknesses

The performance gains are marginal, and the technical novelty is limited.

  1. The gains over prior work are marginal. LIBERO is close to saturation (99.0 vs. 98.5 for LingBot-VA and Faster-WAM), DIDO is on par with or slightly below existing WAMs on RoboTwin (92.0 vs. 92.2 for LingBot-VA and 91.8 for Fast-WAM), and the improvement on LIBERO-Plus is only 1.6 points over Faster-WAM. Under camera perturbation, DIDO reaches only 45.3%, far below X-VLA (96.0%).

  2. The central story is not well supported by the ablations. One-step DMD distillation brings only small gains over naive truncation (97.7 → 98.3 on LIBERO and 71.4 → 72.3 on LIBERO-Plus), and Figure 5 shows that using later denoising steps helps only slightly (97.7 → 98.4). Most of the improvement instead comes from the bounding-box and DINOv3 auxiliary supervision, which has no direct connection to one-step distillation and may help multi-step WAMs or Fast-WAM just as well. In addition, Appendix C.2 states that interaction-centric supervision is already enabled during teacher preparation, so it is unclear what the truncation baseline and the teacher actually contain.

  3. The technical novelty is limited. One-step distillation for WAMs (e.g., Flash-WAM and S-VAM), coarse-to-fine behavior in diffusion, bounding-box trajectory supervision with the LaRA-VLA annotation pipeline, REPA-style feature alignment, and dynamic token pruning are all existing ideas. The paper reads more like a combination of known components than a new insight.

  4. The efficiency claim is overstated. Going from four steps to one step reduces end-to-end latency only from 562 ms to 384 ms, which is still slower than Fast-WAM, and no latency breakdown is provided (the action expert still uses 10 Euler steps). The real-robot evaluation is also small, with only 10 rollouts per task and protocol, scored with a staged rubric.

Questions for authors

See Weakness

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.