- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper presents CoRe-WAM which focusing on reducing the latency of World-Action Models (WAMs). CoRe-WAM has 2 main components:
-
CA-DMD performs action distribution matching under teh same student-generated future video used by the action sampler instead of using ground-truth future videos during distillation.
-
RAR: At each subsequent action chunk, noise is added to the most recent (previous) action latent. Such a perturbed latent will then be refined using the current context and video prediction.
Extensive results were presented, with the proposed method showing a good balance between manipulation performance and inference latency by achieving comparable performance with much lower latency.
Critical strengths
-
The tradeoff is a sensible approach. Given that latency is the bottleneck, the proposed method presented impressive results by having 19.35x speedup over the original teacher LingBot-VA.
-
The idea of passing action-distribution feedback back to the video generator is a sensible idea to support the sequential distillation approach in CA-DMD.
-
RAR is training-free, making it a very smart way to refine teh action latent.
-
Extensive evaluations were presented with a wide range of tasks and scenarios.
Critical weaknesses
-
The evaluation of teacher action prediction and score reliability across tasks could have been stronger.
-
The cumulative design in the ablation does not fully isolate component interactions which weakens the evidence to understand the performance gain associated with each individual component.
-
The recurrent refinement seems to be sensitive to the noise level with the large performance difference demonstrated. It is unclear how generalizable the settings are when transferring to other tasks.
Questions for authors
I would like the authors to clarify the 3 weaknesses stated above.
Thanks for your positive assessment and valuable feedback. Our responses to the main concerns are given as follows.
(The evaluation of teacher action prediction and score reliability across tasks could have been stronger.)
Thanks for the suggestion. We extend the teacher action diagnostic from RGB block ranking to eight RoboTwin Clean tasks and add a directional test of the DMD feedback. Table R1 compares teacher XYZ RMSE under ground truth and student-generated futures using one action snippet from a single arm per task. We also compare Joint DMD and CA-only on the same tasks and scene seeds, with 50 closed-loop trials per task. CA-only improves task success even in cases where teacher RMSE is higher under student-generated futures. RMSE measures agreement with demonstration actions, whereas condition alignment adapts action supervision to the video conditions encountered at inference. These results support the benefit of addressing the conditioning mismatch.
Table R1. Teacher action RMSE on selected snippets and closed-loop success on the corresponding tasks.
To assess whether the DMD feedback provides useful control guidance, we conduct a paired directional test across these eight tasks, with 25 trials per task (200 paired evaluations per setting). The DMD update direction is −ga, where ga is the action DM gradient in Eq. (5). Starting from the same student actions, we compare the original actions with corrections along this direction, its opposite direction, and a random direction of equal magnitude. Following the DMD update direction achieves the highest success rate in Table R2, supporting its control utility under the evaluated conditions.
Table R2. Directional test of DMD feedback.
(The cumulative design in the ablation does not fully isolate component interactions, which weakens the evidence to understand the performance gain associated with each individual component.)
Thanks for the comment. We include additional ablations in Table R3 that show that each component improves success individually. We evaluate CA, VJP, and RAR separately alongside their combined variants on a subset of 25 RoboTwin 2.0 tasks, with 50 trials per task in each of the Clean and Randomized splits. Success rates are averaged over the two splits.
Table R3. Component ablation on a subset of 25 RoboTwin 2.0 tasks. “Student future” denotes condition alignment (CA).
In the variant with VJP alone, the action sampler and both score models use ground-truth futures. The VJP is evaluated under this condition and incorporated into the video update as in Eq. (7).
(The recurrent refinement seems to be sensitive to the noise level with the large performance difference demonstrated. It is unclear how generalizable the settings are when transferring to other tasks.)
Thank you for the comment. We include an additional validation in Table R4 that supports the shared setting σr=0.50. We compare the three noise levels on a held-out validation split, which covers 10 RoboTwin 2.0 Clean tasks, with 50 trials per task. This split is independent of the main evaluation set and used only for this additional noise comparison, with the checkpoint and all other hyperparameters fixed.
Table R4. Noise-level validation on the held-out split.
As shown in Table R4, σr=0.50 performs best, consistent with the shared setting already used in our main evaluations. Both 0.50 and 0.75 achieve high success. The lower success at 0.25 may arise from retaining too much of the historical proposal, limiting adaptation to the current observation and predicted future. These results support using the shared setting across tasks while confirming sensitivity to lower refinement noise.
We will include these additional experiments in the revised PDF.