- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper proposes Vid2WAM, a framework for distilling future prediction priors from a large video diffusion model into a World Action Model. Given an initial observation and language instruction, an adapted video teacher generates task-conditioned future rollouts, which are used to directly supervise the student’s future-prediction branch. In parallel, an inverse dynamics model predicts pseudo-actions from the generated videos to provide additional action supervision. To combine real demonstrations and noisy pseudo-data, the authors introduce source-aware residual action adapters around a shared action backbone. The proposed method is efficient, as only a student WAM is used for action generation.
Critical strengths
- The paper is easy to follow and well-motivated. Moreover, the approach is inference-efficient while learning WAM better.
- Extensive experiments and ablations demonstrate the effectiveness of the proposed method.
Critical weaknesses
- The performance gain from the proposed method is somewhat marginal, e.g., uncertain whether 3% gains on RoboTwin 2.0 is statistically meaningful or not. In particular, the contribution of the source-aware residual adapters seems modest, making it unclear how essential the proposed adapter design is relative to simpler ways of separating real and pseudo supervision.
- The conceptual novelty is somewhat limited. Prior work such as DreamGen already use video-model generated rollouts together with inverse dynamics to construct synthetic robot supervision, while prior WAMs, e.g., FastWAM, leverage future visual prediction as an auxiliary signal for policy training.
Questions for authors
Please answer the weaknesses part.
Discussion with the authors
Dear reviewers,
the authors have provided responses for your reviews. Reviewer zNkM, thank you for letting the authors know that your concerns have been addressed. Can the others respond as well, please?
Thank you, Your AC