acceptodds
Under review as a conference paper at ICLR 2027

World Model Self-Distillation: Turning Video Generators into Task Solvers

Abstract

Pretrained video generators are promising visual world models for planning, as they can depict how actions change a scene, but solving tasks from high-level goals remains difficult without detailed execution descriptions. Common approaches outsource execution reasoning to language or vision-language models (VLMs), or require costly paired task-execution videos. We introduce World Model Self- Distillation (WMSD), combining self-distillation and reinforcement learning with- out curated task-video supervision. From an unlabeled scene image, a VLM generates a task and a step-by-step execution description. This description guides a frozen pretrained video generator, the Demonstrator; self-distillation transfers its behavior into an Executor conditioned only on the image and a short task prompt. VLM feedback further improves the Executor, exploiting the asymmetry between generating a solution and judging a sampled execution. Inference requires only the Executor. On WorldTasks-Bench, eight-step LTX-2 reaches 60.5% task completion, surpassing the Demonstrator’s 49.5% and Base’s 28.5% under VLM evaluation. Transfer to robotic video generation suggests generalization without robotics-specific fine-tuning. Independent and human evaluations support the task-solving gains. Verified maze solving shows how lightweight Demonstrator adaptation can enable transfer to new domains.

Primary area:
generative models
Keywords:
world models, video generation, task solving

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_hsygOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes the WMSD framework, which transforms a pre-trained video generator into a task solver through self-distillation and reinforcement learning. The approach is somewhat insightful, and the experiments are fairly comprehensive. However, there are shortcomings in terms of evaluation reliability and baseline fairness, and the evidence supporting the core claim is not sufficiently robust.

This method uses a VLM to automatically generate data pairs consisting of tasks and detailed execution descriptions from unlabeled scene images. During the training phase, a frozen demonstrator guides a trainable executor, which takes only images and brief task descriptions as input, through on-policy self-distillation; simultaneously, the system performs joint optimization via reinforcement learning (e.g., FlowGRPO/AWM) by combining VLM-based task completion and consistency rewards. During the inference phase, the system only needs to use the pre-trained Executor, thereby eliminating the latency overhead associated with invoking a VLM for planning.

The main contributions of this paper include:

  1. Propose the WMSD framework, which transforms a pre-trained video generator into a data generator without the need for paired task-execution videos.
  2. Construct the WorldTasks dataset and the WorldTasks-Bench evaluation benchmark to assess the general task-solving capabilities of generated videos.
  3. Empirical results show that Executor achieved a task completion rate of 60.5% on the benchmark, outperforming the baseline model (28.5%) and the Demonstrator with privileged information (49.5%), and demonstrated strong zero-shot transfer capabilities on the DreamGen robot video benchmark and the Maze task based on verifiable rewards.
Critical strengths

Strengths:

  1. The introduction of Anchor Loss is an effective engineering design. This design effectively stabilizes the training process, alleviates the crashes commonly encountered in video generation RL, and serves as a valuable engineering reference.
  2. The experiments are fairly comprehensive, lending the paper a reasonable level of credibility.
Critical weaknesses

Main Concerns:

  1. The core modules used in the paper all have clear predecessors, and the paper itself acknowledges this in the “Related Work” section: “DMDR (Jiang et al., 2025) combines DMD with reward optimization,” and “AWM and FlowGRPO provide the optimization algorithms.” Therefore, this paper is more engineering-oriented and I recommended a more insightful algorithm tailored to the temporal characteristics of video. I suggest that the authors explain more clearly why this combination is necessary and unique in video generation.
  2. The paper claims that the consistency score improved from 0.694 to 0.882. However, the temporal stress test in Appendix C.1.2 shows: Table 17: Under the “Shuffle” condition, Qwen’s rejection rate was only 0.613; under the “Remove middle” condition, it was only 0.548. This implies that even when video frames are completely shuffled or middle frames are removed, the VLM still has approximately a 40%–50% probability of classifying the video as coherent. This suggests that the evaluator may not truly understand the physical logic of the actions. In this scenario, the model’s high scores may stem from catering to the evaluator’s preferences (“reward hacking”) rather than a genuine grasp of physical laws. I recommend supplementing the analysis with more reliable evaluation methods.
  3. The SFT baseline may not fully reflect the comparison. The paper attempts to demonstrate that WMSD outperforms traditional SFT, but the SFT baseline in Appendix A.2 uses “automatically annotated internet videos.” The paper itself acknowledges: “This lack of improvement may reflect the limited task-directed action in many routine internet videos.” This baseline is clearly underpowered. A fairer comparison would be to perform SFT directly on the Executor using videos generated by the Demonstrator and rated highly by the VLM. If WMSD cannot significantly outperform this “high-quality teacher data SFT,” then the necessity of its complex training process is needed or not?
Questions for authors

The abstract claims that WMSD is surpassing the Demonstrator (60.5% vs. 49.5%). However, Appendix B.3 clearly states that the p-value from the paired McNemar test is 0.093, which means this improvement is not statistically significant (p > 0.05). So, is it statistically significant or not?

Reviewer_YNAsOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes WMSD, a self-distillation framework for World Models. It transforms a model that can complete tasks in generated videos only when given detailed descriptions into a "Task Solver" that achieves the same effect using short task descriptions that state only the task goal. The method draws on GRPO and LLM context distillation. The student Executor first generates multiple videos from a short task description. A VLM and PickScore then evaluate the videos based on task completion, visual consistency, and visual quality. Based on these videos, the method compares the velocity predictions of the student Executor conditioned on the short task description with those of the teacher Demonstrator conditioned on the detailed task description. This difference is used as part of the reward and loss for training the Executor. After training, the Executor can use only a short task description to predict task scenes that previously required a long task description to generate. The authors train and test the method on their own WorldTasks dataset and benchmark, using two VLMs and human evaluators for scoring.

Critical strengths

The method is well designed and practical. The authors combine task completion, consistency, and visual quality scores with the velocity difference from the Demonstrator as the reward. The latter can also be used directly as a loss. This allows the teacher, which has access to the detailed task description, to guide how the student Executor should adjust while preserving room for exploration through RL. After training, the model can achieve from a single short task description what previously required asking a VLM to rewrite the prompt, without requiring an additional VLM call during inference. The detailed experiments and ablations also show that each of these design components contributes to the final results.

Critical weaknesses

In the second contribution, the authors claim that their method "improves task completion beyond the Demonstrator," but this may be an overclaim. The main score supporting this claim comes from the Qwen VLM that was originally used for scoring during training, so it seems natural that the score would be higher than that of the Demonstrator. The score from another VLM, Luna, is also higher than the teacher's, but the difference is not very clear. The two VLM reviewers have not been compared with human judgment. The evaluation is also conducted entirely on the authors' own benchmark, whose samples were generated using the same VLM and the same pipeline. This raises concerns about whether the comparison is fair. A fairer evaluation could include comparisons on existing image-to-video instruction-following benchmarks.

Questions for authors
  1. How closely do the judgments of the two VLM reviewers agree with human judgment? Can you report the Qwen and Luna judgments for WMSD and the Demonstrator on the generated videos?
  2. What would the results be if the videos selected or generated by the Demonstrator were used directly for SFT?
  3. After training, does the diversity among multiple videos generated for the same task decrease?
Reviewer_MAsuOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes a world model self-distillation (WMSD) framework, which adapts a pretrained video generator for task execution with only an initial image and a short goal. It uses a pretrained video generator as demonstrator, which receives the initial image and detailed execution instructions as inputs. The executor is initialized from the same backbone but only sees the image and a short task. The executor is finetuned to match the flow vector field of the demonstrator, using both reward for RL and a regression anchor. Empirical results show strong improvement compared to the base model, and can be transferred to new domain. However, flaws in the evaluation method limit the proper evaluation of the performance of the proposed method.

Critical strengths

• The proposed method is clean and simple for deployment.

• Experiment results show strong improvement over the base model.

• The method demonstrates promising transfer to robot video generation without robot-specific fine-tuning.

Critical weaknesses

• The evaluation method is not fully convincing. The evaluation is mainly conducted by the model used for training reward (Qwen3.5-27B), which might introduce model-specific bias. While independent and human evaluations mitigate this concern, the human evaluation only covers 40 unique video pairs.

• Comparison to the frozen Base and Demonstrator shows the training works, but does not isolate the contribution of individual WMSD component. Separate RL-only and anchor-only comparisons will provide evidence necessary to evaluate the contribution of each components.

• Evaluation is limited to task success in generated video but not execution in an interactive environment. It does not necessarily translate into physical feasibility or reliable robot control.

Questions for authors

Did you evaluate the method with only RL-reward or anchor training objective?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.