- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
This paper proposes DreamFree, a World Action Model for robotic manipulation. It uses a pretrained video diffusion model as a teacher and applies future representation distillation to train a visual encoder that extracts predictive information from current observations, avoiding iterative future video generation at inference time. An action module is subsequently trained using the visual features and robot states. The evaluation covers LIBERO, RoboCasa-Kitchen, and real-world DROID tasks, along with additional studies on extra video data, cross-embodiment transfer, video-task transfer, inference latency, and visual representation quality. The results demonstrate competitive performance on several robotic manipulation tasks and substantially reduced inference latency. However, the distinction between the proposed representation distillation approach and existing predictive representation learning and feature distillation methods remains insufficiently clear.
Critical strengths
The main strength of this paper is its broad experimental coverage. Beyond reporting task success rates, the authors investigate the role of distillation through loss ablations, representation probing, and video-task transfer experiments. The paper reports substantial performance improvements on RoboCasa-Kitchen and competitive results on real-world robotic tasks. In addition, the inference latency evaluation shows that avoiding iterative future video generation can improve control efficiency. These results provide meaningful empirical evidence for the practical value of predictive visual representations in robotic manipulation.
Critical weaknesses
-
Limited methodological novelty. Equations (2) and (4) largely adopt standard flow-matching objectives, while Eq. (1) primarily transfers predictive information from a pretrained video model into visual representations of current observations through feature distillation. Although this design is reasonable, it does not introduce a clearly novel distillation mechanism or learning objective, limiting the originality of the core method. Furthermore, the paper does not adequately identify fundamental limitations of existing predictive representation learning methods or REPA, nor does it demonstrate through direct comparisons that the proposed distillation approach offers distinct advantages over these methods. Thus, the experiments support the effectiveness of the approach, but the evidence for its originality and relative advantages remains insufficient.
-
Insufficient differentiation from prior work and an underdeveloped motivation. The paper does not clearly identify fundamental limitations of existing predictive representation learning methods or REPA, nor does it demonstrate that DreamFree addresses problems that these methods cannot adequately handle. The current experiments mainly establish that distillation is effective within DreamFree, but do not sufficiently demonstrate its distinct advantages over existing predictive representation learning and feature distillation methods.
-
The performance gains require cautious interpretation. Although the paper covers multiple simulation benchmarks, real-world robotic tasks, and transfer settings, the magnitude of improvement varies across tasks, and the gains on some benchmarks are small. In addition, the number of trials per task in the real-world experiments is limited. The current results support the practical potential of the approach, but do not establish consistent advantages across a broad range of task settings.
Questions for authors
-
Distinction from prior work: Could the authors clarify the technical differences between DreamFree and existing predictive representation learning methods, including REPA? Specifically, what problems does future representation distillation address that existing methods do not adequately resolve?
-
Independent contribution of the proposed method: Beyond the ablation that removes the distillation loss, could the authors provide direct comparisons with alternative predictive representation learning or feature distillation methods to establish whether the proposed distillation design offers distinct advantages?
-
Scope of the performance gains: Given that the magnitude of improvement varies across tasks, could the authors further analyze the task settings in which DreamFree is most effective and clarify whether its performance advantages are consistent across repeated trials?
5 comments
Response to Reviewer ezkb [2/2]
W3: Direct comparisons with alternative predictive representation learning or feature distillation methods
Thank you for suggesting these comparisons. Among the methods discussed, FLARE provides the closest comparison for robot control, as it also uses future representation alignment to improve action prediction. Our RoboCasa-Kitchen evaluation includes this comparison:
- DreamFree achieves 71.7% success, compared with 66.4% success for FLARE.
REPA and V-JEPA 2 address different learning formulations and cannot be directly substituted into our representation-learning stage:
- REPA aligns a diffusion model’s hidden features with pretrained visual representations of the corresponding clean image. Applying it to our setting would require specifying a current-observation-to-future prediction task, including the teacher representations and temporal alignment. This would evaluate an adaptation of REPA to predictive distillation rather than its original formulation.
- V-JEPA 2 jointly learns an encoder and predictor using an evolving target encoder. Applying its learning procedure would require redesigning our representation-learning stage. Using a frozen V-JEPA 2 encoder as our teacher would evaluate the choice of teacher, rather than V-JEPA 2’s predictive representation-learning method itself.
A controlled comparison with adaptations of REPA or V-JEPA 2 would therefore require additional design choices beyond their published formulations.
We use FLARE as the closest existing method-level comparison for predictive representation learning in robot control, although differences in backbone and training setup prevent attributing the gap solely to the alignment design.
W4: Performance gains in simulation (LIBERO, RoboCasa-Kitchen)
We agree that the magnitude of improvement varies across benchmarks and tasks. On LIBERO, performance is nearly saturated, and we interpret DreamFree’s results as competitive performance on a standard benchmark and place greater emphasis on the more challenging RoboCasa-Kitchen evaluation.
On RoboCasa-Kitchen, we evaluate each model over 50 trials per task across 24 tasks. The overall average includes tasks where methods already achieve nearly saturated performance, such as CloseDrawer. Thus, the average gain across all tasks is less pronounced than the gain on challenging pick-and-place (PnP) tasks.
The table below shows success rates (%) for representative tasks.
| Task / group | Fast-WAM 5B | Fast-WAM 14B | DreamFree 5B | DreamFree 14B | DreamFree 14B + extra videos |
|---|---|---|---|---|---|
| PnP: Counter → Stove | 24 | 16 | 34 | 66 | 74 |
| PnP: Counter → Sink | 26 | 20 | 54 | 72 | 72 |
| PnP: Sink → Counter | 42 | 18 | 54 | 68 | 76 |
| CloseDrawer | 98 | 100 | 100 | 100 | 98 |
| TurnOffMicrowave | 98 | 100 | 94 | 100 | 90 |
| All 8 PnP tasks | 19.8 | 11.8 | 37.3 | 58.5 | 65.0 |
| All tasks | 44.4 | 44.7 | 58.5 | 71.7 | 74.4 |
DreamFree 14B achieves substantial improvements over Fast-WAM 14B on all eight PnP tasks, with gains ranging from 20 to 68 percentage points. Across PnP tasks, DreamFree achieves 58.5% success compared to 11.8% for Fast-WAM. Additional videos further increase the average to 65.0%. These results identify pick-and-place as a setting with particularly strong gains over Fast-WAM.
W5: Real-world results and evaluation scope
We appreciate the reviewer’s concern regarding the limited number of real-world trials. Given the practical constraints of real-world evaluation, we prioritized task diversity, evaluating 10 tasks with five trials per task. We acknowledge that additional trials would help further assess the consistency of the observed performance gains.
To improve comparability across policies, we used image overlays to align initial scene configurations and reduce setup variation. All real-world rollouts, including successes and failures, are provided in the supplementary material.
W6: Task-dependent gains in real-world evaluation
DreamFree achieves higher success than on nine of ten tasks, while gains over DreamZero vary across tasks. DreamFree achieves higher success on four tasks and the clearest improvements occur in towel placement (Cover strawberry with towel) and successive object rearrangement (Put bolts in bowl).
We hypothesize that directly supervising predictive features helps the encoder retain information about spatial relations and scene changes that is useful for these tasks. For example, covering the strawberry requires anticipating the towel’s placement and resulting coverage, while placing multiple bolts requires adapting actions as the arrangement changes after each placement. We interpret these results as evidence of practical effectiveness on the evaluated tasks, with particularly promising gains in these specific settings.
Please let us know if there are any remaining points we should clarify. We are happy to provide further details.
Follow-up on Our Response and Revisions for Reviewer ezkb
Thank you again for your time and valuable feedback on our paper.
In our response, we:
- Clarified DreamFree’s contribution and technical distinctions from REPA, V-JEPA 2, and FLARE.
- Discussed the comparison with FLARE and its limitations.
- Provided task-level performance analyses, highlighting gains on challenging pick-and-place tasks while acknowledging the limited number of real-world trials.
As the author–reviewer discussion ends on November 18, we would appreciate hearing whether any concerns remain. We would be happy to provide further clarification before the deadline.
If our responses and revisions have satisfactorily addressed your concerns, we would be grateful if you would consider revising your score upward.
Yours sincerely, Authors
Response to Reviewer ezkb [1/2]
We sincerely thank you for your time and effort in reviewing our paper. We appreciate your positive comments on:
Please find our responses to your comments below.
Our main contribution is future representation distillation for imagination-free robot control, which transfers a pretrained video model’s predictive knowledge into representations computed from the current observation and instruction.
The problem we address is how to make this predictive knowledge directly available in the visual features used for action prediction. The video generation objective does not explicitly align these features with future representations. DreamFree introduces this supervision by training current observation features to anticipate future frame representations from a pretrained video diffusion teacher.
Eq.(1) addresses this through two design choices:
Eq.(1) therefore trains the encoder to anticipate features of future content unavailable to the encoder. Our contribution lies in this predictive distillation formulation and its integration into imagination-free control.
Note that Eqs. (2) and (4) use standard flow matching objectives for video and action modeling. They serve as supporting components of our method, and we do not claim methodological novelty in these objectives.
Thank you for requesting this clarification. DreamFree shares the goal of learning useful visual representations with these methods. However, it addresses the specific problem of transferring a pretrained video diffusion model's predictive knowledge into an encoder for imagination-free control.
Compared with REPA, the information available to the student and the direction of knowledge transfer differ. REPA aligns a diffusion model’s features with clean-image representations from a pretrained visual encoder to improve generation. DreamFree instead uses the video diffusion model as the teacher. Its student must predict representations of future content from the current observation, under the information restriction described in W1. Applying same-content alignment alone does not impose this requirement.
Compared with V-JEPA 2, both methods learn to predict unobserved visual content, but their representation targets are constructed differently. V-JEPA 2 jointly learns an encoder and predictor using masked representations supplied by an exponential-moving-average target encoder. DreamFree instead uses intermediate representations from a frozen pretrained video diffusion teacher. Our objective is to transfer the teacher’s existing video-prediction knowledge into current observation features for action learning.
Compared with FLARE, the differences concern the source and location of supervision. FLARE aligns features inside the action denoising network with future-observation embeddings from a vision-language encoder. On the other hand, DreamFree distills future-informed features from a video diffusion teacher into a separate visual encoder before action learning. This separates predictive visual representation learning from action-policy optimization and places the transferred video knowledge directly in the encoder used for control.
We added this discussion in Appendix E.1 to clarify these technical distinctions and the specific problem DreamFree addresses.
(Response continued in the next comment.)