- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces Diverse Action Experience (DAE), a framework that collects diverse action branches from intermediate demonstration states. It studies how interaction-data composition affects world-model planning. The effects of state coverage, local action diversity, and trajectory duration vary across models. The finding, better candidate-action ranking does not always lead to better closed-loop task performance, is rather interesting.
Critical strengths
-
Important research question. What data helps world models plan better is a question that many researchers care about. The paper studies an important problem in world-model learning.
-
Good experimental design. The experiments compare different data compositions and test both candidate-action ranking and closed-loop task performance. These experiment provides insightful analysis.
-
Detailed analysis. The paper analyzes how different models respond to changes in training data. It also shows why a high ranking score does not always mean better planning.
Critical weaknesses
-
Limited practical guidance. The paper analyzes the experimental results in detail, but does not turn these observations into clear advice for training world models. The main conclusion is that data composition matters and that different models benefit from different data. I find this conclusion somewhat unsurprising. It is still unclear how these findings can help researchers decide what data to collect or how to build their training datasets.
-
Limited evaluation scope. All experiments are conducted on LIBERO. The tasks may be too simple to test whether the findings hold in more complex settings. Models may perform well by fitting patterns specific to this benchmark, without learning more general planning capabilities. Testing on harder and more diverse tasks, such as those in BEHAVIOR-1K and RoboCasa, would make the conclusions more convincing.
-
No real-robot validation. I consider this a critical weakness for a robotics paper because of the large gap between simulation and the real world. Real robots face perception errors, different contact dynamics, and imperfect action execution. These factors may change the conclusion on which training data is most impactful. Without real-robot experiments, it is hard to tell whether the findings can transfer to real-world manipulation.
Questions for authors
-
Can the authors conclude their empirical observations into actionable guidance for constructing world-model training datasets? Without such guidance, it is difficult to clearly position the paper’s contribution beyond documenting model-dependent sensitivity to data composition.
-
Can the authors extend the evaluation to more challenging benchmarks, such as BEHAVIOR-1K or some other LIBERO variants with harder and more diverse tasks, to assess whether the findings generalize beyond the current setting?
-
Can the authors provide real-robot evaluation to determine whether the findings remain valid under real-world perception noise, contact dynamics, and execution uncertainty?