- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes Do-JEPA, a training framework that trains latent world models to identify the causal impacts of specific actions. From an initial simulated state, Do-JEPA generates paired trajectories respectively with the target and the reference actions. The models are trained to predict trajectory differences (i.e., impacts of the target action) as well as vanilla next-state predictions. Do-JEPA also proposes a series of sub-losses to provide fine-grained supervisions. Experimental results show that Do-JEPA manages to improve latent world models' capability to identify impacts of actions in different task scenarios.
Critical strengths
- Clear method design: the method proposes to learn from paired trajectories with multiple losses to provide fine-grained supervisions.
- Extensive experiments with strong results: the experiments include multiple environments, task settings, ablations, and random seeds, and demonstrate the performance of the method.
Critical weaknesses
- Difficulty in data construction: the method requires contrastive control on environment states, which may be difficult to achieve in real-world scenarios.
- Unclear practical significance: there is no concrete evidence that such learning can improve models' overall task completion performance. Specifically in planning, the models suffer from performance degradation.
Questions for authors
- How can the method be applied to real-world scenarios where paired trajectories are difficult to construct?
- Can the authors provide additional results or demos about joint validation of all modules?
- Can the authors provide additional results to validate that the proposed training framework can improve models' overall task completion performance?
Dear authors and reviewers,
The author-reviewer discussion period closes on November 18 (AoE). Please engage in response periods so that we have our issues discussed thoroughly.
AC