- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes a modification based on conservative model-based imitation learning (CMIL). The proposed method adds a scorer on top of the existing CMIL architecture, which distinguishes demonstrated latent transitions from unfiltered world-model rollouts. The method then leverages the generated score and the uncertainty of the scorer for reward augmentation to improve downstream policy and critic learning. The empirical experiments compared their proposed method with standard CMIL, ROT and BC baseline across multiple locomotion (DMControl tasks) and manipulation (Meta World, Franka Kitchen and Shadow Hand) tasks. Results demonstrate slight improvement against standard CMIL method.
Critical strengths
-
The overall presentation of the paper is generally understandable. The proposed method is simple and straight-forward (an additional reward augmentation term with the scorer's uncertainty).
-
The paper provides adequate ablation studies and multi-seed results in the appendix, which facilitates reproducibility and demonstrate the effectiveness of the proposed method.
Critical weaknesses
-
Limited novelty: The paper's proposed method is an additional scorer module on top of existing CMIL architecture for reward augmentation. This is more like a small modification improvement. Compared with standard CMIL, the proposed modification only provides a small amount of gain on cumulative rewards in a diverse set of tasks, where the importance of this proposed modification should be questioned.
-
Lack of recent baselines: The paper should compare with more recent baselines on online imitation learning with world models, such as IQ-MPC [1]. The baselines leveraged in the paper is before 2025 and seem to be quite outdated.
-
More evaluation tasks: The proposed method only evaluates on 8 manipulation and locomotion tasks, which is quite insufficient. The proposed method should evaluate its effectiveness on more complex tasks such as dog-run and humanoid-walk tasks in DMControl and some complex manipulation tasks in ManiSkill3 such as Pick YCB or Peg Insertion Side.
[1] Li, S., Huang, Z., & Su, H. (2025). Reward-free world models for online imitation learning. In Proceedings of the 42nd International Conference on Machine Learning (Vol. 267, pp. 34702–34724). Proceedings of Machine Learning Research.
Questions for authors
See weakness section.