- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper studies why diffusion policies trained on sparse demonstrations may fail to preserve the demonstrator’s multimodal action distribution. The authors adapt vine sampling to behavior cloning by querying the demonstrator multiple times at selected observations, while retaining the standard diffusion training objective. Experiments on a synthetic task and a simulated 2D truck unloading task show improved distribution matching and downstream performance in best-of-N search and expert iteration at matched training dataset sizes. The main contribution is an analysis and empirical evaluation of data collection, rather than a new policy architecture or training objective.
Critical strengths
-
The paper clearly distinguishes a policy’s capacity to represent multimodal distributions from whether its training data support learning those distributions. The controlled synthetic experiments provide useful evidence about the tradeoff between collecting more observations and collecting more actions per observation at a fixed dataset size.
-
The evaluation goes beyond single-rollout success rates. By examining distribution matching, action diversity, and downstream search and expert iteration, the paper connects its data collection strategy to practical uses of policy diversity.
Critical weaknesses
-
The experimental comparison does not fully isolate the effect of collecting multiple actions per observation or establish an advantage at matched collection cost. Appendix B.1 states that the vine dataset contains planned trajectories, whereas the sliding-window dataset contains executed trajectories that include controller tracking errors. The reference actions used for distributional evaluation are also plans. Consequently, differences in training targets and their alignment with the evaluation reference may contribute to the observed gains. In addition, vine collection requires approximately 30 times the computation of sliding-window collection. Matching the number of training samples therefore does not establish which approach is preferable under a fixed collection budget.
-
The evidence is limited relative to the incremental methodological contribution. Collecting multiple teacher actions per observation has precedents, and vine sampling itself is an existing procedure. The paper’s contribution therefore depends substantially on the depth and generality of its analysis. However, the robotics evaluation covers only one 2D simulated task, without real-world validation or a systematic study of query multiplicity and re-query probability in that task. Each synthetic configuration is trained once, and the reported robotics intervals do not establish robustness across independent training runs. The results support a useful observation in the studied settings, but do not yet establish a broadly applicable data collection recommendation.
Questions for authors
-
Clarification of the comparison: Would the advantage remain if both collection methods used planned trajectories, or both used executed trajectories? Can the current evidence separate the effect of multiple actions per observation from differences in trajectory quality and training targets?
-
Collection efficiency: Under the same demonstrator query budget or collection compute budget, how does vine sampling compare with collecting demonstrations from more distinct observations?
-
Robustness: Are results available for different query multiplicities, re-query probabilities, and independent training runs in the truck unloading task? These results would help establish whether the improvement is stable and how the collection budget should be allocated.
-
Interpretation of downstream performance: Appendix B.4 reports expert iteration performance during search-based data collection and excludes search time from throughput. Are standalone policy evaluations or throughput measurements that include search time available? These would clarify whether the gains reflect improved policies and practical execution efficiency.
We thank the reviewers for their time, effort and feedback. We have decided to withdraw the paper.