- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper presents Cross-Realization Consistency (CRC), a method aimed at reducing run-to-run functional variation caused by training randomness during the further training of robot manipulation policies. Starting from the same original policy, the method trains two LoRA branches and introduces a cross-branch output consistency loss alongside the behavior cloning objective to encourage more consistent functional changes across training runs. The authors evaluate CRC on eight MimicGen manipulation tasks using Cross-run Functional Difference (CFD), Cross-run Directional Alignment (CDA), and closed-loop trajectory variation metrics. The results show that CRC reduces functional differences across training runs while largely preserving task success rates.
Critical strengths
-
The paper addresses run-to-run variation during the further training of robot manipulation policies, rather than focusing solely on task success rates. Improving consistency across training runs is a meaningful research direction, particularly for robotic systems that require predictable and repeatable behavior.
-
The experimental evaluation and ablation studies are thorough. The authors compare CRC with methods such as Standard BC, Dropout Consistency, and Mutual Learning across multiple robot manipulation tasks. They also investigate the effects of consistency constraints through controls involving Shared Batch, Batch Size, Shrinkage, and L2 Regularization. In addition, closed-loop trajectory evaluations and experiments with additional training runs provide further validation.
Critical weaknesses
-
The methodological novelty is relatively limited. CRC primarily relies on a consistency loss between the outputs of two branches, which is closely related to existing approaches such as Mutual Learning and Codistillation. Although independent minibatches lead to further improvements, the gains over Mutual Learning and Shared Batch are relatively modest, while the method incurs additional training costs.
-
The practical value of run-to-run consistency has not been fully demonstrated. Although the authors show that CRC reduces differences in action predictions and execution trajectories across training runs, more consistent behavior does not necessarily imply better robot policies. The paper does not provide direct evidence that such consistency improves deployment reliability, robustness, or generalization. Moreover, stronger proximity regularization settings can also achieve competitive CFD and CDA results, suggesting a potential trade-off between consistency and the magnitude of functional changes. However, the practical significance of this trade-off remains unclear.
Questions for authors
-
CRC and Mutual Learning employ closely related consistency objectives. Given that Shared Batch already accounts for a substantial portion of the observed improvements, could the authors further clarify the unique benefits of using independent minibatches and whether the additional training cost is justified?
-
For policies that successfully complete the same task but follow different action trajectories, why is it necessary to reduce these differences? Could the authors further explain how improvements in run-to-run consistency translate into practical benefits for robot deployment?