- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes to learn to improve action generation by contrasting action-conditioned consequence representations anchored by future observation semantics. The proposed method is a plug-in module that can be discarded during inference.
Critical strengths
The shortcut problem in multi-conditioned future prediction is interesting and important. The contrastive learning approach is a promising way to force the model to provide trustworthy future representation with respect to the actual action input.
Critical weaknesses
- The paper lacks a standard benchmarking results to directly compare ACE with prior works. The paper only experiments on LIBERO and RoboCasa with controlled budgets (Sec. 5.1). Even under the customized setting, ACE provides modest performance improvement with no error bar (Tab. 2).
- The exact action representation and coordinate frame are not specified, raising concerns about the factual correctness of the method. Considering the action augmentations (line 237),
batch action donorsandtemporal reversalwill lead to state inconsistency for absolute pose actions, andtemporal reversalwill also lead to state inconsistency on chunk-wise delta actions. Consequently, the model may distinguish these negatives through discontinuities only, rather than consequence understanding.
Questions for authors
- When constructing the contrastive learning objective in Eq. 6, this objective merely pushes away augmented actions from the ground truth future, rather than assigning possible future for the actions. For example, for
temporal reversal, is it possible to use the history observation as the "future", and is it possible to use zero actions and replicate present observation as the "future"?