- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes AEL, a two-stage framework for learning latent representations that capture both robot actions and their visual effects. Stage I learns an action representation through action reconstruction and future visual-feature prediction. Stage II trains a VLA policy to predict this representation from current observations, language instructions, and robot state, then decode it into continuous actions. The method is evaluated on LIBERO, CALVIN, and real-world manipulation tasks. The paper also explores using suboptimal interactions for representation learning without treating them as imitation targets.
Critical strengths
-
The method has a clear motivation. Using the observed effects of actions to shape their representations is a sensible approach, and the future-prediction branch is not required at deployment.
-
The evaluation provides encouraging evidence across different settings. Experiments cover simulation and real-world single-arm and bimanual tasks, while controlled ablations support the benefit of visual-effect supervision. Using suboptimal interactions for representation learning without directly imitating them is also practically relevant.
Critical weaknesses
-
The reuse scope of the Stage-I models could be clearer. Further details about the training data for each benchmark and whether encoders and decoders are shared across tasks would help assess the transferability of the representation and the practical value of the two-stage design.
-
The comparison with direct auxiliary supervision could be strengthened. The ablations support the usefulness of visual-effect supervision, but do not compare against adding a future-feature prediction loss directly to the same VLA policy. This comparison would help clarify the independent benefit of two-stage representation learning.
-
Generalization evidence could be broadened. CALVIN ABC→D provides useful cross-environment evidence. Additional controlled perturbation evaluations, such as LIBERO-Plus, would help establish whether the action-effect representation remains effective under changes in visual conditions and robot state.
Questions for authors
-
Which datasets are used to train each Stage-I model? Are encoders and decoders shared across tasks or benchmarks? Can an existing Stage-I model be reused while training only Stage II for new tasks?
-
Have you compared AEL against direct action prediction with an auxiliary future-feature prediction loss? Does the two-stage approach retain an advantage under matched data and total training budgets?
-
Could you provide a LIBERO-Plus comparison of Direct Action, Recon. Latent, and AEL to further assess the OOD benefits of effect supervision?