- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes TERRA, a framework for learning continuous latent actions from video transitions. Its Temporal Effect Representation combines net visual feature changes with a low-order temporal dynamics component. Effect-Anchored Transport then decodes latent actions in other initial states and regularizes the resulting feature changes to preserve the source transition’s effect direction. Trained on 29 Open X-Embodiment sources, TERRA is evaluated through frozen linear action readers, visual distractor tests, and cross-context transport diagnostics. The authors report improved action readout and transport stability, alongside 93.4% average success on LIBERO compared with 91.8% for UniVLA at matched pretraining scale, using different policy interfaces.
Critical strengths
- The paper tackles a useful question: what latent actions should capture and whether they retain their meaning across different states.
- The same-budget reconstruction baseline and temporal-component ablations provide useful evidence for the proposed approach.
- The authors clearly acknowledge the limitations and provide per-source results and bootstrap intervals.
Critical weaknesses
- There are no real-robot experiments, and transport is evaluated through a linear reader rather than physical execution. Recovering the donor action from generated features does not show that it is feasible or produces the intended effect in the recipient state.
- The LIBERO comparison changes both the latent representation and the policy interface. Without closed-loop ablations using the same policy head, it is unclear what drives the improvement. Results also use only one fine-tuning seed.
- The gains from EAT are not fully isolated. In particular, the paper lacks a reconstruction-plus-invariance baseline, making it hard to separate the proposed transport mechanism from standard invariance regularization.
- Robustness is mainly supported by offline readout metrics. Perturbed closed-loop rollouts are shown only qualitatively, so it remains unclear whether the representation gains translate into more reliable control.
Questions for authors
- Can you test whether transported actions are physically feasible and produce the intended effects in recipient states, rather than relying only on a linear reader?
- How do endpoint-only, TER, and TER+EAT compare in closed-loop performance with the same policy head and multiple training seeds?
- How does reconstruction plus invariance regularization compare with full EAT under the same training budget?
- Can you report quantitative success rates for the perturbed closed-loop evaluations described in Appendix B.2?