- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes EDAR, an environment-dependent action representation for robotic manipulation. It encodes action chunks together with current visual observations into compact continuous latent tokens, trained through action reconstruction and future visual-feature prediction. A decoder with shared attention and modality-specific feed-forward branches couples these objectives. The learned representation is then frozen and integrated into downstream policies that predict latent actions. Experiments across three simulation benchmarks and four real-world tasks show consistent improvements over the evaluated baselines, particularly on long-horizon tasks.
Critical strengths
-
EDAR addresses the limitation that similar motor trajectories can produce different effects depending on the scene. Combining action reconstruction with future visual-feature prediction provides a coherent way to preserve control information while encouraging sensitivity to interaction outcomes.
-
Improvements across multiple benchmarks and policy backbones suggest that EDAR’s benefits extend beyond a particular architecture or task. The action-only latent baseline and component ablations help distinguish the benefits of latent action modeling from the additional contributions of environment conditioning and predictive supervision.
Critical weaknesses
-
EDAR is closely related to world-action models (WAMs), as both use future visual prediction to support action learning. Existing WAM approaches such as UVA (RSS 2025) jointly train the video–action representation with predictive losses, while DreamZero (2026) updates its shared backbone and action modules during both pretraining and task-specific adaptation. EDAR instead learns an action representation separately and freezes its encoder and decoder during downstream policy training. But the paper does not establish the benefit of keeping this representation fixed rather than allowing it to adapt alongside the policy. A controlled comparison between frozen and jointly optimized representations, using the same future-feature supervision, would help clarify the value of this design.
-
EDAR conditions its action representation on visual context, which may introduce sensitivity to task-irrelevant changes in background, camera viewpoint, or object appearance. The paper does not test whether a frozen EDAR representation generalizes to unseen environments without fine-tuning. This leaves an important question about its usefulness as a reusable action interface: does it capture transferable interaction semantics, or does it require adaptation to each new environment?
Questions for authors
How well does a frozen EDAR representation transfer to unseen environments without fine-tuning, particularly under task-irrelevant visual changes?
2 comments
Dear Reviewer,
We hope you have had a chance to look at our point-by-point responses. We would sincerely appreciate hearing your thoughts on our rebuttal and whether the clarifications address the concerns raised in your review. With the discussion period drawing to a close, we would greatly appreciate your feedback. Thank you very much for your time and thoughtful consideration.
Response to Reviewer CBQ1
W1: Frozen versus Jointly Trained Representations
Thank you for the question. We explain this choice below.
Separating representation learning from policy learning and keeping the pretrained representation frozen during downstream training offer important advantages:
A pretrained representation serves as a fixed plug-in executable action interface that could be integrated into different policy backbones by replacing their original action interfaces. Downstream policies only need to predict latent actions, without additional representation-learning objectives or optimization overhead during downstream training. Meanwhile, freezing the representation avoids potential risks associated with joint optimization, including conflicts between imitation learning and representation-learning objectives, representation drift, and policy-specific representations that may not transfer well across different policy backbones. This is essentially a trade-off between task-specific co-adaptation and maintaining a stable, reusable action interface.
We integrate EDAR into SLIM, a WAM that jointly optimizes future prediction and latent representation learning during policy training, improving CALVIN ABC$\rightarrow$D performance from 4.556 to 4.661. This demonstrates that policy-level latent action jointly learning can also benefit from interface-level frozen consequence-aware learning, suggesting they are parallel and complementary research directions. Our contribution addresses a distinct and reusable action-interface-level design.
W2, Q1: Zero-Shot Transfer of Frozen Representations
We thank the reviewer for this suggestion. Our focus is on learning effective environment-conditioned executable action representations, rather than explicitly enforcing invariance to task-irrelevant visual changes.
To further examine generalization, we conducted additional experiments on CALVIN ABC$\boldsymbol{\rightarrow}$D, where both representation and policy training use only environments A–C, and evaluation is performed on the unseen environment D without fine-tuning. EDAR improves the average completed sequence length from 2.831 to 3.299 on SmolVLA and from 4.556 to 4.661 on SLIM. These results demonstrate that EDAR's downstream benefits extend to a held-out simulated environment, although this evaluation does not independently isolate appearance-only variations.