acceptodds
Under review as a conference paper at ICLR 2027

EDAR: Learning Environment-Dependent Action Representations for Robotic Manipulation

Abstract

Learning effective action representations is critical for robotic manipulation, where raw control trajectories are often noisy, redundant, and difficult to model directly. Existing methods mainly encode the structure of the action stream itself, treating the role of actions in the environment as implicit. Yet manipulation is about changing the world: the same action segment can induce different outcomes under different scene contexts, making action semantics inherently environment-dependent. We propose EDAR, an Environment-Dependent Action Representation that grounds action tokens in both executable control structure and expected visual consequences. By coupling motor commands with their environment-conditioned effects, EDAR encourages the learned action space to capture interaction semantics rather than merely command-level patterns. Experiments on simulated and real-robot manipulation benchmarks demonstrate that EDAR improves downstream policy learning, especially in long-horizon manipulation. These results highlight the importance of grounding action representations in executable control structure and environment-conditioned visual change.

Primary area:
applications to robotics, autonomy, planning
Keywords:
action representation learning, vision-language-action models, robotic manipulation, world models

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_CBQ1OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes EDAR, an environment-dependent action representation for robotic manipulation. It encodes action chunks together with current visual observations into compact continuous latent tokens, trained through action reconstruction and future visual-feature prediction. A decoder with shared attention and modality-specific feed-forward branches couples these objectives. The learned representation is then frozen and integrated into downstream policies that predict latent actions. Experiments across three simulation benchmarks and four real-world tasks show consistent improvements over the evaluated baselines, particularly on long-horizon tasks.

Critical strengths
  1. EDAR addresses the limitation that similar motor trajectories can produce different effects depending on the scene. Combining action reconstruction with future visual-feature prediction provides a coherent way to preserve control information while encouraging sensitivity to interaction outcomes.

  2. Improvements across multiple benchmarks and policy backbones suggest that EDAR’s benefits extend beyond a particular architecture or task. The action-only latent baseline and component ablations help distinguish the benefits of latent action modeling from the additional contributions of environment conditioning and predictive supervision.

Critical weaknesses
  1. EDAR is closely related to world-action models (WAMs), as both use future visual prediction to support action learning. Existing WAM approaches such as UVA (RSS 2025) jointly train the video–action representation with predictive losses, while DreamZero (2026) updates its shared backbone and action modules during both pretraining and task-specific adaptation. EDAR instead learns an action representation separately and freezes its encoder and decoder during downstream policy training. But the paper does not establish the benefit of keeping this representation fixed rather than allowing it to adapt alongside the policy. A controlled comparison between frozen and jointly optimized representations, using the same future-feature supervision, would help clarify the value of this design.

  2. EDAR conditions its action representation on visual context, which may introduce sensitivity to task-irrelevant changes in background, camera viewpoint, or object appearance. The paper does not test whether a frozen EDAR representation generalizes to unseen environments without fine-tuning. This leaves an important question about its usefulness as a reusable action interface: does it capture transferable interaction semantics, or does it require adaptation to each new environment?

Questions for authors

How well does a frozen EDAR representation transfer to unseen environments without fine-tuning, particularly under task-irrelevant visual changes?

2 comments
AuthorsOpenReview ↗

Response to Reviewer CBQ1

W1: Frozen versus Jointly Trained Representations

Thank you for the question. We explain this choice below.

Separating representation learning from policy learning and keeping the pretrained representation frozen during downstream training offer important advantages:

A pretrained representation serves as a fixed plug-in executable action interface that could be integrated into different policy backbones by replacing their original action interfaces. Downstream policies only need to predict latent actions, without additional representation-learning objectives or optimization overhead during downstream training. Meanwhile, freezing the representation avoids potential risks associated with joint optimization, including conflicts between imitation learning and representation-learning objectives, representation drift, and policy-specific representations that may not transfer well across different policy backbones. This is essentially a trade-off between task-specific co-adaptation and maintaining a stable, reusable action interface.

We integrate EDAR into SLIM, a WAM that jointly optimizes future prediction and latent representation learning during policy training, improving CALVIN ABC$\rightarrow$D performance from 4.556 to 4.661. This demonstrates that policy-level latent action jointly learning can also benefit from interface-level frozen consequence-aware learning, suggesting they are parallel and complementary research directions. Our contribution addresses a distinct and reusable action-interface-level design.

Method1/52/53/54/55/5Avg. Len.
SLIM99.396.792.387.180.24.556
SLIM-EDAR99.497.694.889.784.64.661

 

W2, Q1: Zero-Shot Transfer of Frozen Representations

We thank the reviewer for this suggestion. Our focus is on learning effective environment-conditioned executable action representations, rather than explicitly enforcing invariance to task-irrelevant visual changes.

To further examine generalization, we conducted additional experiments on CALVIN ABC$\boldsymbol{\rightarrow}$D, where both representation and policy training use only environments A–C, and evaluation is performed on the unseen environment D without fine-tuning. EDAR improves the average completed sequence length from 2.831 to 3.299 on SmolVLA and from 4.556 to 4.661 on SLIM. These results demonstrate that EDAR's downstream benefits extend to a held-out simulated environment, although this evaluation does not independently isolate appearance-only variations.

Method1/52/53/54/55/5Avg. Len.
SmolVLA81.569.255.842.933.72.831
SmolVLA-EDAR87.076.365.355.445.93.299
SLIM99.396.792.387.180.24.556
SLIM-EDAR99.497.694.889.784.64.661

Dear Reviewer,

We hope you have had a chance to look at our point-by-point responses. We would sincerely appreciate hearing your thoughts on our rebuttal and whether the clarifications address the concerns raised in your review. With the discussion period drawing to a close, we would greatly appreciate your feedback. Thank you very much for your time and thoughtful consideration.

Reviewer_NdmeOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

Proposed method encodes action chunks and current visual features into continuous latent tokens supervised by action reconstruction and future visual-feature prediction, then trains a flow-matching policy to generate tokens that a frozen decoder converts into actions. Evaluation includes three simulation benchmarks, four real-world tasks, and ablations of visual conditioning, future prediction, decoder design, and visual targets.

Critical strengths
  1. Proposed method is evaluated on LIBERO, CALVIN, Meta-World, and four real-world tasks across several policy backbones. Ablations examine visual conditioning, future prediction, decoder design, and visual targets, providing empirical coverage of the main components.
  2. Separating representation learning from downstream policy training provides a reusable action interface. The frozen representation can be integrated into different policy backbones without jointly optimizing the representation and policy.
  3. The implementation is described in useful detail, including the attention masks, adaptive loss weighting, and benchmark-specific training settings. These details make the main design choices and training procedure reasonably easy to follow.
Critical weaknesses
  1. The conceptual novelty is limited. Observation-conditioned action representations, including visual conditioning, are already established [1][2][3][4], as is coupling action decoding with transition prediction [1][3][5]. Proposed method appears to be a minor variation within this existing design space, without comparisons establishing a substantive advance over the closest approaches.
  2. The policy already models visual observations, but proposed method does not establish why this information should also be embedded in the action representation. Comparisons with applying the same auxiliary supervision to the policy are missing. Moreover, the future-prediction branch directly receives current visual features, and no intervention establishes how much it actually relies on the action latent.
  3. The claim of learning environment-dependent interaction semantics is overstated. Proposed method conditions action tokens on current visual features and predicts future endpoint features; this does not establish that the tokens capture interaction dynamics beyond appearance. The three selected latent-space examples do not resolve this distinction, and controlled tests separating visual changes from changes in interaction conditions are absent.
  4. Real-world generalization within each task is insufficiently tested. Each task uses 100 demonstrations and a separately trained policy, without a clearly specified held-out variation protocol. Evaluation on unseen object arrangements or perturbed intermediate states is needed to establish robustness beyond the demonstrated configurations.
  5. The long-horizon gains do not establish the proposed explanation of reduced error accumulation. Higher chain-completion rates can result from better individual-skill execution. Without separating local execution improvements from maintaining or recovering multi-stage progress, the claims about sequential stability remain insufficiently supported.
  6. The reported gains lack uncertainty estimates across independent training seeds. This matters particularly for smaller ablation differences, such as the 0.9-point contribution of encoder-side visual context, and for real-world evaluation with only ten runs per task.
  7. Comparisons with the original policy backbones match downstream training budgets rather than total training effort. Proposed method adds representation learning and visual-feature extraction, without a compute-matched comparison that allocates an equivalent budget to the baseline. Training cost and inference latency are also unreported, leaving the performance-compute trade-off unclear.

[1] Ruijie Zheng et al. PRISE: LLM-Style Sequence Compression for Learning Temporal Action Abstractions in Control. ICML, 2024, pp. 61267-61286.

[2] Wenhui Tan et al. RoLD: Robot Latent Diffusion for Multi-task Policy Modeling. MMM, Part III, 2025, pp. 340-353.

[3] Boyu Chen et al. UniT: Toward a Unified Physical Language for Human-to-Humanoid Policy Learning and World Modeling. CoRR, abs/2604.19734, 2026.

[4] Zhexuan Zhou et al. Latent Diffusion Policy: Shaping Latent Spaces for Diffusion-Based Robotic Manipulation. CoRR, abs/2606.08657, 2026.

[5] Xiaoyu Chen et al. villa-X: Enhancing Latent Action Modeling in Vision-Language-Action Models. ICLR, 2026.

Questions for authors
  1. Can you clarify the substantive contribution beyond existing observation-conditioned action representations and joint action-transition supervision, and demonstrate the value of the remaining differences through controlled comparisons?
  2. Can applying the same visual supervision directly to the policy match the reported gains? How much does future prediction degrade when action latents are shuffled while current visual inputs remain fixed?
  3. Can controlled tests vary appearance and interaction conditions independently to examine what the latent representation captures, including zero-shot evaluation under appearance-only changes?
  4. Can the same real-world policies be evaluated on held-out object arrangements and perturbed intermediate states without retraining?
  5. Can you separate improvements in individual-skill execution from improvements in maintaining or recovering multi-stage progress through single-skill evaluation and stage-level failure analysis?
  6. Can you clarify the number of independent training seeds and report uncertainty estimates for the main improvements and ablation differences?
  7. Can you report total training cost and inference latency, and compare against a raw-action policy with an equivalent total training budget?
3 comments
AuthorsOpenReview ↗

Response to Reviewer Ndme

W1, Q1: Novelty and Distinctions from Prior Work

Thank you for highlighting these related works.

We respectfully disagree that EDAR is merely a minor variation of existing methods. Our contribution lies not in introducing individual architectural components, but in addressing a distinct representation-learning problem: how to incorporate environment-conditioned visual consequences into executable action representations directly used by downstream policies. We clarify the differences from the cited works in terms of their research objectives, representation learning strategies, and roles in downstream execution.

(1) PRISE [1]: Temporal Action Abstraction

PRISE is related to EDAR in that it jointly optimizes action reconstruction and forward latent-state prediction. However, its primary objective is to discover variable-duration skills through discrete action quantization and BPE-based sequence compression, rather than learning consequence-grounded continuous action representations. Moreover, PRISE employs separate modules for action decoding and latent-state prediction. In contrast, EDAR predicts future features from a pretrained visual encoder and explicitly couples action reconstruction with consequence prediction through shared attention and modality-specific FFNs. This coupling is supported by our decoder ablation in Table 4(c): replacing the proposed decoder with two separate decoders reduces LIBERO success from 91.8% to 88.7%.

(2) RoLD [2] and LDP [4]: Executable Action Representations without Consequence Supervision

Both methods learn observation-conditioned latent action representations that can be generated by downstream policies and decoded into executable actions. However, neither incorporates future visual-feature prediction into representation learning. Their objectives primarily focus on modeling action trajectories or shaping latent distributions for policy generation. EDAR instead explicitly introduces future visual consequences as supervision for learning the executable action space.

(3) UniT [3] and villa-X [5]: Visually Grounded Intermediate Latent Actions

These methods also exploit visual dynamics, but their learned representations serve different purposes. UniT learns unified latent tokens for cross-embodiment alignment, using them as intermediate prediction targets for VLM training, while a separate flow-matching action head generates executable controls. Similarly, villa-X infers latent actions from visual transitions and uses a latent-action expert to guide a separate robot-action expert. In contrast, EDAR's latent tokens directly constitute the downstream policy's action prediction space and are converted into executable controls through a frozen decoder. Thus, consequence supervision directly shapes the executable action interface rather than an intermediate representation used to guide action generation.

(4) Empirical Validation

These distinctions are supported by our controlled experiments. Removing future visual prediction reduces LIBERO success from 91.8% to 90.1%, even with visual conditioning retained. Furthermore, EDAR outperforms the action-only latent baseline (SmolVLA-VAE: 89.1%), demonstrating benefits beyond latent action modeling alone. Together with the decoder ablation, these results support the effectiveness of consequence-supervised executable representation learning.

In summary, EDAR's novelty lies in its problem formulation and the integration of visual consequence supervision into the executable action space, rather than in architectural differences alone.

W2, Q2: Policy-Level Supervision and Reliance on Action Latents

Thank you for these questions. We address both below.

(1) Why Embed Environmental Information into Action Representations

A pretrained representation serves as a fixed plug-in executable action interface that could be integrated into different policy backbones. Downstream policies only need to predict latent actions, without additional representation-learning objectives or optimization overhead during downstream training. Meanwhile, freezing the representation avoids potential risks associated with joint optimization, including conflicts between imitation learning and representation-learning objectives, representation drift, and policy-specific representations that may not transfer well across different policy backbones.

Importantly, we view policy-level visual modeling and action-interface-level visual modeling as parallel and complementary directions. Both directions explore how to leverage environmental information and future predictions, but at different levels. To validate this complementarity, we additionally integrate EDAR into SLIM, a world-action model that already incorporates future-latent prediction into its policy learning framework. On CALVIN ABC$\boldsymbol{\rightarrow}$D, we obtain:

Method1/52/53/54/55/5Len.
SLIM99.396.792.387.180.24.556
SLIM-EDAR99.497.694.889.784.64.661

Despite already leveraging visual observations and future-predictive learning, SLIM benefits further from EDAR's action interface (4.556 →\boldsymbol{\rightarrow} 4.661). This provides evidence that the two directions can be complementary rather than redundant. We consider policy-level supervision a promising direction; rather, our contribution addresses a distinct and reusable action-interface-level design.

(2) How Much Does Future Prediction Rely on Action Latents

We conducted a controlled latent intervention experiment (Figure F, Appendix C.2), holding current visual features fixed while shuffling or randomizing action latent values. The future-feature prediction error increases from 0.0400 to 0.1630/0.1627 (~4.1$\boldsymbol{\times}$). Importantly, the error visualizations reveal particularly pronounced degradation in task-relevant regions, especially around the robot end-effector and manipulated objects. These results demonstrate that the decoder substantially relies on action latent content despite having direct access to current visual features.


 

W3, Q3: Interaction Dynamics versus Visual Appearance

We thank the reviewer for this helpful comment. Our primary focus is on learning environment-dependent executable action representations through future visual consequence supervision, rather than explicitly modeling or disentangling interaction dynamics. We agree that the latter is not the central objective of this work, and have therefore softened the relevant statements to avoid overinterpretation. Figure B is presented as a qualitative illustration of environmental dependence rather than evidence of disentangled interaction semantics.


 

W4, Q4: Real-World Generalization to Unseen Layouts and Perturbations

We thank the reviewer for this suggestion. Our real-world experiments primarily aim to evaluate whether EDAR provides a more effective executable action interface for downstream policy learning. While the results demonstrate consistent improvements across four manipulation tasks, we acknowledge that robustness to held-out object arrangements and perturbed intermediate states has not been systematically evaluated. We have clarified the scope of our real-world evaluation and avoid making broader generalization claims.


 

W5, Q5: Sources of Long-Horizon Gains

Thank you for this distinction. We clarify our interpretation below.

Our long-horizon experiments are intended to validate EDAR's effectiveness on sequential manipulation tasks, rather than to claim that EDAR specifically reduces accumulated errors. A detailed stage-level failure analysis is beyond the scope of this work. We have refined the corresponding wording to avoid overinterpretation, without affecting EDAR's central claims or empirical conclusions.

W6, Q6: Training Seeds and Uncertainty Estimates

We appreciate your concern about variability.

We have consistently demonstrated EDAR's effectiveness across diverse benchmarks (LIBERO, CALVIN, Meta-World, and real-world tasks) and multiple policy backbones (SmolVLA, π0\pi_0, FLOWER-VLA, and Evo-1). Such consistent improvements across independent experimental settings provide substantial empirical evidence that EDAR's effectiveness is not merely an artifact of random training fluctuations in a particular setting. While multi-seed evaluation would provide additional statistical characterization, the broad consistency of our results already supports the robustness of the overall performance improvements.

For real-world experiments, we have increased the number of evaluation trials from 10 to 20 per task, with the overall conclusions remaining unchanged.

Regarding the encoder-context ablation in Table 4(a), a closer examination reveals substantial improvements on LIBERO-Spatial (+1.6 pp) and LIBERO-Long (+1.4 pp), despite the modest average gain (+0.9 pp). These nontrivial gains across distinct task suites suggest that the benefits of encoder-side environmental conditioning are unlikely to be merely artifacts of training-seed variation.

MethodSpatialObjectGoalLongAverage
Full EDAR94.698.093.880.691.8
w/o Encoder Context93.098.093.279.290.9

 

W7, Q7: Compute Fairness, Training Cost, and Inference Latency

Thank you for asking. We provide the cost breakdown below.

(1) Training Cost and Compute-Matched Comparison

EDAR requires an additional representation-learning stage, taking approximately 13 hours on two RTX 4090 GPUs under our configuration. Importantly, EDAR is trained once and can be reused across multiple policy backbones. To address your concern, we additionally trained a raw-action SmolVLA with an equivalent extra training budget on LIBERO:

MethodSpatialObjectGoalLongAvg.
SmolVLA (extra compute)92.296.892.072.888.5
SmolVLA-EDAR94.698.093.880.691.8

EDAR retains a clear advantage (+3.3 pp overall, +7.8 pp on LIBERO-Long) against the baseline.

(2) Inference Latency

We measured the latency of generating 100 actions:

MethodLatency (s)
SmolVLA2.11
SmolVLA-EDAR2.26

EDAR introduces only 0.15 s of additional latency, which is negligible compared with the total inference latency.

Reviewer_WsA2OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes EDAR, an environment-dependent continuous action representation for robotic manipulation. A Transformer jointly processes an action chunk, current visual features, and learnable register tokens to produce a compact latent representation. This representation supports both action reconstruction and prediction of future visual features. The decoder combines shared attention with modality-specific feed-forward branches. After representation learning, EDAR is frozen, and a downstream flow-matching policy learns to generate its latent action tokens. The main contribution is the proposed coupling of executable action structure with scene-conditioned visual consequences. Experiments cover LIBERO, CALVIN, Meta-World, and four real-world manipulation tasks, with integrations into several policy backbones.

Critical strengths
  • The reported improvements on LIBERO-Long and CALVIN are consequential. They suggest benefits in settings where errors accumulate across sequential interactions.
  • Comparing against SmolVLA-VAE helps assess whether the gains extend beyond learning a compressed action interface. EDAR improves LIBERO average success from 89.1% to 91.8% and real-world normalized progress from 60.5 to 69.0. These comparisons provide useful support for adding visual context and consequence supervision.
Critical weaknesses
  • The visual prediction objective does not require an environment-dependent action code: a context-invariant representation Z=f(A)Z=f(A) could preserve the action trajectory while the decoder combines ZZ with the separately supplied current visual features to predict the future. Table 4(a) is informative here: removing encoder-side context retains 90.9% success, compared with 91.8% for full EDAR. This supports a possible benefit from conditioning, but leaves the stronger mechanistic interpretation unresolved.
  • Appendix B provides only three illustrative pairs. Larger raw latent distances across independently trained models are difficult to interpret without controlling their scales, and separation by scene context does not establish separation by interaction consequences.
  • The contribution also needs sharper positioning against closely related approaches. LatBot[1] jointly supervises latent actions through future-frame reconstruction and physical-action generation, while XR-1[2] aligns visual dynamics and robot motion in a shared latent space. EDAR’s particular inputs, continuous bottleneck, and decoder architecture warrant explicit comparison with these approaches.
  • EDAR and VAE receive additional Open X-Embodiment representation pretraining, so the improvement over standard SmolVLA cannot be attributed solely to the action interface. The EDAR–VAE comparison is more informative, but requires a precise baseline objective and confirmation of matched representation-training budgets.

[1] Z. Li, X. Gao, X. Wang, and J. Fu, “LatBot: Distilling Universal Latent Actions for Vision-Language-Action Models,” arXiv preprint arXiv:2511.23034, 2025. Paper.

[2] S. Fan et al., “XR-1: Towards Versatile Vision-Language-Action Models via Learning Unified Vision-Motion Representations,” arXiv preprint arXiv:2511.02776, 2025. Paper.

Questions for authors
  • What is EDAR’s substantive distinction from LatBot and XR-1 beyond its particular encoder inputs, continuous representation, and decoder implementation? Which existing results demonstrate the benefit of that distinction?
  • What evidence distinguishes effect-sensitive action semantics from ordinary visual-context encoding? How do you rule out a context-invariant action code whose decoder uses the separately supplied scene features?
  • Is SmolVLA-VAE a variational model or a deterministic autoencoder? Please also clarify the action-interface changes used for π0 and Evo-1.
2 comments
AuthorsOpenReview ↗

Response to Reviewer WsA2

W1, W2, Q2: Environment Dependence of Action Latents

Thank you for raising this concern. We clarify below.

(1) Mechanism of Encoder-Side Environment Dependence

Structurally, EDAR employs bidirectional Transformer self-attention, where register tokens aggregate value vectors from all unmasked tokens, including image tokens. Since softmax attention assigns nonzero weights to these tokens, visual features naturally contribute to register-token updates. Our new Figure D (Appendix B.2) further shows substantial register-to-image attention, confirming that this information pathway is actively used. This provides a plausible explanation for the improvement in Table 4(a) (90.9% →\rightarrow 91.8%): incorporating environmental context into action latents benefits downstream policy learning.

More decisively, our new Figure C (Appendix B.1) holds two action chunks fixed and encodes each under three different visual contexts using the same EDAR encoder. The resulting latents differ across environments, whereas the action-only VAE produces identical latents for the same action. This directly verifies encoder-side context dependence in these controlled examples.

(2) Interpretation of Appendix B Visualizations

Figure B is intended only as a qualitative illustration, not definitive evidence of interaction-semantic separation. We emphasize that our central claim concerns environment-dependent executable action representations, rather than explicitly learning or disentangling physical interaction dynamics, and such an interpretation would go beyond our intended claims. We have moderated the relevant claims accordingly.

(3) Effect-Sensitive Semantics versus Ordinary Visual-Context Encoding

The key distinction is that visual-context encoding merely incorporates the current scene, whereas effect-sensitive representation learning additionally encourages the latent to account for the consequences of executing an action.

We provide complementary evidence:

Table 4(b): Removing future prediction reduces success from 91.8% to 90.1%, despite retaining the same environment-conditioned encoder. This demonstrates that consequence supervision which contains effect-sensitive information also helps shape the action latents.

The finding supports a plausible interpretation that EDAR captures a certain form of action-relevant, effect-sensitive information beyond ordinary visual-context encoding.


 

W3, Q1: Novelty and Distinctions from LatBot and XR-1

We thank the reviewer for raising this important question and for pointing out these closely related works. Although EDAR, LatBot [1], and XR-1 [2] share certain methodological similarities, they address distinct research problems.

Both LatBot [1] and XR-1 [2] leverage visual and action supervision to learn latent representations, but primarily use them as intermediate supervision for downstream policies. In contrast, EDAR addresses a distinct problem: learning better executable action representations that serve as plug-in replacements for raw actions in downstream policy learning, instead of using latent actions as an auxiliary prediction target while retaining a separate raw-action generation pathway.


 

W4: Pretraining Fairness and VAE Baseline Definition

Thank you for raising the fairness concern.

To address this concern, we conducted additional real-world experiments using π0\pi_0, which has already been pretrained on OXE data. The results are reported below.

MethodStack-5DrawerAvg.
π0\pi_0606361.5
π0\boldsymbol{\pi_0}-EDAR817779.0
Improvement+21.0+14.0+17.5

These results demonstrate that EDAR delivers substantial improvements even on an OXE-pretrained VLA backbone, indicating that its gains cannot be explained by OXE data exposure alone and providing further evidence for the effectiveness of the EDAR action interface.


 

Q3: Implementation Details

Thank you for asking. We clarify these details below.

(1) SmolVLA-VAE

SmolVLA-VAE (as well as EDAR) is trained with a variational Gaussian latent distribution, but uses the posterior mean deterministically during downstream policy training and inference, with the encoder and decoder frozen.

(2) Action Interfaces for π0\boldsymbol{\pi_0} and Evo-1

For π0\boldsymbol{\pi_0} and Evo-1, we replace their original action interfaces with EDAR latent tokens. For EDAR, π0\pi_0 uses the same action interface as SmolVLA-EDAR, and Evo-1 uses EDAR pretrained on its Meta-World dataset.

Dear Reviewer,

We sincerely appreciate the time and effort you have invested in evaluating our work. We have carefully addressed your comments and provided point-by-point responses. We would greatly appreciate it if you could take a moment to review our responses and share your thoughts. As the discussion phase is drawing to a close, we hope to have the opportunity to hear your feedback.

Comments on the paper

3 comments
AuthorsOpenReview ↗

Clarifying EDAR's Positioning and Contributions

1. EDAR Focuses on Executable Action Interfaces

In this paper, action representations specifically refer to executable representations learned separately from downstream policies and used as plug-in replacements for raw actions. This differs from World Action Models (WAMs), which primarily integrate world prediction into policy learning, and Latent Action Learning, which often exploits action-free videos to learn intermediate representations rather than directly executable interfaces. These are distinct but complementary research directions.

2. EDAR Systematically Explores Environment Dependence in Executable Action Representations

While prior methods such as PRISE, LDP, and RoLD share certain methodological similarities with EDAR, environment dependence is not their primary research objective, and their learning frameworks and architectures are not specifically designed around this goal. In contrast, EDAR explicitly targets this underexplored problem, with its learning objectives and architectural design tailored to grounding executable action representations in environment-conditioned consequences.

3. EDAR Demonstrates the Effectiveness of This Direction

On LIBERO, EDAR outperforms representative action-only representations, including FAST, VQ-VLA, and our controlled VAE baseline. Furthermore, across three simulation benchmarks (LIBERO, CALVIN, and Meta-World), real-world manipulation tasks, and multiple policy backbones, EDAR consistently improves downstream policy performance. These results demonstrate the effectiveness of environment-dependent modeling for learning better executable action interfaces.

Area_Chair_RUkZOpenReview ↗

Disucssion Reminder

Dear Reviewers,

I hope this message finds you well. As we are now in the discussion period, I would like to kindly remind you to leave comments for the authors through OpenReview if you have not already done so.

Thank you very much for your time and effort. Please let me know if you have any questions.

Best

Dear Reviewers,

Thank you very much for your thoughtful and detailed reviews. We sincerely appreciate the time and effort you have devoted to evaluating our work. Your valuable feedback has helped us further clarify and improve our manuscript.

As the discussion phase is drawing to a close, we would like to kindly invite you to share any further questions or comments you may have. We would be very happy to address them and clarify any remaining points that might help with your assessment.

Thank you again for your time, consideration, and engagement.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.