acceptodds
Under review as a conference paper at ICLR 2027

Actions Need Not Align: Predictive Consequence-Aware Transfer across Embodiments

Abstract

Learning from heterogeneous robot experience promises more data-efficient and reusable manipulation policies, yet incompatible native action spaces make cross-embodiment sharing difficult. Existing approaches often address this heterogeneity by making actions comparable through shared or explicitly aligned action representations. We instead explore whether transfer can be organized around the predictive consequences of native actions without explicitly aligning their control representations. We formulate cross-embodiment learning as an abstraction-dynamics-grounding problem and introduce Predictive Dynamics with Embodiment Grounding (PDEG). PDEG preserves each robot's native action space: embodiment-specific motion adapters expose heterogeneous trajectories to shared temporal abstraction and predictive modeling, while embodiment-conditioned queries retrieve control-relevant information from the shared predictive representation and ground it back into native controls. By sharing predictive computation rather than an explicitly aligned low-level action code, PDEG requires neither predefined joint correspondence, paired cross-embodiment demonstrations, cross-robot retargeting, nor a hand-designed common geometric action space. Experiments across three embodiments and three real-world manipulation tasks show positive cross-embodiment transfer under fixed target-robot data budgets.

Primary area:
applications to robotics, autonomy, planning
Keywords:
cross-embodiment learning, robot manipulation, imitation learning, predictive world models, heterogeneous action spaces, embodiment grounding

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_CZntOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes PDEG, a cross-embodiment imitation-learning framework combining shared predictive modeling with embodiment-specific encoding and control grounding, without explicit action-space alignment. Experiments on three manipulation tasks using three end effectors mounted on the same RM65 arm report improved success rates over UVA and adapted EgoMimic, alongside lower parameter growth per embodiment.

Critical strengths
  • The paper provides a conceptually clear alternative to explicit cross-embodiment action alignment by sharing predictive interaction dynamics while retaining embodiment-specific control interfaces.

  • The Single and Multi comparison keeps the target-embodiment data budget and predictive initialization fixed, providing reasonably controlled evidence that additional heterogeneous demonstrations can improve target-embodiment performance.

Critical weaknesses
  • The conclusion that cross-embodiment transfer does not require action alignment appears stronger than the current evidence supports. All three embodiments share the same RM65 arm and differ only in their end effectors, and the evaluation covers only three relatively simple manipulation tasks. It remains unclear whether the approach can handle broader embodiment gaps involving different arm kinematics, workspaces, and control structures.

  • Despite positioning explicit action alignment as the main alternative, the paper does not provide a controlled comparison with alignment-based approaches. Therefore, the experiments show that PDEG can achieve positive transfer without explicit alignment, but do not establish whether avoiding alignment is preferable, or under what conditions alignment is unnecessary.

Questions for authors
  • Why not evaluate cross-embodiment transfer under a more rigorous setting where the evaluated task is absent from the target embodiment’s training data but present in the training data of the other embodiments? Given the three relatively simple tasks considered in the paper, such an evaluation seems feasible and would provide more convincing evidence of cross-embodiment transfer.

  • Why are both Stage I and Stage II need 500 training epochs? Given the relatively limited size of the dataset, this appears to be a large training budget. Could the authors report the actual epochs of the validation-selected checkpoints and clarify whether such a long training horizon is necessary?

Reviewer_v1HpOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes PDEG, a cross-embodiment imitation learning method that keeps each robot's native action space instead of aligning actions across robots. Robot-specific adapters feed a shared temporal encoder and a UVA-style MAR backbone trained with future-video prediction. An embodiment-conditioned adapter (WCA) and a shared diffusion denoiser with robot-specific input/output layers then produce native actions. Experiments use three end effectors (10, 6, and 1 DoF) on the same 6-DoF arm across three real-world tasks. With the target robot's Stage-II data fixed, adding the other robots' data raises average success from 74.9% to 83.3% (+8.4 pp), versus +4.7 pp for UVA.

Critical strengths

S1. The paper evaluates positive transfer directly, on real robots. Showing that one network can run several robots does not show that their data helps each other. The paper asks the right question: it fixes the target robot's Stage-II data and measures the gain from adding other robots. Gains are positive in all 9 task–embodiment settings, with 50 physical rollouts per entry and Wilson confidence intervals. This careful real-world protocol is the main reason I see value in the paper.

S2. Native action spaces at low per-robot cost. PDEG needs no retargeting, joint correspondence, or paired demonstrations. Each new embodiment adds 8.32M parameters, compared with 52.26M for UVA. This makes the design easy to extend to more robots, which matters for the multi-embodiment setting the paper targets.

Critical weaknesses

W1. The experimental setup cannot test the paper's central claim. The title claims that actions need not be aligned for transfer. The setup does not let us check this.

  • All three end effectors are mounted on the same RM65 arm, and the action vector includes the arm joints. So 6 of the 7, 12, and 16 action dimensions are identical across robots by construction. Only the hand dimensions differ, and in these tasks the hand mostly opens and closes.
  • Success in peach placement, cup stacking, and ball pouring is mostly decided by the arm trajectory. The Multi gain therefore has two explanations: (A) knowledge transfers across unaligned hand actions through shared prediction, as the paper claims, or (B) the other robots simply add ~600 same-arm trajectories for the same task. The results fit both. Under (B), no alignment is needed because there is little to align. Robot-specific motion adapters do not rule out (B), since the arm inputs are the same physical quantities.
  • The two parts of the claim are also not tested separately. Not aligning actions is already what shared-policy methods (e.g., CrossFormer, HPT) do, and the predictive core is taken from UVA. So the claim needs (i) a comparison with alignment, e.g., PDEG plus a contrastive alignment loss on the action tokens, or LAD/XL-VLA, and (ii) evidence that prediction drives transfer, e.g., removing the video-prediction modes in Stage II. Neither is provided. The UVA comparison cannot isolate (ii), because UVA has the same predictive core.

This matters because the title and the main contribution rest on this claim. An arm-only Multi variant (source robots contribute only arm actions) or a failure breakdown (reaching errors vs. grasp/release errors) would help separate (A) from (B).

W2. The comparisons do not isolate where the transfer gain comes from.

  • UVA-Multi shares only the MAR backbone and keeps a separate action diffusion module per robot (App. D.3). PDEG shares its 6-block denoiser trunk. Other robots' data can therefore improve PDEG's action generator but not UVA's. The larger gain (+8.4 vs. +4.7 pp) may come from sharing the action head, a known effect in shared-policy work, rather than from predictive sharing.
  • Multi is trained for the same 500 epochs on ~3× the data, i.e., ~3× more gradient steps (App. D.2). The gain mixes extra data with extra training.
  • Single starts from the three-embodiment Stage-I checkpoint, so it has already seen the other robots' data. The reported transfer covers only Stage II.
  • UVA also starts from PDEG's Stage-I checkpoint. If that MAR was pretrained with PDEG's Morphology-Encoder state tokens, UVA starts from a mismatched initialization. This could explain part of the 5.8 pp gap between PDEG-Single and UVA-Single.

The main quantitative claim, that PDEG transfers better than UVA, depends on these comparisons. With these confounds, the evidence does not yet support it.

Questions for authors
  1. At Stage II, does UVA reuse PDEG's Morphology Encoder, or its own proprioception encoding? Was the Stage-I MAR trained with Morphology-Encoder state tokens?
  2. How large is the gain if source robots contribute only arm actions (arm-only Multi)? Alternatively, which failure types (reaching vs. grasp/release) decrease from Single to Multi?
  3. What is the transfer gain of PDEG with per-robot denoiser trunks, or of UVA with a shared action trunk? A step-matched Single baseline would also help.
  4. Does the transfer gain drop if the video-prediction modes are removed in Stage II? Please also report FVD for PDEG-Single vs. PDEG-Multi on the target robot.
  5. Does adding an explicit alignment loss on the action tokens (e.g., contrastive) help, hurt, or make no difference?
Reviewer_NnFWOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

PDEG shares prediction and control across robots while keeping each robot’s own action space, thereby reusing much of the same neural network trunk but receiving commands expressed in their own joints, without paired demonstrations or predefined action mappings. Tests cover three end effectors on the same arm and three tasks. Adding other robots’ demos raises average success from 74.9% to 83.3%, exceeding UVA’s 73.8%, while keeping the target robot’s second-stage training data fixed.

Critical strengths

I like the good real-world gains (Section 4.2; Table 1): PDEG-Multi outperforms UVA-Multi across all nine task/embodiment combinations. This provides practical evidence that the complete approach is effective.

I like that each additional embodiment requires fewer active parameters, compared to UVA (Appendix D.3; Table 5). At three embodiments, PDEG uses approximately 320M active parameters versus UVA’s 366M while achieving higher success, supporting the usefulness of its shared control architecture.

Critical weaknesses

Section 4.4, Table 2: The ablations test individual components but do not fully isolate the benefit of predicting future outcomes. They support PDEG as a complete system, without showing whether predictive learning explains its performance gains.

Section 4.2 (L413-414), Appendix D.2: Extra training may partly explain the gains. 'Single' and 'Multi' both train for 500 epochs, but 'Multi' uses more data and receives more updates. Without giving 'Single' the same number of updates, the benefit of other robots’ data is difficult to separate from extra training. The authors acknowledge this limitation in the manuscript, but, for me, this remains a concerning potential confounder.

Questions for authors

Section 4.4, Table 2: What specifically does predictive learning add? Can you compare PDEG with versions trained without future prediction and, separately, without conditioning predictions on actions, while keeping the control architecture and training budget fixed? This would clarify whether predictive learning explains the gains beyond the component ablations.

Does the transfer advantage remain with equal training updates? How many optimizer updates do 'Single' and 'Multi' receive, and does 'Multi' still outperform target-only training when these counts are matched? This would help separate the benefit of other robots’ data from extra training, alleviating the limitation that the manuscript self-admits.

Comments on the paper

1 comment

As we near the end of the discussion period for this submission, please check whether the authors have answered questions or responded to critiques. If they have, we encourage the reviewers to leave a comment for the authors. If reviewers have raised new points, a gentle nudge to the authors to respond would also be appreciated.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.