acceptodds
Under review as a conference paper at ICLR 2027

BATCH: Behavior Alignment for Cross-Embodiment Learning from Humans

Abstract

Human demonstrations provide a scalable source of diverse manipulation data, but leveraging them for robot learning remains challenging due to differences in embodiment, viewpoint, action spaces, and temporal structure. We introduce BATCH, a cross-embodiment learning framework that explicitly aligns human and robot behaviors. BATCH learns self-supervised temporal representations with a cycle-consistency objective to establish cross-embodiment correspondences without manual annotation. We further introduce two complementary training objectives that leverage these correspondences within an autoregressive video-action diffusion model: behavior-aligned co-training and human-to-robot style transfer. On real-world manipulation tasks, BATCH improves generalization to unseen task configurations compared with training on unaligned data and prior cross-embodiment baselines, with additional gains from the style-transfer objective. Our results demonstrate that explicit behavior alignment enables more effective transfer from human demonstrations to robot policies.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Cross-Embodiment Learning, Learning from Human Data, Robot Manipulation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 2 reviews, above 27% of the others.
Reviewer_zX4bOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

BATCH addresses cross-embodiment robot learning from human demonstrations. It learns temporal representations using GTCC and applies subsequence dynamic time warping to retrieve and temporally align human demonstration segments with robot windows. The aligned data are used to fine-tune a video-action diffusion model through two objectives: attention-isolated human–robot co-training with shared parameters, and human-to-robot future-video prediction conditioned on aligned human observations and robot scene context. Deployment requires only the robot stream.

The evaluation measures correspondence quality using phase overlap and phase-progress error, and evaluates real-world policy execution on Object in Container and Cup on Saucer. The policy experiments test configurations represented in human demonstrations but absent from robot training data. The paper reports improved aggregate performance over the compared baselines and ablations.

Critical strengths
  1. The cross-embodiment prediction objective provides a concrete extension of human–robot co-training. The style-transfer branch predicts robot future video using an aligned human clip while masking the robot conditioning latent and action/state registers. This explicitly couples the embodiments during training while preserving robot-only inference. Its integration into a video-action model provides a plausible mechanism for transferring information from human demonstrations to robot control.
  2. The evaluation connects correspondence learning with real-world policy performance. Table 1 compares temporal representations, generic visual features, pose information, and temporal-matching alternatives. The policy experiments separately compare the complete method with co-training, OT-based transfer, and removal of alignment. The improved aggregate performance on both single-arm and bimanual manipulation supports the practical potential of the framework.
Critical weaknesses
  1. The downstream advantage of the proposed alignment design over simpler alternatives remains insufficiently established. Table 1 compares several representations and matching strategies, but the policy evaluation contrasts the proposed alignment primarily with randomly sampled, unaligned trajectories. It does not include DINOv3 + DTW or the GTCC variant without DTW, although both are evaluated for correspondence quality. Consequently, the experiments support the complete alignment procedure but do not establish whether the advantages of its particular representation and matching choices translate into better control.

    Moreover, the “without alignment” ablation changes both which human segments are selected and their temporal correspondence. It therefore does not separate relevant-segment selection from temporal alignment. I regard this as a limitation in attributing the policy gains, rather than evidence that the alignment procedure is ineffective. Nevertheless, this attribution is consequential because behavior alignment is a central contribution of the paper.

  2. The stability of the reported policy advantage is unclear from the presented evaluation. Figures 4 and 6 provide aggregate and per-scenario point estimates without showing uncertainty. The training and evaluation descriptions do not clearly identify how many independently trained policies contribute to these results. Thus, the consistency of the gains over EgoVerse and the OT variant across training runs cannot be assessed from the reported comparisons.

    This does not establish that the results are irreproducible. It does, however, limit confidence in the empirical advantage, particularly when assessing the incremental contribution of the proposed training objective. Reporting the number of independent training runs and any available uncertainty would materially strengthen the evidence.

Taken together, these limitations leave me leaning toward weak rejection: the complete framework is promising, but the evidence for the downstream value of its alignment choices and the stability of its empirical advantage is not yet sufficiently convincing.

Questions for authors
  1. How does downstream policy performance change when the same training objectives use DINOv3 + DTW or GTCC without DTW? Do the correspondence-quality differences in Table 1 translate into differences in robot execution?
  2. Can the authors distinguish relevant-segment retrieval from temporal correspondence through a control that retains the retrieved windows and sampling frequencies while perturbing their pairing or relative temporal alignment? This would clarify which aspect of the alignment procedure explains the policy gains.
  3. How many independently trained policies underlie Figures 4 and 6? If multiple independently trained policies were evaluated, are the gains over EgoVerse and the OT variant consistent across runs? Please clarify any available uncertainty estimates.

These questions concern validation of the current contribution rather than requests for additional application domains.

Reviewer_9Um9OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper investigates how to improve human-to-robot transfer by explicitly aligning human and robot behaviors in time. Specifically, the authors learn a shared temporal embedding with a self-supervised cycle-consistency objective (GTCC) and apply subsequence DTW to retrieve human segments matching each robot window despite differences in execution speed. The aligned pairs are then used to finetune a video-action diffusion model with joint co-training and a style-transfer objective that predicts the robot’s future video from the aligned human clip. Experiments on two real-world tasks show that the learned embedding gives the strongest visual-only alignment and that the resulting policy outperforms co-training alone in out-of-domain scenes seen only in the human data. Furthermore, this work compares against an optimal transport (OT) objective and an unaligned variant to assess each component.

Critical strengths
  • While prior works align human and robot demonstrations with DTW over actions or static visual features, this work learns a self-supervised temporal embedding (GTCC) for alignment without manual phase labels. Evaluated separately (Table 1), it clearly outperforms DINOv3 features, especially on out-of-domain Cup on Saucer scenes (71.7% vs. 22.1% overlap).
  • The model supports both co-training and human-to-robot style transfer in a single network while leaving the robot-only deployment unchanged. It performs best on both tasks, reaching 83.3% success on Cup on Saucer versus 63.3% for co-training alone and the OT variant (Figure 4). This suggests that predicting the robot’s future from the human clip transfers more than matching human and robot features.
Critical weaknesses
  • Incorporating action information into the alignment cost (Section 4.1) seems heuristic, as GTCC features are simply concatenated with end-effector poses with no normalization or weighting described. The effect is also inconsistent across tasks, with absolute poses reducing the ID overlap on Object in Container from 69.0% to 43.7% but raising it on Cup on Saucer from 64.8% to 79.2% (Table 1). Furthermore, Figure 3 presents GTCC + abs. actions as the method for Cup on Saucer, yet Appendix B.1.4 states that the training pairs use GTCC embeddings only, so the best-aligned variant is not the one used for policy learning.
  • Since DTW-based alignment of human and robot demonstrations already appears in prior work (Section 2), the contribution of the alignment stage rests on showing that the learned GTCC alignment improves policy learning, which is not clearly isolated. The only policy-level test is against random pairing, which also makes the style-transfer target ill-posed, and on Cup on Saucer this variant even falls below co-training alone (50.0% vs. 63.3%). Policy results with pairs from simpler alignments (e.g., DTW on DINOv3 features or poses), and co-training with aligned versus random pairs, would help clarify what drives the gains.
Questions for authors
  • Section 4.2 describes the EgoVerse baseline as co-training without the style-transfer objective, but does it use the GTCC-aligned, time-warped pairs or randomly sampled human data? This determines whether its gap to the full method reflects style transfer alone.
  • Could the authors report co-training results with aligned versus randomly sampled pairs, or policy results using pairs from a simpler alignment such as pose-based DTW? Evidence that the learned alignment improves the policy beyond any temporal pairing would raise my assessment.

Comments on the paper

1 comment

Dear reviewers,

Thanks for serving as a reviewer. As the discussion period comes to a close and the authors have submitted their rebuttals, I kindly ask you to take a moment to review them and provide any final comments.

If you have already updated your comments, please disregard this message.

Thank you once again for your dedication to the OpenReview process.

Best,

Area Chair

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.