acceptodds
Under review as a conference paper at ICLR 2027

CrossWAM: Imagine, Execute, Refine for Zero-Shot Cross-Embodiment Manipulation

Abstract

Generalist robot policies, including vision-language-action (VLA) models and world-action models (WAMs), are typically pre-trained on data from many embodiments and then fine-tuned for one fixed robot, camera, and control interface. This coupling makes a trained policy difficult to reuse once the robot morphology or the camera configuration changes, even when the task and the underlying object dynamics are unchanged. We therefore study zero-shot cross-embodiment transfer, in which a policy trained on a single source embodiment must control robots never seen during training, under camera configurations never seen either, without collecting any target demonstration. Our starting point is that, how a task should unfold (how objects move and how contacts form) is largely independent of the robot that executes it, whereas the source robot's appearance is tightly bound to one particular arm and camera. Building on this observation, CrossWAM makes three components. First, we make the imagined future of a world-action model embodiment-agnostic, removing the source arm by video inpainting and re-expressing its motion as an abstract gripper trace. Second, because a viewpoint change reprojects the same world state onto different pixels and thereby biases the predicted actions, we close the loop with an imagination–observation refiner that compares the imagined future against what actually happened after execution and corrects the actions not yet executed. Third, an asynchronous denoising schedule assigns fewer denoising steps to near-future frames, reducing first-video latency by 48.5%. On the bimanual RoboTwin 2.0 benchmark, which jointly varies embodiment and camera pose, CrossWAM improves success rate by up to 22.1% over the strongest prior method, and real-robot experiments on two platforms confirm that it transfers under simultaneous embodiment and viewpoint shift.

Primary area:
applications to robotics, autonomy, planning
Keywords:
cross-embodiment, world action model

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_TNu4OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

In this paper, the authors propose CrossWAM, a world-action model that operates robustly across different robot embodiment morphologies and camera configurations. The proposed pipeline trains the world-action model based on video inpainting and gripper position overlays, which allows the WAM to predict consistent observations even when the target robot features a different embodiment or camera viewpoint, thereby enabling zero-shot cross-embodiment manipulation. Furthermore, to overcome the open-loop limitation of existing models that execute a predicted action chunk to completion without feedback, the authors introduce an Refiner. By training with this pipeline and utilizing it as a reference for closed-loop trajectory refinement during inference, the authors demonstrate improved success rates across three different embodiments compared to VLA and WAM baselines.

Critical strengths
  • The refiner compares the ideal imagined future image without the robot predicted by the WAM and the actual observation image of the target robot taken in real-time. By analyzing the discrepancy between the traces and object positions in both images, it calculates a translation residual, pulling the trajectory of the remaining unexecuted actions toward the correct contact point.

  • It demonstrates that the proposed method operates successfully not only on the source embodiment (Aloha-AgileX) but also on unseen embodiments such as ARX and Franka.

  • Although utilizing Wan2.2 for both the WAM and the refiner could potentially lead to a significant increase in latency, the authors show that their approach introduces no additional latency compared to Fast-WAM, thanks to asynchronous prediction and the parallelization of generation and execution.

Critical weaknesses
  • The refiner predicts offset given RGB images. Although the authors claim robustness to camera viewpoint changes, this mapping would completely collapse under significant camera movements (e.g., if the camera is shifted by 180 degrees to the opposite side.

  • The proposed method may not generalize beyond simple grippers. For instance, the approach of inpainting the robot body and leaving only the end-effector trace is unlikely to be effective for dexterous hands.

  • Despite the claim of a zero-shot setting, accurately overlaying the trace onto the observation image during deployment strictly requires the target robot's forward kinematics and precise camera calibration (extrinsics and intrinsics).

Questions for authors
  • WAM is trained to take the current observation containing the robot as input and predict a future outcome where the robot is erased. Given this setup, how can the model be inherently robust to the visual appearance of the specific embodiment in the input image? Furthermore, would applying robot inpainting to the input observation as well lead to further improvements in cross-embodiment performance?

  • Figure 3 illustrates four refinement steps occurring for a single video denoising step. If the underlying mechanism is essentially a 1-to-1 mapping, how exactly does this structure translate into a 1-to-4 latency ratio? Could you clarify the details of this execution timing?

4 comments
AuthorsOpenReview ↗

Response to Reviewer TNu4 (1/2)

We thank the reviewer for the positive review and the precise questions; we address each point below.

A1. Large camera changes. (W1)

We agree that an RGB-based refiner will fail under very large changes, such as moving the camera to the opposite side; our experiments cover moderate changes (up to ±15 cm with pitch compensation in simulation, and 15 cm with a 13° tilt on the real robot). Refining with 3D information, such as depth or point clouds, would indeed reduce the dependence on the camera pose. We have added this discussion to the limitations (App. A).

A2. Beyond simple grippers. (W2)

Agreed. The trace is defined by a set of end-effector keypoints, so it could include the fingertips of a hand, but we have not tested this. The revision states this in the limitations (App. A): "The embodiment-agnostic target assumes a parallel gripper. For dexterous hands, the trace would have to encode finger keypoints and contact patterns, which we have not tested. The premise that object motion is independent of the executing robot is also weaker for non-prehensile manipulation, where the shape of the end-effector determines the contact."

A3. Kinematics, calibration and "zero-shot". (W3)

We use "zero-shot" in the same sense as previous cross-embodiment methods such as Mirage (Chen et al., 2024b): no target demonstrations and no parameter updates. All three also need the target robot model and the camera calibration, and every baseline that runs on a new arm already needs its kinematic model for IK. Moreover, for a robot with a fixed setup, the camera needs to be calibrated only once.

To measure how much CrossWAM depends on accurate calibration, we perturb the camera extrinsics used to draw the gripper trace on the observation, with a fixed offset of 1 cm in translation and 1° in rotation in a random direction for each episode (ARX, raised and lowered cameras, randomized scenes). This perturbation is larger than the error of typical calibration: standard hand-eye calibration reaches about 1 mm and 0.3° (Li et al., 2023 (https://arxiv.org/pdf/2311.01335)).

SettingARX RARX L
Fast-WAM (Pretrained)36.442.0
CrossWAM, exact calibration60.756.7
CrossWAM, extrinsic error 1 cm / 1°56.152.9

With this error, CrossWAM drops by 4.6 / 3.8 points under the raised / lowered cameras and remains 19.7 / 10.9 points above Fast-WAM (Pretrained). A calibration error shifts the trace drawn on the observation, and this error therefore degrades the closed loop only slightly rather than breaking it. Moreover, for a robot with a fixed setup, the camera needs to be calibrated only once.

AuthorsOpenReview ↗

Response to Reviewer TNu4 (2/2)

A4. Robot appearance in the input. (Q1)

The model is not invariant to the robot in its input. What the agnostic target changes is what the model has to encode: with raw targets, predicting the future means rendering the source arm, so the features must carry its appearance; with agnostic targets they need not. Fig. 5 and Sec. 4.4 quantify this: the ratio of between-embodiment to within-embodiment feature distance falls from 0.923 to 0.316. Applying robot inpainting to the input observation may further improve cross-embodiment transfer. Mirage (Chen et al., 2024) follows this idea: it inpaints the target robot in the input and renders the training robot in its place. In the table below, Mirage improves over the baseline but remains below CrossWAM. Input inpainting also has two drawbacks: its result depends on the quality of the inpainting model, which can blur objects near the arm, especially when the arm is in contact with them; and it adds inference latency at every control step.

Mirage built on the same pretrained backbone as CrossWAM (Table 5 of the revision).

MethodAloha DRLARX DRLFranka DRL
Clean scenes
Fast-WAM (Pretrained)92.168.359.564.243.444.227.220.718.9
+ Mirage92.168.359.570.349.645.730.823.520.7
CrossWAM92.081.468.373.462.253.732.526.928.1
Randomized scenes
Fast-WAM (Pretrained)91.768.861.661.936.442.025.917.016.2
+ Mirage91.768.861.665.240.747.927.119.019.8
CrossWAM91.581.172.071.260.756.733.026.428.5

A5. Timing in Fig. 3. (Q2)

Thank you for pointing this out. The 1-to-4 ratio arises because, following existing WAMs such as Fast-WAM, CrossWAM predicts a 32-step action chunk together with a video that is temporally down-sampled by 4, i.e., 8 frames; each imagined frame therefore serves one refiner call, once every 4 actions. "Video Denoising I" and "II" in Fig. 3 are not single denoising steps but the two halves of the asynchronous schedule: I ends when the near-future frames (the first 4 of the 8) are clean, and II when the far-future frames are clean. In the revision, Sec. 3.2 defines the down-sampling factor nn (n=4n=4, H=32H=32), and Eqs. (5) and (6) in Sec. 3.3 are rewritten accordingly: the jj-th imagined frame I^t+j′\widehat{I}^{\prime}_{t+j} is compared with the observation ot+n⋅jo_{t+n\cdot j} taken after the first n⋅jn\cdot j actions, and the resulting offset corrects the remaining actions τ=t+n⋅j,…,t+H−1\tau=t+n\cdot j,\ldots,t+H-1.

Thank you to the authors for the response and additional experiments. I will keep my score.

Thank you for reading our response and for your positive assessment of the paper.

Reviewer_ncuwOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

CrossWAM aims to enable transfer of WAMs to new embodiments or viewpoint changes. This goal is accomplished by leveraging an embodiment-agnostic imagined future from the WAM model and introducing a refiner module that corrects discrepancies between actual execution and the imagined future. The proposed method takes an interesting perspective on the purpose of the imagined future in WAM models. In particular, recent work like FastWAM finds that the imagined future is useful for training, but can be bypassed using an action only output at test time. CrossWAM instead retains that imagined future, but uses it as a resource for enabling cross embodiment transfer. This perspective makes the paper valuable to the broader WAM community.

The authors evaluate their method in RoboTwin 2.0 simulation experiments (across two new embodiments and two new camera viewpoints) and in real-world experiments transferring from ARX arm to Piper arm. These experiments are well-defined and validate their methodology. The authors also isolate the benefits of their proposed approach with an ablation study and identify why it helps with a t-SNE plot which shows their model aligns features across embodiments. The additional experiments in the appendix show careful study of their findings. Overall, the contribution would benefit the WAM and cross-embodiment communities.

As this paper is a method contribution, my review focuses on evaluating whether the experimental design, empirical benefits, and ablation study are rigorous. Overall, the method is well-justified, with a few points of concern detailed below that are reasonably addressable during the rebuttal.

Critical strengths

S1: The main strength of the paper is the unique perspective on the role of the imagined future from WAM models. From FastWAM it seems the imagined future might only be necessary during training, but if the imagined future can be used to help cross-embodiment transfer that provides a new perspective to the field.

S2: The experimental setup in simulation and real world is well constructed. The simulation experiments compare different camera viewpoints and robot embodiments and find a clear improvement. The real-world experiments support the same findings.

Critical weaknesses

W1: The method makes a number of assumptions that are not properly evaluated or discussed.

(a) The choice of translation-only correction in the refiner is not empirically supported; an ablation that includes rotational or gripper width correction in the refiner would address this. Currently the motivation for translation-only is only supported by text in the main paper and appendix C, but not with empirical numbers.

(b) The claim that the task execution is independent of the robot breaks down for more complex embodiments (e.g., dexterous hand) or with different contact modes (e.g., non-prehensile manipulation). Expanding the limitations section to address this would satisfy this concern.

(c) From Fig 9 it appears the system uses both wrist and head camera, but it’s unclear how the proposed inpainting and robot-agnostic approach is handled for the wrist camera inputs. Please clarify the methodology for handling the embodiment differences on the end effector as seen from the wrist camera views. Clarifying the methodology for wrist cameras would address this concern as long as the methodology preserves the robot-agnostic property. Also, even if the head camera is moved, the wrist cameras could help compensate for most of that viewpoint gap; in other words, a model that learns to ignore the head view altogether might have better cross-embodiment transfer already.

W2: Table 3 is only ablate the method on an unseen embodiment and not on a source embodiment (across multiple viewpoints and original viewpoint). Performing this ablation on a source embodiment is an interesting addition because it would help isolate that the refiner embodiment mismatch issue that the authors claim causes the refiner to not help without the agnostic future. I’m curious if the refiner helps on a source embodiment since that could make the refiner module a useful component even in non cross-embodiment settings.

Questions for authors

Q1: Typically in WAM models the future video and action are jointly decoded, which helps ensure consistency between the video and actions. In FastWAM, only actions are decoded at inference time, meaning the consistency with future video is not applicable. In CrossWAM the action denoising happens independently from the future video denoising. Thus, it might happen that the predicted action is not consistent with the predicted video future (e.g., due to multimodality in the output space). In this case the refiner may correct actions against an imagined future that is inconsistent with the actions the robot has been executing. Is there a way to validate and mitigate this issue?

Q2: Since the refiner is itself based on a Wan2.2 model (i.e., large parameter count), how does the latency of that model impact robot execution? A diagram that highlights how many refiner calls occur per predicted action chunk and at what intervals and latency would help clarify how frequent these corrections are in a practical execution setting.

Q3: Why is the performance of Franka much worse than ARX arm in Table 1? This questions the authors’ claims that the task is independent of the robot that executes it. Right now it’s hard to know if CrossWAM itself doesn’t perform well on Franka because of the embodiment or because the embodiment-gap is not fully addressed. I’m specifically curious about how much of the in-domain embodiment performance is recovered through CrossWAM since that would inform how much of the embodiment gap is addressed by this method.

Minor notes:

N1: Please clarify in the text that Table 3 corresponds to the simulation experiment and not the real-world experiment. This change would help clarify that the ARX is in fact an unseen embodiment and not a source embodiment, which is necessary for the ablation.

N2: Only 10 rollouts per setting in Table 2 is quite low, which reduces the statistical significance of the findings in the real world. For example the highlighted jump from 30% to 40% only is attributed to one rollout difference.

6 comments
AuthorsOpenReview ↗

Response to Reviewer ncuw (1/3)

We thank the reviewer for the positive assessment and for recognizing that using the imagined future at test time offers a new perspective for the WAM and cross-embodiment communities; we address each point below.

A1. Translation-only correction. (W1(a))

We correct only the translation for two reasons. First, an embodiment or camera change mainly corrupts the mapping from pixels to positions, which displaces where an action lands much more than how the gripper is oriented on arrival; consistent with this, all real-robot failure modes in App. C come from a spatial offset at contact. Second, a gripper correction carries more risk. Because the offset is applied to all subsequent actions of the chunk, a close or open command issued at the wrong moment would affect the rest of the chunk and can make the grasp fail, whereas a translation error can still be corrected at the next refiner call.

We also add the ablation: a refiner that predicts both a translation and a rotation offset, trained on the same pairs (ARX, randomized scenes). Adding the rotation correction lowers success under all three cameras, by 0.4 / 3.8 / 3.6 points. The trace is drawn as a line connecting the gripper keypoints, which shows where the gripper is but only partly how it is oriented, so predicting translation and rotation from it at the same time is ambiguous。

Refiner outputARX DRL
Translation (CrossWAM)71.260.756.7
Translation + rotation70.856.953.1

A2. Complex embodiments and contact modes. (W1(b))

We agree and have expanded the limitations in the revision (App. A): "The embodiment-agnostic target assumes a parallel gripper. For dexterous hands, the trace would have to encode finger keypoints and contact patterns, which we have not tested. The premise that object motion is independent of the executing robot is also weaker for non-prehensile manipulation, where the shape of the end-effector determines the contact."

A3. Wrist cameras. (W1(c))

We apply no special processing to the wrist views: they are neither inpainted nor overlaid with the trace. This is deliberate. In a wrist view, the robot appears mostly as its own gripper fingers, and the contact between the fingers and the object is information the policy relies on for grasping and wiping; masking or inpainting the fingers would discard it. The embodiment-agnostic construction therefore covers the head view. Handling different end-effectors is a separate problem and not the focus of this paper; Cloak (Piseno et al., 2026) addresses it by masking the end-effector in the wrist view, and combining it with CrossWAM is possible future work. The revision describes the wrist-camera handling in Sec. 4.1.

On your second point, the wrist views do not remove the embodiment gap either. In RoboTwin 2.0, the wrist cameras of Aloha-AgileX, ARX and Franka are mounted at different positions relative to the end-effector, and the grippers differ in shape and appearance; on the real robots we also use different camera models (fisheye cameras on ARX, Dabai cameras on Piper). We also trained Fast-WAM on RoboTwin 2.0 with the wrist cameras only. It reaches only 27.5%/27.0% on the clean and random setting of Aloha-AgileX with default camera, which shows that the wrist views alone are not sufficient for the RoboTwin tasks, which need the global context of the head view.

AuthorsOpenReview ↗

Response to Reviewer ncuw (2/3)

A4. Ablation on the source embodiment. (W2)

This is a clean test of our explanation. On Aloha-AgileX, the imagined future and the observation show the same robot even without the agnostic target. If "+Refiner" hurts on ARX because of the robot-appearance mismatch, it should not hurt on Aloha-AgileX. We extend the ablation of Table 3 to Aloha-AgileX and Franka (randomized scenes):

VariantAloha DRLARX DRLFranka DRL
Fast-WAM91.857.254.053.929.233.222.414.613.7
Fast-WAM (Pretrained)91.768.861.661.936.442.025.917.016.2
+ Agnostic future92.070.363.269.344.048.530.421.120.4
+ Refiner91.174.365.958.731.539.923.715.016.7
+ Agnostic future + Refiner91.680.872.371.060.956.432.826.928.4
+ Async (full CrossWAM)91.581.172.071.260.756.733.026.428.5

The results confirm this explanation. On the source Aloha-AgileX, where the imagined future and the observation contain the same robot, the refiner helps even without the agnostic target. On the unseen ARX and Franka, the same variant lowers success in five of the six columns, because the dominant difference between its two inputs is then the appearance of the robot body. With the agnostic target, the refiner helps on all three embodiments under shifted cameras. The agnostic target itself mainly helps on the unseen embodiments and only marginally on Aloha-AgileX, since there is no appearance gap on the source embodiment. At the default camera of the source embodiment there is little to correct, and the refiner changes success by less than one point. Asynchronous denoising leaves success within 0.5 points in every column. The real robot gives further evidence: on the source ARX with the raised camera, adding the refiner raises success from 60 to 90 on Pick-and-Place and from 40 to 50 on Whiteboard Wiping (Table 2), which supports your point that the refiner can help outside cross-embodiment settings.

A5. Consistency between predicted actions and video. (Q1)

We agree that this can happen. One remedy is to decode video and actions jointly, so that the actions attend to the predicted video, but this increases the latency of the action output. Instead, we filter the refiner output: when the predicted offset is abnormally large (above 10cm). This can discard some valid corrections, but it improves results in our experiments.

A6. Refiner latency and call frequency. (Q2)

The refiner is called once per imagined frame, i.e., every 4 predicted actions; on the real robot, where actions are executed after 2× temporal interpolation (App. C), this corresponds to every 8 executed actions. It shares one RTX 4090 with video generation and takes about 83 ms per call, because it uses only the head-camera view (no wrist views) and only 4 denoising steps. As stated in App. C, it runs asynchronously: execution continues while it computes, which delays the affected actions by only 3 steps. At our control rate of 20 Hz, the 8 actions between two calls take 400 ms, so each call finishes well before the next one is due. The revision adds the refiner latency to App. C.

We also vary how often the refiner is called (Aloha-AgileX, randomized scenes; App. D.1 and Table 8 of the revision). An interval of 4 actions, i.e., one call per imagined frame, is the smallest possible, since the imagined video has one frame per 4 actions.

Refinement intervalRaisedLowered
Every 12 actions77.067.5
Every 8 actions80.271.4
Every 4 actions (default)81.172.0

More frequent refinement is better: calling the refiner every 4 actions instead of every 12 raises success by 4.1 / 4.5 points under the raised / lowered cameras.

AuthorsOpenReview ↗

Response to Reviewer ncuw (3/3)

A7. Why is Franka much lower? (Q3)

Part of the gap comes from Franka's appearance and kinematics. In the same grasping tasks, the Franka arm occludes a large part of the workspace from the head camera because of its configuration (see, e.g., the official RoboTwin 2.0 task video at https://robotwin-platform.github.io/doc/tasks/blocks_ranking_size.html), which limits what the model can infer from the image, even though the same manipulation strategy remains feasible. The in-domain reference in Table 6 separates the two effects. Fast-WAM-ID starts from the same pretrained backbone as CrossWAM and is trained and tested on the target's own data. On Franka it reaches only 50.9 (clean) / 46.7 (randomized) at the default camera, against 81.9 / 77.8 on ARX, so much of the Franka deficit is not caused by transfer.

N1. Fixed: the Table 3 caption now states that the ablation is run in simulation (RoboTwin 2.0, randomized scenes).

N2. We agree that 10 trials per condition is few and that a one-trial difference such as 30 → 40 is not significant on its own. Taken together, however, the results are consistent: in every shifted condition, CrossWAM has the highest progress on both tasks and the highest or tied-highest success, and its average success is the highest on both tasks. We also report task progress, a finer-grained metric that follows the same trend.

Reviewer_KKedOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies zero-shot transfer of a world-action model to an unseen robot and an unseen camera pose at the same time. CrossWAM makes three changes to a Fast-WAM style model. First, the source arm is masked and inpainted out of the training video targets, and the end-effector is drawn back in as a colored gripper trace. Second, at test time a separate refiner compares an imagined frame with the live frame, which carries the same trace drawn from the target URDF and the calibrated camera, and adds a translation offset to the actions not yet executed. Third, a diffusion-forcing style schedule denoises the near video latent faster than the far one, roughly halving the time to first video. On RoboTwin 2.0, with two unseen arms and a vertically shifted camera, CrossWAM beats the VLA and WAM baselines in every column except the source robot at the default camera, and in randomized scenes its average over the unseen-robot columns is 17.6 points above the best baseline. An ablation on one unseen arm and a small real-robot study support the trend.

Critical strengths
  1. The strongest idea here is the imagination-observation refiner, and the ablation in Table 3 backs it up. Instead of executing a chunk open loop, CrossWAM keeps the WAM's imagined future as a reference and corrects the actions not yet executed from the gap between that reference and what the camera sees. On ARX the refiner adds up to 16.9 points under the camera shifts on top of the embodiment-agnostic target, and its corrections grow as the camera moves. Without the agnostic target it lowers success, which supports the point that imagination and observation have to be expressed in the same terms.

  2. The embodiment-agnostic video target is a cheap change that helps on its own. Against Fast-WAM initialized from the same backbone, it adds a steady gain of several points on ARX at all three camera heights. It only changes what the video branch is trained to predict, so other WAMs could pick it up easily.

Critical weaknesses
  1. The evidence seems lacking, and the closest baselines are missing/not fair. The only component ablation on success rate in simulation covers ARX alone. The real-robot study has 10 trials per condition, and pooled over conditions the gap to the matched baseline is not significant on either task. The most direct competitor, LAP (RSS 2026), is cited as enabling zero-shot cross-embodiment transfer but is not compared, even though, as far as I can tell, it needs no camera calibration. Mirage is compared only in an appendix, on Fast-WAM without the large-scale pretraining, and it seems to have been run without its depth reprojection for camera changes. Shadow (CoRL 2024), which also masks the robot to transfer policies across arms, is not cited. This matters because CrossWAM's refiner uses the target URDF and the calibrated deployment camera, and I could not find any baseline that is given the new camera pose.

  2. The scope is narrow and the presentation is rough. All simulation results come from one benchmark, RoboTwin 2.0, with two unseen arms and a camera that only moves vertically; the real study adds one arm and two tasks, and the only embodiment-specific reference is an in-domain model at the default camera. The writing also needs another pass. The intro and the Figure 2 caption say asynchronous denoising emits near-future actions early, but the method section applies it to video latents. The refiner's equation pairs one imagined frame with each action step, while the model predicts one frame per four actions, and the refiner's training target has the opposite sign to how it is used, which may just be wording. Symbols are reused, some captions are wrong, and RoboTwin 2.0 itself is never cited. I like the refiner idea, and I am willing to raise my score if the rebuttal resolves these points, in particular a comparison with LAP or a clear reason why one isn't possible, Mirage with its camera reprojection on the same backbone, and seeds or confidence intervals for the main tables.

Questions for authors
  1. (Clarification) For each baseline in Tables 1 and 4, which checkpoint, training data and action space did you use, how many actions were executed per chunk, and how were its actions mapped to ARX and Franka? The Aloha default cells for Fast-WAM, LingBot-VA and X-WAM match the numbers in the Fast-WAM and X-WAM papers to one decimal place, even though the appendix says Fast-WAM was retrained with end-effector actions, so were these cells measured with the same checkpoints as the shifted cells? And why does LingBot-VA drop so far on Franka?
  2. (Suggestion) Please compare with LAP, or explain why that is not possible. Post-training LAP on the same Aloha-AgileX data and testing on ARX and Franka would show whether CrossWAM's extra test-time information, the URDF and the calibrated camera, is needed.
  3. (Suggestion) Please add baselines that get the same test-time information on the same pretrained backbone: Mirage with its depth reprojection, since RoboTwin 2.0 already provides point clouds, and the WAM fed the trace-overlaid observation as input. A sweep over extrinsic calibration error would show how much the refiner depends on accurate calibration, which matters for the real-robot claim.
  4. (Clarification) How often is the refiner called within a chunk, and do successive offsets accumulate, given that the offset is added to the original action? At training time, is the "imagined" input a ground-truth inpainted frame or a WAM sample, and how are the wrist views handled? Does the gain survive if the refiner sees only the predicted gripper trace, with the imagined object and background pixels removed, which would show whether it needs the imagined world state or mostly tracks an end-effector reference?
  5. (Suggestion) The refiner is trained only under the source camera, so its own pixel-to-world mapping should also be off once the camera moves. Can you show in simulation that the offset points the right way under raised and lowered cameras, for example by reporting the end-effector-to-target distance at contact with and without the refiner?
  6. (Suggestion) Please report seeds or confidence intervals for Tables 1 and 3, say how many episodes per task and per cell were run, and extend the ablation in Table 3 to Franka and to the Aloha camera shifts.
  7. (Suggestion) Why only RoboTwin? A second testbed with strong embodiment-specific references, such as the simulated DROID environments or MolmoSpaces-Bench, would make the transfer claim much stronger, and camera changes beyond vertical translation would help the viewpoint claim.
  8. (Suggestion) Please position the work against RoVi-Aug (CoRL 2024), which already reports zero-shot deployment on an unseen robot with different camera angles; Shadow (CoRL 2024), which masks the robot to transfer across arms; CLOVER (NeurIPS 2024), which corrects actions from the error between generated visual plans and observations; and Know Thyself (ICLR 2022), which already separates transferable world dynamics from robot-specific dynamics.
7 comments
AuthorsOpenReview ↗

Response to Reviewer KKed (1/5)

A1. Comparison with Mirage and Dex-BEV on the same backbone and LAP. (W1, W2, Q2, Q3)

Thank you for the suggestion. We added LAP and re-implemented Mirage and Dex-BEV on the same pretrained backbone as CrossWAM; Table 5 of the revision now reports Mirage (without depth) and Dex-BEV on this backbone. LAP is post-trained from its released checkpoint on the same Aloha-AgileX data, with bimanual actions written as per-arm language clauses. Mirage is evaluated with and without its depth-based camera reprojection, which maps each image back to the training view before cross-painting.

Mirage and Dex-BEV. Building on the pretrained backbone raises the results of Mirage and Dex-BEV. However, CrossWAM still outperforms both by a clear margin on the unseen embodiments and under camera shifts, in both clean and randomized scenes.

Reprojection with depth. Depth-based reprojection is very effective when the depth is exact. With ground-truth depth, Mirage gains 13.8–20.1 points on Aloha-AgileX and ARX, and it exceeds CrossWAM without depth. Given the same ground-truth depth, however, CrossWAM outperforms Mirage with ground-truth depth, so reprojection is complementary to our method.

Ground-truth depth is privileged information: it is rendered by the simulator, so it is noise-free and complete. Depth from real sensors is far less reliable. It has large holes on reflective, transparent, thin and dark surfaces and at object boundaries, and it degrades further while the arm is moving. Mirage itself does not use depth in its real-robot experiments, citing noise and imprecision. To approximate a realistic setting, we replace the ground-truth depth with depth estimated from the RGB image by Depth Anything V2. Mirage still improves over its no-depth variant, but it remains below CrossWAM in every shifted-camera cell. Depth estimation, reprojection and inpainting also take about 130 ms per control step, which adds directly to the inference latency. CrossWAM uses only RGB images at test time.

LAP. LAP transfers poorly in our setting; the revision adds it to Tables 1 and 4. Although LAP describes actions in natural language, which provides robot-agnostic supervision for the VLM, the robot is controlled by the output of its action expert, and in post-training this expert only sees VLM features of images from a single embodiment under a single camera. CrossWAM instead trains the video branch to predict inpainted, robot-free futures, so its features carry less embodiment appearance (Sec. 4.4), and the refiner corrects the error a camera change introduces at test time.

Table R1. All rows except Fast-WAM and LAP use our pretrained backbone. D/R/L = default/raised/lowered camera. Rows marked "w/ GT depth" use ground-truth simulator depth, and "w/ DepthAnything" uses depth estimated from the RGB image by Depth Anything V2.

MethodAloha DRLARX DRLFranka DRL
Clean scenes
Fast-WAM91.957.851.157.231.734.225.517.416.2
Fast-WAM (Pretrained)92.168.359.564.243.441.127.220.718.9
+ Mirage (w/o depth)92.168.359.570.349.645.730.823.520.7
+ Mirage (w/ GT depth)92.187.880.570.368.765.230.828.628.9
+ Mirage (w/ DepthAnything)92.171.764.070.351.749.230.824.923.5
+ Dex-BEV91.371.565.263.349.947.227.322.823.1
LAP84.450.845.258.238.637.811.69.78.0
CrossWAM92.081.468.373.462.253.732.526.928.1
CrossWAM (w/ GT depth)92.087.081.173.472.370.632.530.431.9
Randomized scenes
Fast-WAM91.857.254.053.929.233.222.414.613.7
Fast-WAM (Pretrained)91.768.861.661.936.442.025.917.016.2
+ Mirage (w/o depth)91.768.861.665.240.747.927.119.019.8
+ Mirage (w/ GT depth)91.784.981.765.260.561.727.125.726.0
+ Mirage (w/ DepthAnything)91.773.866.565.252.551.927.122.122.7
+ Dex-BEV90.974.264.961.147.249.725.720.821.5
LAP80.547.146.153.833.336.111.58.48.5
CrossWAM91.581.172.071.260.756.733.026.428.5
CrossWAM (w/ GT depth)91.586.283.171.267.865.033.030.131.7
AuthorsOpenReview ↗

Response to Reviewer KKed (2/5)

A2. Baseline that is given the camera pose. (W1,Q3)

Mirage, the cross-embodiment baseline in Table 5, already uses the target URDF and the calibrated deployment camera to mask the target robot and render the source robot in its place. Its reprojection variant additionally uses depth, which is more information than CrossWAM receives, and Dex-BEV also takes the camera pose as input.

A3. Writing improvements. (W2)

Thank you for the careful reading. All changes below are shown in blue in the revised PDF.

(1) Asynchronous denoising acts on the video latents, so the introduction and the Fig. 2 caption now say that it emits near-future frames early, instead of near-future actions.

(2) Following existing WAMs such as Fast-WAM, CrossWAM predicts a 32-step action chunk together with a video that is temporally down-sampled by 4, i.e., 8 frames. Sec. 3.2 now defines the down-sampling factor nn (n=4n=4, H=32H=32), and Eqs. (5) and (6) in Sec. 3.3 are rewritten accordingly: the jj-th imagined frame I^t+j′\widehat{I}^{\prime}_{t+j} is compared with the observation ot+n⋅jo_{t+n\cdot j} taken after the first n⋅jn\cdot j actions, and the resulting offset corrects the remaining actions τ=t+n⋅j,…,t+H−1\tau=t+n\cdot j,\ldots,t+H-1. Sec. 3.3 and Sec. 3.5 also state that the refiner is called, and a new observation is taken, once every nn actions.

(3) You are right about the sign. The refiner is trained to predict the translation from the imagined state to the observed one, so the offset must be subtracted from the remaining actions. We changed the "+" in Eq. (6) to "−" and updated the text in Sec. 3.3, Sec. 3.5 and Sec. 4.4 accordingly.

(4) RoboTwin 2.0 (Chen et al., 2025) is now cited in Sec. 4.1 and Sec. 4.2.

A4. Clarification for training details in Table1 and Table4. (Q1)

π0.5 has no official RoboTwin 2.0 checkpoint, so we trained it ourselves. The released checkpoints of StarVLA-α and Fast-WAM use joint-angle actions, which cannot be transferred directly to ARX and Franka, so we retrained both with end-effector actions and kept all other settings of their official code. All methods are trained on the same official RoboTwin 2.0 data, 550 trajectories for each of the 50 tasks. Every method acts in end-effector space, and its actions are executed on each robot through the robot-specific IK solver provided by the simulator (App. B.2 of the revision). Table below lists the checkpoint of each baseline and the number of actions it executes per chunk.

In Tables 1 and 4, the baselines' Aloha-AgileX default-camera cells are previously published numbers, which we reported for consistency with prior work. For StarVLA-α and Fast-WAM, these numbers come from the official joint-space models. Our end-effector variants reach 91.6 / 91.2 (Fast-WAM) and 87.9 / 87.7 (StarVLA-α) in clean / randomized scenes, slightly below the published 91.9 / 91.8 and 88.2 / 88.3. And the other eight columns of each row are measured with our retrained end-effector checkpoints. The small gap comes from the change of action space. The revision states the source of these cells and our reproduced numbers in Sec. 4.2.

In the submitted Table 5, Fast-WAM, Mirage and Dex-BEV shared the training data, action space and action chunk length described above, but were trained without our pretraining. The revised Table 5 instead builds Mirage and Dex-BEV on the same pretrained backbone as CrossWAM, which gives a fairer comparison (see A1).

MethodCheckpointActions executed per chunk
π0.5retrained24/30
RotVLAofficial RoboTwin weights24/30
StarVLA-αretrained50/50
Hy-VLAofficial RoboTwin weights7/20
Fast-WAMretrained24/32
LingBot-VAofficial RoboTwin weights32/32
X-WAMofficial RoboTwin weights32/32
CrossWAMours24/32

A5. Why does LingBot-VA drop on Franka? (Q1)

LingBot-VA decodes actions conditioned on its predicted video tokens. Franka differs most from the source Aloha-AgileX in appearance and occludes a large part of the workspace, so LingBot-VA's video prediction degrades on Franka, and the actions decoded from it degrade with it. On ARX, whose appearance is closer to the source, the video prediction remains usable and LingBot-VA is the strongest baseline at the default camera.

AuthorsOpenReview ↗

Response to Reviewer KKed (3/5)

A6. Calibration error. (Q3)

To measure how much CrossWAM depends on accurate calibration, we perturb the camera extrinsics used to draw the gripper trace on the observation, with a fixed offset of 1 cm in translation and 1° in rotation in a random direction for each episode (ARX, raised and lowered cameras, randomized scenes). This perturbation is larger than the error of typical calibration: standard hand-eye calibration reaches about 1 mm and 0.3° (Li et al., 2023 (https://arxiv.org/pdf/2311.01335)).

SettingARX RARX L
Fast-WAM (Pretrained)36.442.0
CrossWAM, exact calibration60.756.7
CrossWAM, extrinsic error 1 cm / 1°56.152.9

With this error, CrossWAM drops by 4.6 / 3.8 points under the raised / lowered cameras and remains 19.7 / 10.9 points above Fast-WAM (Pretrained). A calibration error shifts the trace drawn on the observation, and this error therefore degrades the closed loop only slightly rather than breaking it. Moreover, for a robot with a fixed setup, the camera needs to be calibrated only once.

A7. Trace input for WAM and Refiner. (Q3, Q4)

Trace input for the WAM (Q3). To give a baseline exactly the test-time information of CrossWAM, we train and test Fast-WAM (Pretrained) with the same calibrated gripper trace drawn on its input observation, without the embodiment-agnostic future target and without the refiner.

Trace input for the refiner (Q4). The trace-input WAM above still predicts a raw future, and its predicted trace cannot be extracted directly from the generated images, so it cannot be paired with the refiner. To test whether the refiner needs the imagined world state or mostly tracks an end-effector reference, we instead train a WAM whose video target contains only the gripper trace on a blank background, and evaluate it with and without the refiner.

Method (ARX, randomized scenes)DRL
Fast-WAM (Pretrained)61.936.442.0
+ Trace input for WAM61.837.142.3
CrossWAM71.260.756.7
CrossWAM (trace on blank background, w/o refiner)54.933.734.7
CrossWAM (trace on blank background, w/ refiner)63.747.245.7

Drawing the trace on the WAM input changes success by less than one point, so the extra test-time information alone does not explain CrossWAM's improvement. The trace-only WAM is worse than Fast-WAM (Pretrained) without the refiner, because most of its video target is blank, so the video branch learns little about the scene and its representation degrades. Adding the refiner raises it by 8.8 / 13.5 / 11.0 points, so the end-effector reference alone already gives a useful feedback signal. It remains 7.5 / 13.5 / 11.0 points below CrossWAM, which shows that the refiner needs both the end-effector position and the imagined world state.

A8. Refiner Call frequency and accumulation. (Q4)

The refiner is called once per imagined frame. The WAM predicts one frame per 4 actions, so the refiner runs every 4 actions. Each call sees the observation produced by the actions corrected so far, and its offset is subtracted from the current remaining actions. Offsets therefore accumulate, as in integral control: once execution matches the imagination, δ≈0\delta \approx 0 and the accumulated correction is kept. Eq. (6) did not make this clear; the revision writes the update explicitly. We have clarified this in Sec. 3.3 of the revised version.

We also vary how often the refiner is called (Aloha-AgileX, randomized scenes; App. D.1 and Table 8 of the revision). An interval of 4 actions, i.e., one call per imagined frame, is the smallest possible, since the imagined video has one frame per 4 actions.

Refinement intervalRaisedLowered
Every 12 actions77.067.5
Every 8 actions80.271.4
Every 4 actions (default)81.172.0

More frequent refinement is better: calling the refiner every 4 actions instead of every 12 raises success by 4.1 / 4.5 points under the raised / lowered cameras.

A9. Training input and wrist views. (Q4)

During refiner training, the "imagined" input is frame built from the source video, not a WAM sample. For the wrist views, we use the raw camera images without any additional processing. We add these details in the revised version in Sec.4.1.

Comments on the paper

2 comments
AuthorsOpenReview ↗

General Response and Revision Summary

We thank all reviewers for their constructive feedback. Reviewers found that using the imagined future at test time for closed-loop correction offers a new perspective for WAMs (ncuw, KKed, TNu4), and that the embodiment-agnostic target is cheap and effective (KKed). We answer the main concerns with new experiments below; all changes in the revised PDF are shown in blue.

  • More baselines (KKed). Mirage and Dex-BEV are rebuilt on our pretrained backbone (Table 5) and LAP is added (Tables 1 and 4); CrossWAM leads all of them under embodiment and camera shifts.
  • Refiner clarifications (KKed, ncuw, TNu4). The refiner is called once per imagined frame (every 4 actions), its offsets accumulate over the chunk, it is trained on ground-truth agnostic frames, and it takes about 83 ms per call (Secs. 3.3 and 4.1, App. C).
  • Writing improvements (KKed). We fixed the frame-to-action alignment and the sign in Eqs. (5)–(6), clarified that asynchronous denoising emits near-future frames early, added the baseline details (Sec. 4.2, App. B.2), and discuss CLOVER, Know Thyself and Shadow in the related work.
  • More discussion of limitations (ncuw, TNu4). App. A now covers dexterous hands, non-prehensile contact and large viewpoint changes.
AuthorsOpenReview ↗

Summary of the discussion

Dear Area Chair,

As the discussion period is nearing its end, we would like to provide a brief summary of the author-reviewer discussion.

All three reviewers have responded to our rebuttal, and all three now recommend acceptance:

  • Reviewer KKed (2 → 3). Stated that "the LAP comparison, matched-backbone Mirage results with reprojection, expanded ablations, and additional seeds substantially address my main concerns" and that "the new results provide sufficient support for the proposed approach." The remaining point, that the trace-only ablation also changes the WAM's training, was addressed in our follow-up with a control that keeps the WAM fixed.
  • Reviewer ncuw (3, kept). Stated that "most of my concerns have been addressed" after the source-embodiment and rotation-output ablations. The two suggested discussions (the discrepancy between predicted video and actions, and why the wrist views are not edited) have been added to the revised PDF (App. A and App. B.1).
  • Reviewer TNu4 (3, kept). Thanked us for the response and additional experiments and kept the positive score.

Key new evidence provided during the rebuttal:

  • Stronger baselines on the same pretrained backbone. Under the raised camera, CrossWAM reaches 60.7%, against 40.7% for Mirage and 47.2% for Dex-BEV (Table 5); LAP is added to Tables 1 and 4 and transfers worse than CrossWAM on every unseen-embodiment cell.
  • Depth-based reprojection. Given the same ground-truth depth, CrossWAM stays ahead of Mirage (67.8% vs. 60.5%, raised camera); with depth estimated by Depth Anything V2, Mirage reaches 52.5% and adds about 130 ms per step.
  • Correction direction. On three tasks, the refiner reduces the end-effector-to-target distance at contact by 31–85% under camera shifts, to about 1 cm or less; adding a rotation output lowers success (70.8 / 56.9 / 53.1% vs. 71.2 / 60.7 / 56.7%).
  • Robustness. Over three evaluation runs, the standard deviation is at most 0.8 points per cell; a 1 cm / 1° calibration error, larger than typical calibration error, lowers success by less than 5 points.
  • Refiner invocation. The revision specifies how the refiner is called: once per imagined frame, i.e., every 4 actions (every 8 executed actions on the real robot after 2× interpolation); its offsets are subtracted from the remaining actions and accumulate over the chunk; and it runs asynchronously in about 83 ms per call.
  • Wrist cameras. App. B.1 now explains that the wrist views are used unedited because they mainly show the gripper fingers, whose contact with the object the policy relies on; masking end-effectors in the wrist view, as in Cloak, is complementary.
  • Extended limitations. App. A now discusses dexterous hands, non-prehensile contact, the range of viewpoint changes tested, and the possible inconsistency between the predicted video and actions.

We believe the discussion has been productive and that the reviewers' main concerns have been satisfactorily resolved. We thank the Area Chair for overseeing this process.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.