acceptodds
Under review as a conference paper at ICLR 2027

Emergent Depth-Wise Specialization in a Unified Embodied Model: From Semantic Understanding through Visual Foresight to Action

Abstract

Vision-language-action (VLA) policies emphasize semantic understanding, whereas world-action models (WAMs) learn predictive representations of environment dynamics. Systems that expose a policy to both sources often still concentrate action computation on a single expert. We present EWAM, an action-centric unified embodied model whose asymmetric joint attention lets action tokens read semantic, current-visual, predicted-future, and action information at every layer while the perceptual experts retain their distinct roles. Without layer-wise supervision, EWAM develops an emergent depth-wise specialization: action queries attend mainly to vision-language features in shallow layers, to predicted future frames in intermediate layers, and to action tokens themselves in deep layers. This handoff replicates across tasks and is stable across denoising steps. Checkpoint tracking and causal interventions show that it is learned and that action generation depends on it. EWAM is pretrained in two separate regimes, one on cross-embodiment robot trajectories and one on human egocentric video. In simulation and real-robot experiments, it surpasses existing VLA, WAM, and hybrid baselines. Human egocentric data improve both cross-embodiment transfer and real-robot robustness, and subtask-phase supervision improves long-horizon completion. Together, these results suggest that unified embodied learning can induce an ordered internal progression from semantic understanding, through visual foresight, to action formation. Code will be made publicly available.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Embodied AI, Unified Model, Vision‑Language‑Action, World‑Action‑Model

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_VEeMOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper proposes EWAM, a unified world-action model that aligns a vision-language expert (Qwen3-VL-2B), a video generation expert (Wan2.2-TI2V-5B), and an action expert across layers under an asymmetric attention mask. By analyzing action-query attention across depth, the authors argue that the model develops an emergent three-stage specialization—shifting from VL tokens in shallow layers to predicted-future tokens in middle layers and action self-attention in deep layers—and report strong results on RoboTwin 2.0, LIBERO, and real-robot tasks.

Critical strengths
  1. I appreciated the care the authors took in measuring attention routing. They normalize attention by token count, report bootstrap confidence intervals across 50 tasks, and track how routing changes during training. The targeted KV injections and the ablations on attention masks and layer-wise access made the analysis more useful to me because they let me compare the observed routing patterns with what happens when the policy’s access to each stream changes.
  2. EWAM scores 72.1% on RoboTwin 2.0 clean-to-random (C2R), well above the previously reported results. I give this result weight because many world-action models struggle when they move from clean training layouts to randomized test layouts.
Critical weaknesses
  1. My main reservation is about the paper’s central claim that depth-wise specialization emerges during joint training and is critical to policy performance. In Table 1, switching from the asymmetric mask to symmetric full attention brings the score from 83 to 81, even though the bidirectional variant in Appendix B.6 largely loses the middle-layer handoff from image to future tokens. Removing shallow-layer VL access or middle-layer video access costs 5 points in each case. Restricting the deep layers to action self-attention leaves the score unchanged. Raw attention mass is also hard to interpret as a measure of functional importance. VL tokens receive 75.4% of shallow-layer attention, and masking them costs 16 points. Video tokens receive just 15% of deep-layer attention, and masking them costs 27 points. Given the layer-by-layer alignment with pretrained Qwen3-VL and Wan2.2, I suspect EWAM inherits much of the depth profile from those backbones.
  2. I find it hard to tell how much of the 72.1% C2R result comes from the architecture. EWAM is pretrained on roughly 1,800 hours of curated robot data, and the paper gives no C2R result without that pretraining. I have a similar concern about the human-video experiments. The cross-embodiment comparison uses training from scratch as its only baseline, so it leaves open how human egocentric pretraining compares with robot pretraining. In Table 20, training on 90 robot-only episodes reaches 70%, compared with 60% for the best human-robot co-training setup.
Questions for authors
  1. For the Table 1 variants that score 83, 81, and 78, how much do the results vary across seeds? If you ran multiple seeds, could you report confidence intervals? Separately, does the three-stage depth profile persist under a different backbone alignment, such as feeding the final Qwen3-VL layer into every block?
  2. Given the mismatch between raw attention mass and the masking results, do you see the same routing pattern with value-weighted attention or gradient attribution? How does masking future keys in the middle layers compare with masking the same number of randomly selected keys?
  3. What C2R score does EWAM reach on RoboTwin 2.0 without the 1,800 hours of robot pretraining? On the held-out Aloha-Agilex-2 embodiment, how does human egocentric pretraining compare with robot pretraining?
5 comments
AuthorsOpenReview ↗

Response to Reviewer VEeM (1/2)

We thank Reviewer VEeM for the careful review and for finding the routing analysis well beyond standard attention heatmaps and the clean-to-random generalization strong. In the revised paper, revisions specific to your comments are marked in green, and revisions addressing points raised by both you and other reviewers are marked in orange. Replaced wording is struck through.

W1: Magnitude of the Table 1 ablations

  • Table 1 (left) is trained from scratch; the bidirectional variant itself retains the coarse three-stage profile (App. B.6, p. 24), so a small gap is expected. What it loses is the image-to-future handoff.
  • Removing shallow VL or intermediate video access costs 5 points each over 5,000 episodes, well beyond evaluation noise; deep action-only layers cost nothing, which is exactly what "deep layers focus on action" predicts.
  • [New experiment] A stronger structural intervention, removing future access at all layers, costs 14 points (83 → 69; Table 1, p. 8).

W1 & Q1: Inheritance from layer-aligned backbones, and multi-seed intervals

  • At 50 training steps the same layer-aligned backbones are already in place, yet no depth-wise division is visible; it appears only during training. We have made this explicit in Sec. 4.2 (p. 7) and Fig. 4 (p. 7). An inherited profile would be present from the start.
  • [New experiment] As suggested, we also broke the layer alignment: every action block reads keys and values projected from the final Qwen3-VL layer, with parameters, data, and budget unchanged.

Action-attention mass by stage, final-layer variant [New] (4 GPUs, same data and recipe as Table 1 left):

Training stepShallow (0–9): VLMiddle (10–19): future / condition frame / VL imageDeep (20–29): action
500.3580.297 / 0.180 / 0.2550.185
3000.5600.202 / 0.118 / 0.0370.569
1,0000.8150.204 / 0.095 / 0.0840.667

Although every block now reads identical final-layer VL features, the same three stages emerge during training: shallow layers read VL (0.815), intermediate layers favour predicted future frames over the current image (0.204 vs. 0.095 / 0.084), and deep layers focus on action (0.667), with no division at 50 steps. The routing is therefore learned in joint training, as claimed in Sec. 4.2 (p. 7), not inherited from layer-by-layer alignment. With the full training budget of Table 1 (left), this variant reaches 79%, compared with 83% for the layer-aligned model. We added these results to Table 1 (p. 8), Secs. 4.2 and 4.3 (p. 7), and Appendix B.7 (p. 27).

  • [New experiment] As requested, we retrained the four Table 1 variants with three seeds each, keeping data and budget unchanged:
Table 1 variantMean ± std (3 seeds)95% CI
Full EWAM83.2 ± 0.4[82.2, 84.2]
Symmetric full attention80.9 ± 0.4[80.0, 81.8]
No VL access, layers 0–977.9 ± 0.5[76.8, 79.1]
No video access, layers 10–1977.7 ± 0.5[76.4, 79.0]

The intervals of the three levels do not overlap, so the ordering 83 > 81 > 78 holds across seeds.

AuthorsOpenReview ↗

Response to Reviewer VEeM (2/2)

W1 & Q2: Attention mass versus masking impact

  • As stated in Sec. 4.2 (p. 6), attention mass shows where action tokens retrieve information, while Sec. 4.3 tests functional necessity. The paper does not use mass to predict the size of a masking drop. As noted in the Limitations (Appendix F, p. 44), neither measure alone provides a complete causal decomposition of the computation.
  • [New experiment] As suggested, we recomputed the stage profile with value-weighted attention on the same 50 RoboTwin tasks (one sequence each, robot-pretrained checkpoint):
StageRaw attention: VL / video / action (%)Value-weighted [New]: VL / video / action (%)
Shallow (0–9)75.4 / 14.1 / 10.659.6 / 22.7 / 17.7
Intermediate (10–19)36.6 / 24.3 / 39.127.2 / 33.2 / 39.5
Deep (20–29)25.2 / 15.3 / 59.519.8 / 22.4 / 57.8

The three stages hold. VL dominates the shallow stage and action the deep stage. In the intermediate stage, attention still shifts from the current image to predicted future frames: VL-image attention drops from 31.4% in the shallow stage to 7.9%, while future-frame attention rises from 14.4% to 22.4% and exceeds the VL image.

  • [New experiment] Random-key masking with an equal number of keys, layers 10–19:
Masked keys, layers 10–19 (robot-pretrained checkpoint)Success (%)
None92.80
Future keys3.55
Equal number of random keys [New]82.80

Removing the same number of random keys costs 10 points, whereas removing future keys collapses success to 3.55%; the collapse is therefore specific to predicted-future content, not to the number of masked keys. We added this row to Table 1 (p. 8) and Sec. 4.3 (p. 7).

W2 & Q3 (first part): C2R without robot pretraining

[New experiment] As requested, we trained EWAM for C2R without robot pretraining and compared it with published results of methods that also use no robot pretraining:

RoboTwin 2.0 C2R, no robot pretrainingC2R (%)
Fast-WAM1.90
starVLA3.16
EWAM [New]24.2

Without robot pretraining, EWAM still clearly outperforms starVLA and Fast-WAM trained under the same condition. Fast-WAM builds on the same Wan2.2 video backbone yet reaches only 1.90, so EWAM's advantage does not stem from the video prior or from pretraining scale alone.

W2 & Q3 (second part): Human versus robot pretraining, and G1-D co-training

  • Co-training results. We agree that 90 robot demonstrations outperform the best 60-robot/30-human mixture on G1-D pouring (70% vs. 60%). This gap shows that human demonstrations do not yet fully substitute for additional robot demonstrations in this setting. On the same 60 robot + 30 human mixture, however, human-pretrained initialization improves success from 40% to 60% compared with robot-pretrained initialization. In the robustness evaluation with tablecloth and cup changes, 30 egocentric episodes match 30 additional robot episodes (both 20%), and scaling to 1,000 raises success to 60% (Fig. 5(d), p. 10). Our goal is to reduce costly teleoperation, and these results show that cheap egocentric human data can partly replace robot demonstrations.

[New experiment] Human versus robot pretraining. We added robot-pretrained EWAM under the same protocol: training on ARX-X5, UR5, Franka, and Piper, then evaluating transfer to Aloha-Agilex-2 across 50 RoboTwin tasks.

[New table] Success rates (%) at downstream training steps:

EWAM initialization40K50K80K120K
No embodied pretraining33.537.240.136.2
Robot pretraining [New]50.860.461.655.3
Human pretraining44.353.458.966.9

Robot pretraining leads at 40K–80K, while human pretraining continues improving to 66.9% at 120K, an 11.6-point gain over robot pretraining. Comparing the best observed checkpoints also favors human pretraining: 66.9% vs. 61.6%, a 5.3-point gain. These results appear in Sec. 5.2 (p. 9), Fig. 5(a) (p. 10), and Appendix D.6 (Table 23, p. 40).

Dear Reviewer VEeM,

Please respond to the author soon and engage in the discussion. Please revise the review and avoid AI usage.

Best,

AC

Reviewer_SYKFOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces EWAM, a unified embodied model combining vision-language, video, and action experts through action-centric asymmetric attention. Action tokens can access all three streams, while the perceptual streams retain separate computation. The central finding is an emergent depth-wise progression: shallow layers emphasize semantic and current-image information, intermediate layers increasingly access predicted-future features, and deeper layers emphasize action self-attention. The authors investigate this pattern through attention analysis, checkpoint tracking, masking, and counterfactual feature injection. Experiments cover RoboTwin 2.0, LIBERO, and three physical robot platforms, with additional studies on human egocentric data and subtask-phase supervision.

Critical strengths
  • Substantial investigation of internal computation. The paper goes beyond reporting policy performance. Token-count normalization, cross-task analysis, training dynamics, and interventions provide complementary evidence that the observed routing pattern is learned and functionally relevant.

  • Broad empirical coverage. Evaluation includes two established simulation benchmarks and seven physical manipulation tasks across single-arm, dual-arm, and humanoid platforms. The RoboTwin clean-to-random result is particularly encouraging.

  • Coherent architectural design. The asymmetric connectivity provides a clear integration interface and makes the provenance of attended features easier to interpret. The human-data and subtask-supervision studies further demonstrate the framework’s practical flexibility.

Critical weaknesses
  • The source of the performance gains remains insufficiently isolated. EWAM benefits from substantial embodied pretraining and multiple pretrained experts, but the paper does not establish comparable data, capacity, and optimization budgets across baselines. The provenance and adaptation settings of the main baseline results also need clearer documentation. Consequently, the results demonstrate a competitive complete system more convincingly than the superiority of the proposed fusion mechanism.

  • The interventions support learned dependence more strongly than computational necessity. Masking intermediate future features severely damages an already trained policy, but this does not establish that iterative future prediction is necessary for strong performance. A model trained to use single-pass predictive features, as in Fast-WAM or DiT4DiT, may avoid this dependency. Likewise, attention routing and donor-feature transfer do not fully establish the proposed semantic-understanding–foresight–action interpretation.

  • Positioning against closely related work is incomplete. DiT4DiT and EgoScale deserve explicit discussion. The relationship to Qwen-RobotManip camera-relative action representation and cross-embodiment alignment should also be clarified. The current transfer experiments primarily establish the benefit of human pretraining within EWAM rather than an advantage over existing transfer methods.

  • Several improvements have limited statistical support. The LIBERO advantage is only 0.2 percentage points over several baselines, and physical evaluations use ten trials per task. Main performance comparisons and structural ablations lack sufficient reporting of training-seed variability. Long-horizon improvements are promising, but absolute completion remains low on the hardest tasks.

Questions for authors
  • Baseline comparability. Which results were reproduced, evaluated from released checkpoints, or taken from prior papers? Please specify downstream data, adaptation budgets, and checkpoint selection. How were potential overlaps between pretraining simulation data and evaluation tasks, embodiments, or randomization settings handled?

  • Necessity of future denoising. Under matched training conditions, how does EWAM compare with a variant retaining video supervision but using single-pass video features during action inference? Please report both success and latency.

  • Contribution of the proposed architecture. Can matched-data comparisons against VL–action and video–action variants separate fusion benefits from additional capacity and pretraining? Does enforcing or disrupting the proposed depth ordering affect performance after retraining?

  • Related works. Please clarify the distinctions from DiT4DiT, EgoScale, and Qwen-RobotManip, and provide a suitable cross-embodiment baseline or explain the practical obstacles to a comparable evaluation.

  • Reliability and auxiliary supervision. Can the authors provide repeated-run uncertainty for the main gains and separately ablate subtask-text and phase-index supervision?

6 comments
AuthorsOpenReview ↗

Response to Reviewer SYKF (1/3)

We thank Reviewer SYKF for the thorough review and for recognizing the in-depth study of internal computation, the broad evaluation, and the coherent architectural design. In the revised paper, revisions specific to your comments are marked in red, and revisions addressing points raised by both you and other reviewers are marked in orange. Replaced wording is struck through.

W1 & Q1: Baseline provenance, adaptation settings, and data overlap

We agree and have added the provenance statement to Appendix C (p. 27).

  • RoboTwin and LIBERO baseline numbers are taken from the original publications or official leaderboards. Real-robot baselines and the new cross-embodiment DiT4DiT baseline are trained and evaluated by us under the identical protocol.
  • EWAM downstream data follow each benchmark's official protocol. Adaptation uses a global batch of 256 for 60K steps on RoboTwin in-domain and for 20K steps on RoboTwin C2R, and a global batch of 64 for 20K steps on LIBERO. All reported results use the final checkpoint.
  • The RoboTwin portion of robot pretraining contains only clean demonstrations. Clean-to-random evaluation follows the official RoboTwin 2.0 protocol, and downstream adaptation uses only clean demonstrations of the 50 tasks. The randomized Hard scenes used for C2R evaluation therefore appear in neither pretraining nor adaptation.

W2 & Q2: Single-pass video features (success and latency)

[New experiment] [New table] We ran the requested comparison and report both success and latency in the new Table 19 (Appendix C.5, pp. 32–33), referenced from Sec. 4.3 (p. 7). Video and action noise levels are sampled independently during training (Sec. 3.3), so single-pass video features are within the training distribution of the same model; this satisfies the matched-training condition without retraining. In single-pass mode the video stream is evaluated once and its features are reused for all five action-denoising steps.

RoboTwin Randomized, same robot-pretrained checkpoint, RTX 5090:

Inference modeSuccess (%)Latency (ms / chunk)
Joint video–action denoising (default)92.8258
+ mask future, layers 10–193.55–
Single-pass video features90.0198
+ mask future, layers 10–193.2–
  • Single-pass features retain strong performance (90.0%) at 23% lower latency.
  • The dependency is not avoided: blocking future frames in layers 10–19 collapses success in both modes (3.2% vs. 3.55%). Reliance on predicted-future features is therefore not a by-product of iterative denoising; iterative joint denoising adds 2.8 points for about 60 ms per chunk.
  • [New table] Further execution-level optimizations (CUDA-Graph capture, parallel video and action streams, compiled kernels, and FP8/NVFP4 video linear layers) lower five-step chunk latency from 258 ms to 71.0 ms (3.63×; Table 20, Appendix C.5, p. 34).

W2: Evidence for the understanding–foresight–action interpretation

The interpretation rests on independent lines of evidence rather than attention alone: (i) the routing is learned (absent at 50 steps, Fig. 4, p. 7); (ii) each stage is functionally used at its depth (Table 1, p. 8: shallow VL masking −16.27, intermediate future masking −89.25, shallow video masking only −0.82); (iii) future content, not mere presence, steers actions (Table 2, p. 8). We further strengthened it with new experiments: removing future access during training costs 14 points (W1 & Q3 below), the dependence holds without iterative denoising (W2 & Q2 above), injecting predicted futures in an identical scene redirects the first grasp to the donor's block in 40/40 rollouts (see our response to Reviewer EmfM), and masking an equal number of random keys costs only 10 points, whereas masking future keys collapses success to 3.55% (see our response to Reviewer VEeM).

AuthorsOpenReview ↗

Response to Reviewer SYKF (2/3)

W1 & Q3: Fusion benefit versus capacity, and depth ordering

[New experiment] We added matched-data, matched-capacity retrained variants to Table 1 (p. 8). RoboTwin Randomized, trained without pretraining, identical data and budget:

Variant (retrained)What it testsSuccess (%)
Full EWAM–83
No future access, all layers [New]Fusion of predicted futures, at identical capacity69
No VL access, layers 0–9Single-stage constraint (shallow)78
No video access, layers 10–19Single-stage constraint (intermediate)78
Action only, layers 24–29Single-stage constraint (deep)83
VL + action only, no video expert [New]Reduced expert, matched data and budget78
Video + action only, no VL expert [New]Reduced expert, matched data and budget80

Both reduced-expert variants fall below the full model (VL–action 78, video–action 80 vs. 83), and capacity alone does not explain the gain: adding the video expert back without future access gives 69.

[New experiment] To test the depth ordering, we retrained two variants with the same hard masks placed at swapped depths:

Ordering variant (retrained)Layers 0–9Layers 10–19Layers 20–29Success (%)
Full EWAMAll sourcesAll sourcesAll sources83
Enforced [New]VL + action (no video)Video + action (no VL)Action only72.3
Reversed [New]Video + action (no VL)VL + action (no video)Action only56.8

The two variants remove the same sources and differ only in where. Following the observed ordering, with VL in shallow layers and video in intermediate layers, gives 72.3. Swapping the two gives 56.8, so the depth ordering itself matters. Both fall below the full model because every stage loses a source. Together, these results show that all three experts are needed and that they work best in this order.

W3 & Q4: DiT4DiT, EgoScale, Qwen-RobotManip, and a cross-embodiment baseline

We agree and have added these discussions to Sec. 2 (p. 4) and Appendix D.1 (p. 33).

  • DiT4DiT reads video features once, at a fixed timestep and layer, into a separate action transformer. EWAM lets action tokens read semantic, current-visual, and predicted-future features at every layer, which is what allows the depth-wise routing to emerge. DiT4DiT is now in the LIBERO tables (Table 5, p. 9; Table 16, p. 29; 98.6 vs. our 98.8).
  • EgoScale and Qwen-RobotManip use human data to pretrain VLA policies without a predictive video stream; EgoScale additionally needs aligned human–robot mid-training, and Qwen-RobotManip renders robots into human videos. EWAM trains on unmodified egocentric video and also transfers the video expert. Qwen-RobotManip and EWAM both use a camera-frame delta end-effector action space; EWAM applies it directly to human wrists (Appendix D.2, p. 34).

[New experiment] Cross-embodiment baseline. EgoScale and Qwen-RobotManip have not released code or model weights, so they cannot be trained under our protocol. We therefore compare with DiT4DiT, using its official implementation under the same protocol as EWAM: training on ARX-X5, UR5, Franka, and Piper, then evaluating on Aloha-Agilex-2 across the 50 RoboTwin tasks at the same four checkpoints.

[New table] Success rates (%) on Aloha-Agilex-2 at downstream training steps:

MethodEmbodied pretraining40K50K80K120K
DiT4DiT [New]None30.032.636.534.3
EWAMNone33.537.240.136.2
EWAMHuman44.353.458.966.9

None denotes no additional embodied pretraining; foundation backbones are retained. At 120K, EWAM without embodied pretraining reaches 36.2% versus 34.3% for DiT4DiT, while human pretraining raises EWAM to 66.9%. The comparison is included in Sec. 5.2 (p. 9), Fig. 5(a) (p. 10), and Appendix D.6 (Table 23, p. 40).

AuthorsOpenReview ↗

Response to Reviewer SYKF (3/3)

W4 & Q5 (first part): Statistical support

[New experiment] We tripled the real-robot trials to 30 per task and added training-seed repeats.

Real robot (7 tasks)Trials / taskEWAMπ0.5
Original submission1072.9% (51/70)51.4% (36/70)
Revision [New]3074.8% (157/210)49.5% (104/210)

EWAM now exceeds π0.5 on all seven tasks. We updated Tables 6–7 and Sec. 5 (p. 9), and Appendix C (p. 29).

Training seeds [New]Mean ± std (3 seeds)95% CI
Main result, in-domain Randomized92.8 ± 0.2[92.4, 93.1]
Table 1: full EWAM83.2 ± 0.4[82.2, 84.2]
Table 1: symmetric full attention80.9 ± 0.4[80.0, 81.8]
Table 1: no VL access, layers 0–977.9 ± 0.5[76.8, 79.1]
Table 1: no video access, layers 10–1977.7 ± 0.5[76.4, 79.0]

The lower bound of the main result (92.4) remains above the strongest baseline, Fast-WAM (91.8; Table 4, p. 8). The structural ablations keep their ordering across seeds.

LIBERO: most methods exceed 97.5%; the benchmark is saturated, which compresses all differences. Our main generalization evidence is RoboTwin C2R, where EWAM leads the strongest baseline by 4.2 points and π0.5 by 26.1.

Q5 (second part): Separate ablation of subtask-text and phase-index supervision

[New experiment] As requested, we post-trained two additional variants with only the subtask-text head or only the phase-index head, under the same backbone, data, and training budget, and evaluated them as in the original table (200 episodes per task). The results are added to Table 8 (p. 10) and Sec. 5.3 (pp. 9–10), and the protocol is updated in Appendix C.4 (p. 31). Full-episode success (%):

SettingInstructed RankingInstructed StackingInstructed Ranking & Stacking
Neither83.544.51.5
Subtask only [New]87.049.57.5
Phase only [New]88.056.511.0
Both91.058.516.0

Each form of supervision improves over no supervision on all three tasks, and combining both performs best.

Reviewer_EmfMOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

EWAM combines vision-language, video, and action experts using depth-aligned asymmetric attention and joint video-action flow matching. The paper studies a learned depth-dependent redistribution of action-query attention, supported by token-normalized cross-task measurements, checkpoint tracking, masking, and donor-feature injection. It also evaluates robot and human pretraining, real-robot manipulation, and subtask-phase supervision, reporting 72.1 percent RoboTwin clean-to-random success, 92.8 percent in-domain randomized success, and 98.8 percent average LIBERO success.

Critical strengths
  1. The routing analysis is substantially stronger than an isolated attention visualization. The image-to-future crossover survives token-count normalization on 49 of 50 tasks, and the paper reports task-level uncertainty and training-checkpoint evolution (Tables 9-11). Fixed-checkpoint video masking across depth and donor-future injection add functional evidence: intermediate future masking reduces success from 92.80 to 3.55 percent, and donor injection changes arm choice and spatial targeting on Adjust Bottle. These results support a useful, content-dependent role for the predictive stream, even though the stronger functional interpretation needs qualification.

  2. The system is evaluated beyond one benchmark, with physical experiments on three robot platforms and controlled tests of human initialization and phase supervision. In particular, the held-out-embodiment comparison at matched 120K steps improves from 36.2 to 66.9 percent, and the with/without phase-supervision comparison uses the same backbone and training data (Section 5.3 and Appendix C.4). These provide practical evidence that the overall training approach is useful, rather than only producing an interesting internal attention pattern.

Critical weaknesses
  1. The central interpretation goes beyond the demonstrated routing and source dependence. The abstract and Figure 1 describe intermediate computation as mainly using, or being dominated by, predicted futures. However, Table 10 reports only 24.31 percent middle-stage video attention, versus 36.55 percent VL and 39.14 percent action attention; the appendix appropriately describes this stage as distributed. More importantly, future-token locations do not identify exclusively future-state information: the video stream also reads the current condition frame. The donor intervention changes object identity, orientation, and position between donor and target (Appendix B.5), so it changes present-state information and cross-stream consistency together with the proposed future content. The performance collapse and transferred grasp behavior establish that these features affect control, but do not isolate predictive foresight as their relevant information. The depth-specific video masking is valuable, and Appendix F acknowledges the causal limitation. Nevertheless, the paper's main contribution should be framed as learned routing with content-dependent predictive-stream use unless a controlled intervention separates future-outcome information from current-scene information. This distinction matters because the ordered semantic-understanding/foresight/action account is the principal new insight beyond existing multi-expert models.

  2. The policy results do not sufficiently isolate the benefit of the proposed unification from the training recipe. On RoboTwin Randomized, the model without robot pretraining scores 83.0 percent (Table 1), while the robot-pretrained model scores 92.8 percent (Table 4); Fast-WAM reports 91.8 percent without embodied pretraining. These different recipes do not establish an architectural ranking. The 83-versus-81 asymmetric/bidirectional comparison controls the mask within EWAM, but does not compare the complete design against a strong reduced-expert or future-independent counterpart using the same data and optimization budget. Consequently, the current results leave the incremental policy value of this unification unclear. The clean-to-random evidence additionally needs an auditable data boundary: Appendix E.2 includes simulation pretraining sources citing RoboTwin 2.0, whereas Section 5.1 excludes target-embodiment randomized data only during downstream adaptation. This does not demonstrate contamination, but it leaves unclear whether the claimed generalization holds without such exposure in pretraining. Resolving the training comparison and data provenance is more consequential here than a small additional leaderboard gain.

Questions for authors
  1. Can you provide a content-intervention control that holds the current scene compatible while changing the predicted outcome, or otherwise separates future information from current-state information in the patched features? Please also align the abstract/Figure 1 dominance wording with Table 10.

  2. Is there a reduced-expert or future-independent control trained with the same robot-pretraining corpus, downstream demonstrations, initialization, and optimization budget as EWAM? Please identify which architectural benefit remains after these factors are held fixed. This is a targeted control for the claimed mechanism, not a request to retrain every published baseline.

  3. Which RoboTwin tasks, embodiments, and scene-randomization regimes occur in the simulation portion of robot pretraining? Please specify the exclusions used for the clean-to-random evaluation and distinguish directly rerun baseline results from numbers taken from prior publications.

6 comments
AuthorsOpenReview ↗

Response to Reviewer EmfM

We thank Reviewer EmfM for the constructive review and for finding the routing analysis substantially stronger than an isolated attention visualization. In the revised paper, revisions specific to your comments are marked in blue, and revisions addressing points raised by both you and other reviewers are marked in orange. Replaced wording is struck through.

W1 & Q1 (first part): Separating future-outcome from current-scene information

[New experiment] [New table] We added a same-scene control (Sec. 4.3, p. 8; Appendix B.5, p. 22; Fig. 8 and Table 12, p. 23). On Instructed Stacking, donor and target start from the same seed, so the scene, condition frame, and all object poses are identical. They differ only in the instruction, which specifies the first block to grasp (e.g., red for the donor and blue for the target in Fig. 8). During the first three policy calls, we inject the donor's future features into the target at layers 10–19, keeping the target's own VL, condition-frame, and action tokens. The current scene is thus identical, while the predicted futures differ. In every rollout, the target's first grasp switches to the donor's block. The redirection therefore comes from the predicted outcome, ruling out the influence of the current scene.

Condition (4 instruction pairs × 10 seeds)InstructionInjected futuresFirst grasp = target's blockFirst grasp = donor's block
BaselineBnone40/400/40
Inject ABdonor A0/4040/40

W1 & Q1 (second part): Abstract and Figure 1 wording

We agree and have revised the wording. The abstract (p. 1) and Figure 1 caption (p. 2) now state that intermediate layers "shift their visual attention from the current image to predicted future frames".

W2 & Q2: Architecture versus recipe

[New experiment] As requested, we trained a future-independent counterpart with the same parameters, initialization, downstream data, and optimization budget: action tokens never read future-frame tokens at any layer, while the condition frame, the video expert, and video supervision are kept. It is added as a new row to Table 1 (p. 8) and discussed in Sec. 4.3 (p. 7).

RoboTwin Randomized, trained without pretraining, same evaluation as Table 1:

ConfigurationSuccess (%)Δ
Full EWAM83–
Symmetric full attention81−2
No VL access, layers 0–978−5
No video access, layers 10–1978−5
Action only, layers 24–29830
No future access, all layers [New row]69−14
VL + action only, no video expert [New row]78−5
Video + action only, no VL expert [New row]80−3

With these factors held fixed, the benefit that remains is the unified understanding–foresight–action design: removing shallow semantic grounding (no VL access, layers 0–9) gives 78, removing the foresight expert gives 78, and keeping the expert while hiding its predicted futures gives 69, whereas the full model reaches 83. Each stage contributes, and foresight contributes at least in part through its predicted futures rather than through added capacity. Due to limited computational resources within the short discussion period, we were unable to repeat the full robot pretraining for this control, and therefore conducted the comparison without pretraining, keeping all other factors identical. We would be glad to include the robot-pretrained version in the final paper once the training resources become available.

Fast-WAM's 91.8 and our from-scratch 83.0 are not trained under the same budget: Fast-WAM uses a global batch of 1,024, whereas Table 1 uses 256. A public reproduction with Fast-WAM's official code and a global batch of 256, at a matched number of training samples, reports 66.3% on RoboTwin Randomized (issue #65 in the official Fast-WAM repository). Our state-of-the-art claim rests on the robot-pretrained comparison (92.8) and especially on clean-to-random (72.1 vs. 46.0 for π0.5 and 1.9 for Fast-WAM).

W2 & Q3: Simulation pretraining data and baseline provenance

  • The RoboTwin portion of robot pretraining contains only clean demonstrations. Clean-to-random evaluation follows the official RoboTwin 2.0 protocol, and downstream adaptation uses only clean demonstrations of the 50 tasks. The randomized Hard scenes used for C2R evaluation therefore appear in neither pretraining nor adaptation.
  • We agree and now state the baseline provenance in Appendix C (p. 27): simulation baseline numbers are taken from the original publications or official leaderboards, and real-robot baselines are trained and evaluated by us under the identical protocol.

Dear Reviewer EmfM,

I hope this message finds you well. With less than three days left in the discussion period, I wanted to ensure we have addressed all your concerns satisfactorily. Your insights are invaluable to us, and we're eager to address any remaining issues to improve our work.

Thank you for your time and effort in reviewing our paper.

Dear Reviewer EmfM,

As the discussion period closes in less than 24 hours, we would like to kindly follow up on our responses. We have tried to address each of your comments with point-by-point responses and revisions to the paper. If any concern remains unresolved, we would be glad to clarify it before the deadline. If our responses have addressed your concerns, we would greatly appreciate it if you could take them into account in your final assessment.

Thank you again for your time and thoughtful feedback.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary Response to All Reviewers

We thank all reviewers for their careful reading and constructive suggestions. Reviewers found the routing analysis "substantially stronger than an isolated attention visualization" (Reviewer EmfM), "beyond standard attention heatmaps" (Reviewer VEeM), and the evaluation broad, with the RoboTwin clean-to-random result "particularly encouraging" (Reviewers SYKF and VEeM).

Following the reviews, we ran new experiments and uploaded a revised paper. All changes are colour-coded by reviewer (blue: Reviewer EmfM, red: Reviewer SYKF, green: Reviewer VEeM, orange: multiple reviewers), with replaced wording struck through.

Revisions to the paper ([New] marks new experiments, tables, and figures)

PageLocationRevisionReviewer
1, 2Abstract; Figure 1 captionIntermediate-layer wording aligned with Table 10EmfM
4Sec. 2Discussion of DiT4DiTSYKF
7Sec. 4.2Routing is absent at 50 steps despite the same layer-aligned backbones, and also emerges when every block reads only the final Qwen3-VL layerVEeM
7–8Sec. 4.3; Table 1[New] Retrained controls: no future access at any layer 69, VL + action only 78, video + action only 80, final-layer Qwen3-VL 79 (full model 83). [New] Masking an equal number of random keys 82.80 vs. 3.55 for future keysAll
8Sec. 4.3[New] Same-scene control: injected futures redirect the first grasp in 40/40 rollouts (0/40 without injection)EmfM
9Sec. 5; Tables 6–7[New] Real-robot trials tripled to 30 per task: EWAM highest on all 7 tasks, 74.8% vs. 49.5% for π0.5SYKF
9Table 5DiT4DiT added to LIBEROSYKF
9–10Sec. 5.2; Fig. 5(a)[New] Cross-embodiment transfer at 120K: robot pretraining 55.3% vs. human pretraining 66.9%; DiT4DiT 34.3%SYKF, VEeM
9–10Sec. 5.3; Table 8[New] Separate subtask-text and phase-index ablation: each head alone helps, both perform bestSYKF
22–23Appendix B.5; Fig. 8, Table 12[New] Same-scene control protocol and resultsEmfM
27Appendix B.7[New] Routing analysis without layer-wise backbone alignmentVEeM
27Appendix CBaseline provenance statementEmfM, SYKF
29Appendix C.3; Table 1630-trial real-robot protocol; DiT4DiT added to full LIBERO resultsSYKF
31Appendix C.4Protocol for the four subtask-supervision variantsSYKF
32–33Appendix C.5; Table 19[New] Single-pass video features: 90.0% at 198 ms vs. 92.8% at 258 ms; future masking still collapses success (3.2%)SYKF
34Appendix C.5; Table 20[New] Further inference optimization: five-step chunk latency 258 → 71.0 ms (3.63×)SYKF
33–34Appendix D.1, D.2Discussion of EgoScale and Qwen-RobotManipSYKF
39–40Appendix D.6; Table 23[New] Cross-embodiment comparison of three EWAM initializations and DiT4DiTSYKF, VEeM

Additional new results reported in the individual responses

  • Enforced vs. reversed depth ordering, retrained: 72.3 vs. 56.8 (Reviewer SYKF).
  • Training-seed repeats (3 seeds): in-domain 92.8 ± 0.2; Table 1 variants 83.2 / 80.9 / 77.9 / 77.7 with non-overlapping 95% CIs (Reviewers SYKF and VEeM).
  • Value-weighted attention on 50 tasks preserves the three stages (Reviewer VEeM).
  • C2R without robot pretraining: 24.2% vs. 3.16% for starVLA and 1.90% for Fast-WAM (Reviewer VEeM).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.