acceptodds
Under review as a conference paper at ICLR 2027

OpenWAM: An Open, Modular Exploration Towards Systematic World–Action Model Pretraining

28 views

Abstract

World–Action Models inherit world knowledge from video-generative priors, and channel it into executable control signals through embodied experience. Existing systems, however, are monolithic: the generative backbone, visual representation, architecture, information flow, inference procedure, and training data are tightly coupled, obscuring which design choices matter and why. We introduce OpenWAM, an open research stack that turns world–action pretraining into a controlled experimental program. OpenWAM-Infra factorizes the WAM design space into composable modules with unified training, inference, deployment, and evaluation. On this substrate, OpenWAM-Study examines three questions through controlled experiments: what to inherit, how world and action learning interact, and how their synergy scales; and distills three principles: upstream knowledge transfers through a sufficiently capable generative backbone and a compact, information-rich latent space; world–action synergy requires dedicated action capacity, explicit world-to-action information flow, and synchronized joint denoising; and embodied pretraining principally improves out-of-domain generalization, with one-stage co-training over egocentric and robot data integrating world coverage and action grounding. Composing these principles, we build OpenWAM-α, an open WAM pretrained on roughly 6,400 hours of egocentric human and robot data and evaluated across simulation and real-world benchmarks. Across the eight simulation benchmarks and the real-robot experiments, which together span embodiments from single-arm and bimanual manipulation to dexterous hands, OpenWAM-α delivers consistently excellent performance, sustaining its top-tier standing from simulation to the physical world. We release the full stack, including infrastructure, evaluation protocols, pretrained models, and data recipes, to facilitate future research.

Primary area:
applications to robotics, autonomy, planning
Keywords:
World Action Model

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_VVgpOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper presents OpenWAM, an open research stack for World-Action Models (WAMs), policies that inherit world knowledge from video-generation priors and learn control from embodied data. It has three parts. OpenWAM-Infra factorizes the WAM design space into composable modules (visual encoder, stream backbones, visibility mask) behind one trainer, one policy server, and one evaluation protocol, assembling six architectures from configuration. OpenWAM-Study runs controlled experiments on RoboTwin2.0 and distills three findings: world knowledge transfers best through a capable generative backbone and a compact, information-rich latent space; world-action synergy requires world-to-action information flow in training and synchronized denoising at inference; and embodied pretraining mainly improves out-of-domain generalization, with egocentric video and robot trajectories contributing different strengths and one-stage co-training the best integration. OpenWAM-alpha composes the recipe into a 5B world stream plus 1B action stream model pretrained on 518.5M frames (6,369 hours) of egocentric and robot data, evaluated on eight simulation benchmarks and three real-robot embodiments, including a dexterous hand whose 30-D action space never appears in pretraining. It sits near the top on most benchmarks, and the paper includes an unusually honest analysis of its one clear failure (LIBERO-Plus).

Critical strengths
  • Open source contribution. Six architectures, five video backbones, four visual encoders, four attention masks, one trainer, one policy server, eight benchmark adapters, released with pretrained weights and data recipes. StarVLA[1] did this for VLAs; nothing equivalent existed for WAMs, and the design decisions (self-contained checkpoints, benchmarks as thin WebSocket clients, the 80-D unified action space) are the right ones. Independent of any finding, this lowers the cost of doing careful WAM research for everyone, and OpenWAM-alpha is immediately useful as a strong open baseline across three real embodiments.
  • The controlled studies ask the right questions, and several answers are crisp. The mask ablation isolates the direction of information flow with a clean five-point split (isolated and video-sees-action at 87.4-87.6 versus action-sees-video and mutual at 92.2-92.4), which is a real result. The denoising sweep is a thorough null result: sixteen asynchronous configurations, none beating synchronized denoising, which usefully kills the plan-then-imagine design direction at test time. The LIBERO-Plus analysis (E.5) is model self-criticism: it traces the deficit to single-arm data coverage and pixel-latent sensitivity, corroborated across three baselines with different prediction targets.

References

[1] Community, StarVLA. "StarVLA: A Lego-like Codebase for Vision-Language-Action Model Developing." arXiv preprint arXiv:2604.05014 (2026).

Critical weaknesses
  • There is no variance anywhere, and the recipe-driving decisions are sub-point. The entire Study rests on one benchmark and one run per configuration, success rate only. The margins that decide the recipe are 92.36 versus 92.60 (dual versus tri-system), 92.39 versus 92.15 (action-sees-video versus mutual), 93.0 versus 92.3 (sync versus the best async schedule), and the claimed mask reversal after pretraining (Figure 7) is +0.70, -0.24, +0.72, and +0.16 points. Without seeds or any significance testing, Findings 2 and 3 and the reversal claim could be noise, and the stakes are higher because the findings are confirmatory of results the paper itself cites (the test-time schedule conclusion, the representation-latent results, the co-training recipe): their value depends on being measured better than the originals, and they are not.
  • The cross-model rankings are not auditable. Figure 9 and Appendix H compare against roughly 25 models, but the paper never states which numbers were re-run under the OpenWAM-Infra protocol (same preprocessing, episode counts, and evaluation conditions) and which are quoted from papers or leaderboards. The comparison is also confounded by the variable the paper itself calls decisive: pretraining data coverage, with baselines ranging from 6.4k hours (here) to over 100k hours. Finally, "top-tier performance across eight simulation benchmarks" overstates the record: on LIBERO-Plus the model scores 69.2 against 89.0 for the best baseline, 12th of 13 in its own table. The honest claim, consistently strong with one explained weak spot, is a good claim, and the abstract should say that.
Questions for authors
  • Can you add multi-seed runs (3 or more) for the Study's decision-driving comparisons and report mean and std: the architecture table (Table 1), the mask tables (Table 2 and Figure 7), and the denoising sweep (Figure 5)? Most of these margins are under one point, so Findings 2 and 3 and the mask-reversal claim stand or fall on whether they survive seed noise. This is the single change that would most affect my confidence in the paper.
  • For every baseline number in Figure 9 and Appendix H, please state whether it was re-run under your protocol or quoted, and where quoted, how episode counts, observation preprocessing, and evaluation conditions differ from yours. As it stands, the cross-model rankings, and therefore the "top-tier" claim, cannot be audited.
5 comments
AuthorsOpenReview ↗

Rebuttal for Reviewer VVgp – Question1 (Fixed-checkpoint evaluations runs)

Q1. Stability across evaluation and training seeds.

We sincerely thank you for highlighting this concern. We separately report variability across evaluation seeds with fixed checkpoints and across independent training runs.

(a) Fixed-checkpoint evaluations: seeds 42, 0, and 10.

Keeping each configuration's original trained checkpoint fixed, we ran two additional evaluations with seed=0 and seed=10, supplementing the original seed=42 evaluation for Tables 1 and 2 and Figures 5 and 7. RoboTwin evaluation comprises 10,000 test episodes. Below we report mean ± sample std over these three evaluation seeds (n=3, ddof=1). Success rates are in %, with std and differences in percentage points.

Architecture ablation (Table 1).

ArchitectureCleanRandomizedAverage
Single-system Vanilla85.000 ± 0.17885.827 ± 0.36185.413 ± 0.177
Single-system MoE86.093 ± 0.32983.120 ± 0.17484.607 ± 0.078
Dual-system Joint Self-Attention92.493 ± 0.24892.700 ± 0.27892.597 ± 0.226
Dual-system Joint Cross-Attention87.960 ± 0.24688.740 ± 0.08788.350 ± 0.140
Dual-system Detached Cross-Attention92.200 ± 0.20991.593 ± 0.06491.897 ± 0.072
Dual-system IDM87.527 ± 0.22188.020 ± 0.11187.773 ± 0.166
Tri-system Joint Self-Attention92.787 ± 0.30492.607 ± 0.23292.697 ± 0.134

Attention-mask ablation (Table 2).

Attention maskCleanRandomizedAverage
Isolated88.293 ± 0.18686.853 ± 0.12187.573 ± 0.148
Video Sees Action88.007 ± 0.28087.427 ± 0.20487.717 ± 0.085
Action Sees Video93.260 ± 0.24692.047 ± 0.22592.653 ± 0.235
Mutual92.413 ± 0.35892.047 ± 0.22592.230 ± 0.221

Denoising schedules (Figure 5). Synchronized denoising achieves 92.840 ± 0.260%, compared with the following asynchronous schedules:

Schedule familyParameterVideo-leadingAction-leading
Variance shiftα=491.773 ± 0.14291.073 ± 0.081
Variance shiftα=891.220 ± 0.13190.553 ± 0.140
Variance shiftα=1691.940 ± 0.06989.600 ± 0.322
Variance shiftα=3291.453 ± 0.08188.320 ± 0.156
Linear offseto=0.291.653 ± 0.08192.140 ± 0.354
Linear offseto=0.490.860 ± 0.25191.913 ± 0.186
Linear offseto=0.691.533 ± 0.13392.307 ± 0.250
Linear offseto=0.891.047 ± 0.18692.307 ± 0.250

Information flow before and after pretraining (Figure 7). ASV denotes Action Sees Video; Δ = Mutual − ASV, with its mean and std computed from the paired per-seed differences.

SettingASVMutualΔ (percentage points)
From scratch / RoboTwin-Full (Avg.)92.653 ± 0.23592.230 ± 0.221−0.423 ± 0.222
Pretrained / RoboTwin-Full (Avg.)92.513 ± 0.15192.600 ± 0.087+0.087 ± 0.070
Pretrained / RoboTwin-C2R (ID)86.720 ± 0.23687.467 ± 0.186+0.747 ± 0.064
Pretrained / RoboTwin-C2R (OOD)25.607 ± 0.23426.520 ± 0.140+0.913 ± 0.200

The observed success-rate standard deviations are at most 0.361 percentage points. Across all three evaluation seeds, the average-score rankings in Tables 1 and 2 remain unchanged, synchronized denoising outperforms every tested asynchronous schedule, and the preference for Mutual after pretraining is reproduced. These results support the consistency of the reported qualitative conclusions under evaluation-seed variation.

AuthorsOpenReview ↗

Rebuttal for Reviewer VVgp – Question1 (Independent training runs)

(b) Independent training runs: seeds 42 and 0.

All original experiments were trained with seed=42. We retrained the configurations in Tables 1 and 2 and the pretrained information-flow study with seed=0, using exactly the same data and training configurations as the original runs, and independently evaluated the resulting models. Below we report the new scores and mean ± population std across the original and new training runs (n=2, ddof=0). Success rates are in %.

Architecture ablation on RoboTwin2.0-Full (Table 1).

ArchitectureSeed=0 CleanSeed=0 RandomizedSeed=0 Avg.Clean mean ± stdRandomized mean ± stdAvg. mean ± std
Single-system Vanilla85.4886.1085.7985.340 ± 0.14085.950 ± 0.15085.645 ± 0.145
Single-system MoE86.3883.1684.7786.300 ± 0.08083.100 ± 0.06084.700 ± 0.070
Dual-system Joint Self-Attention92.4892.6492.5692.410 ± 0.07092.510 ± 0.13092.460 ± 0.100
Dual-system Joint Cross-Attention87.9788.7988.3887.915 ± 0.05588.715 ± 0.07588.315 ± 0.065
Dual-system Detached Cross-Attention92.3491.9492.1492.200 ± 0.14091.790 ± 0.15091.995 ± 0.145
Dual-system IDM87.4687.9087.6887.610 ± 0.15088.020 ± 0.12087.815 ± 0.135
Tri-system Joint Self-Attention93.1092.5692.8392.970 ± 0.13092.460 ± 0.10092.715 ± 0.115

Attention-mask ablation on RoboTwin2.0-Full (Table 2).

Attention maskSeed=0 CleanSeed=0 RandomizedSeed=0 Avg.Clean mean ± stdRandomized mean ± stdAvg. mean ± std
Isolated88.2186.9387.5788.145 ± 0.06586.835 ± 0.09587.490 ± 0.080
Video Sees Action88.0987.6187.8588.005 ± 0.08587.475 ± 0.13587.740 ± 0.110
Action Sees Video93.2892.0892.6893.130 ± 0.15091.940 ± 0.14092.535 ± 0.145
Mutual92.6492.0892.3692.570 ± 0.07091.940 ± 0.14092.255 ± 0.105

Information flow before and after pretraining (Figure 7). ASV denotes Action Sees Video; Δ = Mutual − ASV is measured in percentage points.

SettingSeed=0 ASVSeed=0 MutualSeed=0 ΔASV mean ± stdMutual mean ± stdΔ mean ± std
From scratch / RoboTwin-Full (Avg.)92.6892.36−0.3292.535 ± 0.14592.255 ± 0.105−0.280 ± 0.040
Pretrained / RoboTwin-Full (Avg.)92.4592.76+0.3192.395 ± 0.05592.630 ± 0.130+0.235 ± 0.075
Pretrained / RoboTwin-C2R (ID)87.2387.79+0.5687.105 ± 0.12587.735 ± 0.055+0.630 ± 0.070
Pretrained / RoboTwin-C2R (OOD)25.5626.22+0.6625.630 ± 0.07026.320 ± 0.100+0.690 ± 0.030

Across these two independent training runs, the average-score rankings in Tables 1 and 2 are unchanged, and the preference for Mutual after pretraining is reproduced. The new scores differ from the originals by at most 0.30 percentage points, providing additional evidence for the stability of these qualitative conclusions.

AuthorsOpenReview ↗

Rebuttal for Reviewer VVgp – Question2

Q2. Baseline-score sources and evaluation protocols.

We sincerely thank you for raising this concern. OpenWAM-α scores are our evaluations; the external baseline scores in Figure 9 and Appendix H are quoted from published papers or official leaderboards.

Our benchmark clients are built on the official benchmark repositories, environments, or evaluation services. They reuse benchmark-owned tasks and success conditions, with local adapters handling policy communication and observation/action conversion:

BenchmarkEvaluation implementationBaseline-score source
LIBEROOfficial LIBERO environments, task files, and initial statesPublished comparison tables in the three papers cited below
LIBERO-PlusOfficial LIBERO-Plus environments and task variantsPublished comparison tables in the three papers cited below
VLABenchOfficial VLABench Evaluator and frozen evaluation configurationsPublished comparison tables in the three papers cited below
RoboTwin2.0-FullOfficial RoboTwin evaluation script, tasks, and success criteriaPublished comparison tables in the three papers cited below
RoboTwin2.0-Clean2RandomOfficial RoboTwin evaluation; clean-only downstream trainingOfficial RoboDojo leaderboard, including its RoboTwin panel
RoboDojoExternal XPolicyLab benchmark evaluation infrastructureOfficial RoboDojo leaderboard
EBenchOfficial Generalist/Test-Mini split and EBench/GenManip online evaluation procedureOfficial EBench evaluation website
RoboCasa365Official RoboCasa environments, task horizons, and success signalsOfficial RoboCasa leaderboard
RoboCasa-GR1Official GR1 tabletop environments and success signalsOriginal benchmark paper and subsequent papers evaluating the same benchmark

The published comparison tables used for LIBERO, LIBERO-Plus, VLABench, and RoboTwin2.0-Full are from Qwen-RobotManip, Xiaomi-Robotics-1, and ABot-M0.5.

We screened quoted results for compatibility with each benchmark's task definitions, downstream training-data regime, and evaluation setting, excluding known incompatible entries. Observation preprocessing and pretraining data remain policy-specific: each policy retains its native camera processing, normalization, action representation, and pretrained checkpoint.

We sincerely thank you for your careful and thorough review. We will incorporate these clarifications and additional evaluation details into the final version of the paper.

Reviewer_kSmpOpenReview ↗
Rating
4
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

OpenWAM is a modular research framework designed to systematically investigate the pretraining of World-Action Models. OpenWAM-Infra decouples visual encoders, world and action backbones, cross-stream information visibility, and training and deployment pipelines. OpenWAM-Study systematically compares visual representations, network architectures, information flow, denoising schedules, and pretraining data configurations within a unified framework. OpenWAM-alpha integrates the selected designs into a large-scale pretrained model.

Critical strengths
  1. Rather than merely reporting the performance of a single model, this work attempts to investigate previously coupled design choices, including representations, architectures, information flow, and inference strategies, within a unified experimental framework. Such standardized interfaces and composable experimental capabilities offer practical value and reusability for future WAM research.

  2. The proposed model is evaluated across benchmarks, with additional evidence from real-world robotic execution.

Critical weaknesses

Some mechanistic interpretations go beyond what the controlled experiments can establish.

For example, synchronous denoising achieves the best performance among the evaluated schedules, which is a valuable empirical finding. However, this result alone does not sufficiently distinguish between the proposed representation-learning interpretation and an alternative mechanism involving implicit planning followed by inverse dynamics.

Similarly, the observation that mutual information flow outperforms unidirectional information flow after pretraining demonstrates the benefits of bidirectional access under the evaluated settings. However, it does not directly establish that the two streams have learned the specific semantic correspondences described by the authors.

Questions for authors
  1. Although the dexterous-hand platform was not included during pretraining, the model was subsequently fine-tuned on tasks involving this platform. The authors should characterize this result as adaptation or transfer rather than zero-shot control, to avoid potentially misleading interpretations.

  2. Do the main design conclusions remain valid beyond RoboTwin 2.0? Could the authors reproduce a small subset of the key ablations on another embodied benchmark or under different camera perturbation settings?

  3. How large are the performance differences reported in Tables 1 and 2, as well as the differences in information-flow configurations after pretraining, relative to the variance across independent training runs?

3 comments
AuthorsOpenReview ↗

Rebuttal for Reviewer kSmp – Question1& 2

Q1. Clarifying transfer claims and mechanistic interpretations.

We sincerely thank you for your careful review and constructive comments on dexterous-hand transfer and the mechanistic interpretations of synchronized denoising and mutual attention. We acknowledge that our wording could be more precise and will revise the relevant statements accordingly.

Q2. Cross-benchmark validation on LIBERO and LIBERO-Plus.

We extended the architecture, attention-mask, and pretraining studies to LIBERO and LIBERO-Plus.

LIBERO-Plus evaluation correction. In six non-Language categories, the official evaluation code derives instructions from filenames and appends perturbation metadata (e.g., view ... initstate ..., light 1), unintentionally changing the policy's language input.

This issue is reported in #48, #62, #64, and #65, with related examples in #46 and a proposed fix in PR #71. As of 15 November 2026, the issues remain open and the PR is unmerged; the upstream code still uses filename-derived instructions.

We restore the original LIBERO instructions for these six categories and leave Language unchanged. Success metrics and criteria, scenes, perturbations, and initial states are unchanged. Auditing 10,030 variants found no mapping errors to the original 40 LIBERO instructions.

For the paper's LIBERO checkpoint (previously 69.2% on LIBERO-Plus), paired re-evaluation on the same tasks gives:

Instruction protocolOverall success (%)Camera success (%)Noise success (%)
Original instructions70.130.442.5
Corrected instructions81.155.357.5

Despite an overall gain of about 11 percentage points, Camera and Noise weaknesses remain, supporting the paper's robustness discussion. All LIBERO-Plus results below use the corrected protocol.

All study models were trained on LIBERO for 10 epochs, batch size 256. Scores are mean ± sample std over three evaluation seeds per checkpoint, with tasks and initial states fixed and policy/environment randomness varied.

Architecture study. All seven architectures were initialized from Wan2.2.

ArchitectureLIBEROLIBERO-Plus
Single-system Vanilla (Mutual)98.4 ± 0.271.5 ± 0.6
Single-system MoE (Mutual)98.5 ± 0.470.1 ± 0.7
Dual-system Joint Self-Attention (Mutual)98.6 ± 0.172.2 ± 0.5
Dual-system Joint Cross-Attention97.7 ± 0.371.9 ± 1.0
Dual-system Detached Cross-Attention98.6 ± 0.172.0 ± 0.6
Dual-system IDM98.6 ± 0.571.1 ± 0.7
Tri-system Joint Self-Attention (Mutual)98.2 ± 0.769.6 ± 0.9

Dual-system joint self-attention has the highest mean LIBERO-Plus score and remains competitive on LIBERO; adding the third stream provides no gain here.

Attention-mask and pretraining studies. All use dual-system joint self-attention; “Wan2.2 only” means no embodied pretraining.

InitializationAttention maskLIBEROLIBERO-Plus
Wan2.2 onlyMutual98.6 ± 0.172.2 ± 0.5
Wan2.2 onlyAction Sees Video99.1 ± 0.672.4 ± 0.5
Ego + robot co-trainingAction Sees Video98.7 ± 0.675.9 ± 0.2
Ego + robot co-trainingMutual98.7 ± 0.376.5 ± 0.8
Robot-only pretrainingAction Sees Video99.0 ± 0.877.7 ± 0.6

Mutual and Action Sees Video do not significantly differ before embodied pretraining. After co-training, Mutual has a slightly higher LIBERO-Plus mean. Pretraining has little effect on near-saturated LIBERO but improves LIBERO-Plus, supporting OOD generalization benefits beyond RoboTwin.

The preferred data recipe is embodiment-dependent. With Action Sees Video, robot-only outperforms co-training on LIBERO-Plus (77.7% versus 75.9%). This may reflect embodiment mismatch: bimanual ego data may transfer less effectively to single-arm LIBERO/LIBERO-Plus. On bimanual RoboTwin, Figure 6 shows co-training improves OOD success over robot-only (26.62% versus 23.80%), supporting ego data's benefit for bimanual transfer.

AuthorsOpenReview ↗

Rebuttal for Reviewer kSmp – Question3 (Independent training runs)

Q3. Stability across training and evaluation seeds.

We separately examined variability across independent training runs and across evaluation seeds with fixed checkpoints. The two sets of statistics are reported separately.

(a) Independent training runs: seeds 42 and 0.

All original experiments were trained with seed=42. We retrained the configurations in Tables 1 and 2 and the pretrained information-flow study with seed=0, using exactly the same data and training configurations as the original runs, and independently evaluated the resulting models. Below we report the new scores and mean ± population std across the original and new training runs (n=2, ddof=0). Success rates are in %.

Architecture ablation on RoboTwin2.0-Full (Table 1).

ArchitectureSeed=0 CleanSeed=0 RandomizedSeed=0 Avg.Clean mean ± stdRandomized mean ± stdAvg. mean ± std
Single-system Vanilla85.4886.1085.7985.340 ± 0.14085.950 ± 0.15085.645 ± 0.145
Single-system MoE86.3883.1684.7786.300 ± 0.08083.100 ± 0.06084.700 ± 0.070
Dual-system Joint Self-Attention92.4892.6492.5692.410 ± 0.07092.510 ± 0.13092.460 ± 0.100
Dual-system Joint Cross-Attention87.9788.7988.3887.915 ± 0.05588.715 ± 0.07588.315 ± 0.065
Dual-system Detached Cross-Attention92.3491.9492.1492.200 ± 0.14091.790 ± 0.15091.995 ± 0.145
Dual-system IDM87.4687.9087.6887.610 ± 0.15088.020 ± 0.12087.815 ± 0.135
Tri-system Joint Self-Attention93.1092.5692.8392.970 ± 0.13092.460 ± 0.10092.715 ± 0.115

Attention-mask ablation on RoboTwin2.0-Full (Table 2).

Attention maskSeed=0 CleanSeed=0 RandomizedSeed=0 Avg.Clean mean ± stdRandomized mean ± stdAvg. mean ± std
Isolated88.2186.9387.5788.145 ± 0.06586.835 ± 0.09587.490 ± 0.080
Video Sees Action88.0987.6187.8588.005 ± 0.08587.475 ± 0.13587.740 ± 0.110
Action Sees Video93.2892.0892.6893.130 ± 0.15091.940 ± 0.14092.535 ± 0.145
Mutual92.6492.0892.3692.570 ± 0.07091.940 ± 0.14092.255 ± 0.105

Information flow before and after pretraining (Figure 7). ASV denotes Action Sees Video; Δ = Mutual − ASV is measured in percentage points.

SettingSeed=0 ASVSeed=0 MutualSeed=0 ΔASV mean ± stdMutual mean ± stdΔ mean ± std
From scratch / RoboTwin-Full (Avg.)92.6892.36−0.3292.535 ± 0.14592.255 ± 0.105−0.280 ± 0.040
Pretrained / RoboTwin-Full (Avg.)92.4592.76+0.3192.395 ± 0.05592.630 ± 0.130+0.235 ± 0.075
Pretrained / RoboTwin-C2R (ID)87.2387.79+0.5687.105 ± 0.12587.735 ± 0.055+0.630 ± 0.070
Pretrained / RoboTwin-C2R (OOD)25.5626.22+0.6625.630 ± 0.07026.320 ± 0.100+0.690 ± 0.030

Across these two independent training runs, the average-score rankings in Tables 1 and 2 are unchanged, and the preference for Mutual after pretraining is reproduced. The new scores differ from the originals by at most 0.30 percentage points, providing additional evidence for the stability of these qualitative conclusions.

AuthorsOpenReview ↗

Rebuttal for Reviewer kSmp – Question3 (Fixed-checkpoint evaluations runs)

(b) Fixed-checkpoint evaluations: seeds 42, 0, and 10.

Keeping each configuration's original trained checkpoint fixed, we ran two additional evaluations with seed=0 and seed=10, supplementing the original seed=42 evaluation for Tables 1 and 2 and Figures 5 and 7. RoboTwin evaluation comprises 10,000 test episodes. Below we report mean ± sample std over these three evaluation seeds (n=3, ddof=1). Success rates are in %, with std and differences in percentage points.

Architecture ablation (Table 1).

ArchitectureCleanRandomizedAverage
Single-system Vanilla85.000 ± 0.17885.827 ± 0.36185.413 ± 0.177
Single-system MoE86.093 ± 0.32983.120 ± 0.17484.607 ± 0.078
Dual-system Joint Self-Attention92.493 ± 0.24892.700 ± 0.27892.597 ± 0.226
Dual-system Joint Cross-Attention87.960 ± 0.24688.740 ± 0.08788.350 ± 0.140
Dual-system Detached Cross-Attention92.200 ± 0.20991.593 ± 0.06491.897 ± 0.072
Dual-system IDM87.527 ± 0.22188.020 ± 0.11187.773 ± 0.166
Tri-system Joint Self-Attention92.787 ± 0.30492.607 ± 0.23292.697 ± 0.134

Attention-mask ablation (Table 2).

Attention maskCleanRandomizedAverage
Isolated88.293 ± 0.18686.853 ± 0.12187.573 ± 0.148
Video Sees Action88.007 ± 0.28087.427 ± 0.20487.717 ± 0.085
Action Sees Video93.260 ± 0.24692.047 ± 0.22592.653 ± 0.235
Mutual92.413 ± 0.35892.047 ± 0.22592.230 ± 0.221

Denoising schedules (Figure 5). Synchronized denoising achieves 92.840 ± 0.260%, compared with the following asynchronous schedules:

Schedule familyParameterVideo-leadingAction-leading
Variance shiftα=491.773 ± 0.14291.073 ± 0.081
Variance shiftα=891.220 ± 0.13190.553 ± 0.140
Variance shiftα=1691.940 ± 0.06989.600 ± 0.322
Variance shiftα=3291.453 ± 0.08188.320 ± 0.156
Linear offseto=0.291.653 ± 0.08192.140 ± 0.354
Linear offseto=0.490.860 ± 0.25191.913 ± 0.186
Linear offseto=0.691.533 ± 0.13392.307 ± 0.250
Linear offseto=0.891.047 ± 0.18692.307 ± 0.250

Information flow before and after pretraining (Figure 7). ASV denotes Action Sees Video; Δ = Mutual − ASV, with its mean and std computed from the paired per-seed differences.

SettingASVMutualΔ (percentage points)
From scratch / RoboTwin-Full (Avg.)92.653 ± 0.23592.230 ± 0.221−0.423 ± 0.222
Pretrained / RoboTwin-Full (Avg.)92.513 ± 0.15192.600 ± 0.087+0.087 ± 0.070
Pretrained / RoboTwin-C2R (ID)86.720 ± 0.23687.467 ± 0.186+0.747 ± 0.064
Pretrained / RoboTwin-C2R (OOD)25.607 ± 0.23426.520 ± 0.140+0.913 ± 0.200

The observed success-rate standard deviations are at most 0.361 percentage points. Across all three evaluation seeds, the average-score rankings in Tables 1 and 2 remain unchanged, synchronized denoising outperforms every tested asynchronous schedule, and the preference for Mutual after pretraining is reproduced. These results support the consistency of the reported qualitative conclusions under evaluation-seed variation.

Thank you again for your careful and constructive review. We will incorporate these corrections, clarifications, and additional results into the final version of the paper.

Reviewer_Y91COpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
3(+1)
Presentation
2
Soundness
3(+1)
Read review
Paper summary

This paper presents OpenWAM, an open and modular framework for systematically studying World-Action Models (WAMs). The paper makes three main contributions. First, it introduces a unified infrastructure that supports different visual encoders, video backbones, model architectures, training strategies, and inference pipelines. Second, it conducts systematic ablation studies to investigate important WAM design choices, including visual representations, world-action interaction, and denoising strategies. Third, based on these findings, the authors pretrain OpenWAM-α on approximately 6,400 hours of human and robot data, achieving competitive performance across simulation and real-robot benchmarks.

Critical strengths
  1. One major contribution is the unified infrastructure for WAM research. Existing WAM implementations often couple different design choices together, making it difficult to understand which components actually contribute to performance. OpenWAM provides a modular framework for systematically comparing visual encoders, architectures, and inference strategies, which could be useful for future WAM research.

  2. The paper provides extensive empirical studies on WAM design choices. In particular, the findings on world-action interaction, synchronized denoising, and human-robot co-training are useful. The authors further validate these findings by scaling up to OpenWAM-α and evaluating across different robot embodiments.

Critical weaknesses
  1. As an infrastructure paper, I would expect a more detailed analysis of inference efficiency. The paper reports approximately 170 ms inference latency on an RTX 5090, but it would be better to clarify where the inference time is spent and which optimizations contribute most to the speedup. I encourage the authors to provide an incremental inference optimization table similar to Flex-π Table 10 [1], reporting action-only inference latency under different denoising steps (K=1, 2, 4, 10), together with fixed and per-step latency. It would also be interesting to compare action-only inference against joint video-action generation to better understand the tradeoff between inference efficiency and policy performance.

  2. I am mostly curious about which visual encoder works best for human-robot co-training. The paper compares different visual encoders, including Wan2.2-VAE, DINOv3, and V-JEPA 2.1. However, these experiments are mostly conducted under a restricted training setting rather than the full human-robot co-training setup. Interestingly, DINOv3 with S-VAE achieves performance very close to Wan2.2-VAE (90.18% vs. 90.30%). It remains unclear whether reconstruction-based encoders or semantic representation encoders are more effective for transferring knowledge from human videos to robot policies. A comparison under the same human-robot co-training setting would strengthen the paper.

[1] flex-π: a multi-stream world-action model with compute flexibility, arXiv:2608.10860

Questions for authors
  1. Could the authors provide a detailed inference optimization breakdown similar to Flex-π Table 10, including latency under different denoising steps, fixed overhead, and per-step cost? How much faster could OpenWAM be with action-only inference?

  2. Which visual encoder works best for human-robot co-training?

  3. Do the conclusions from the small-scale architecture ablations still hold at OpenWAM-α scale? In particular, are mutual world-action attention and synchronized denoising consistently better under large-scale co-training?

9 comments
AuthorsOpenReview ↗

Rebuttal for Reviewer Y91C – Question1

Q1. Inference optimization and the speed–performance trade-off of action-only inference.

We measured policy-call latency on one RTX 5090 in BF16, averaging the same 20 observations after warm-up. Timing spans observation preprocessing through action output, excluding simulator/network time and video pixel decoding. Joint inference still generates future video latents.

Joint inference configurationK=1K=2K=4K=10Fitted fixed costFitted cost/step
Eager68.5122.2229.0555.413.754.1
Compilation only54.193.7173.6412.514.239.8
DiT velocity cache only69.0123.6123.4231.667.316.4
Compilation + velocity cache, prompt re-encoded94.9134.8135.5216.493.212.3
Compilation + velocity cache + prompt cache54.693.994.6175.752.612.3

All values are in ms; the first three rows use cached prompt embeddings. Fixed and per-step costs are fitted using T(K) = fixed + K × per_step, where K counts scheduled denoising steps. With velocity caching, K=1, 2, 4, 10 executes 1, 2, 2, 4 DiT forwards, respectively.

At K=10, compilation alone saves 25.7% and velocity caching alone saves 58.3% relative to eager inference. In the same benchmark run, combining both reduces latency from 555.4 to 174.8 ms (3.18× faster). The last two rows separately isolate prompt caching: it saves 40.7 ms (18.8%), primarily reducing fixed overhead. These measurements use OpenWAM-α's RoboTwin-Full checkpoint and torch.compile(mode="default").

Action-only inference depends on the training setup. Flex-π trains with visual stream dropout: attention masks expose the action stream to different subsets of future visual streams, including none (§3.2, Appendix A.5). Its five-task RoboTwin ablation with RGB-only input reports 40.2% for action-only, 60.4% for action + video, and 63.8% for full joint generation (Figure 11b). It also reports Flex-π*, fine-tuned without stream dropout to improve LIBERO fitting (§4.4). Flex-π

OpenWAM-α uses mutual attention without this stream-dropout training. Direct action-only inference with the trained checkpoint reduces success on RoboTwin2.0-Clean2Random:

Evaluation splitJoint success (%)Action-only success (%)Drop (percentage points)
Clean89.462.826.6
Randomized48.715.133.6

For action-only latency, we benchmarked the isolated-mask checkpoint separately, with the same hardware, precision, and timing scope:

Action-only inference configurationK=1K=2K=4K=10Fitted fixed costFitted cost/step
Eager44.474.8136.2320.213.630.7
Compilation only42.170.3126.6295.813.828.2
Compilation + velocity cache42.070.271.4129.140.78.8
Compilation + velocity cache + prompt cache———68.3——

Values are in ms; “—” denotes unmeasured values. These latency results use a different checkpoint from the OpenWAM-α Clean2Random success-rate comparison above.

At K=10, the compilation + velocity cache + prompt cache configuration takes 68.3 ms for action-only (isolated checkpoint) versus 175.7 ms for joint inference (OpenWAM-α RoboTwin-Full checkpoint), corresponding to 61.1% lower latency (2.57× speedup).

In our paper, we also compares models trained with isolated versus mutual masks (Table 2, RoboTwin2.0-Full). Under the isolated mask, actions attend to the current observation without accessing future video tokens, providing the action-only counterpart:

Training maskClean success (%)Randomized success (%)
Isolated88.0886.74
Mutual92.5091.80

Mutual attention improves success by 4.42 and 5.06 percentage points, respectively.

Action-only inference offers substantial speed gains, but incurs a marked drop in policy success compared with joint inference under mutual attention.

AuthorsOpenReview ↗

Rebuttal for Reviewer Y91C – Question2 & 3

Q2. Visual encoder comparison under human–robot co-training.

We compared Wan2.2-VAE, DINOv3, and V-JEPA 2.1 using the same 600-hour pretraining mixture: 350 hours from EgoDex and 250 hours from RoboCOIN. All models used dual-system joint self-attention with the mutual mask and randomly initialized DiTs, excluding the influence of pretrained DiT weights. For DINOv3 and V-JEPA 2.1, we first trained an S-VAE on the 600-hour mixture, froze it, and then pretrained the WAM on the same mixture. Each model was subsequently fine-tuned on RoboTwin2.0 clean data for 5 epochs, with a batch size of 256, using 16 NVIDIA H200 GPUs.

Visual encoderClean success (%)Randomized success (%)
Wan2.2-VAE75.883.96
DINOv3 + S-VAE82.0234.24
V-JEPA 2.1 + S-VAE83.1243.40

Under this 600-hour co-training setup without DiT pretraining, both semantic representation encoders outperform the pixel-reconstruction encoder. V-JEPA 2.1 + S-VAE performs best, exceeding DINOv3 + S-VAE by 1.10 percentage points on Clean and 9.16 percentage points on Randomized, with the larger advantage on the OOD split.

Q3. Mutual attention and synchronized denoising at OpenWAM-α scale.

We trained an action-sees-video variant using the same pretraining data and training configuration as OpenWAM-α, and evaluated it on RoboTwin2.0-Clean2Random under the same fine-tuning protocol. Clean measures in-distribution (ID) performance, while Randomized measures out-of-distribution (OOD) performance.

Attention maskClean / ID success (%)Randomized / OOD success (%)
Action Sees Video82.745.8
Mutual (OpenWAM-α)89.448.7

Mutual attention improves success by 6.7 percentage points on Clean and 2.9 percentage points on Randomized.

We also evaluated OpenWAM-α with different inference-time denoising schedules on RoboTwin2.0-Full. Synchronized denoising achieves 93.6% success, compared with the following asynchronous schedules:

Denoising scheduleParameterVideo-leading success (%)Action-leading success (%)
Variance shiftα = 492.893.1
Variance shiftα = 893.092.5
Variance shiftα = 1692.493.0
Variance shiftα = 3292.892.4
Linear offsetoffset = 0.293.392.9
Linear offsetoffset = 0.493.192.8
Linear offsetoffset = 0.692.293.3
Linear offsetoffset = 0.892.592.8

Synchronized denoising outperforms all 16 asynchronous settings; the best asynchronous score is 93.3%, 0.3 percentage points below synchronized denoising. At OpenWAM-α scale, mutual world–action attention and synchronized denoising remain the best-performing choices among the evaluated configurations.

We sincerely thank you for your careful and thorough review. We will incorporate these clarifications and additional evaluation details into the final version of the paper.

~Hairuo_Liu1OpenReview ↗

The strange strange statistcs in randomized success rate for Wan2.2-VAE

I wonder if you could analyze this strange statistcs, because I'm also faced with this problem.

Comments on the paper

6 comments
~Royal_Westwater1OpenReview ↗

AI-hallucinated References

Warning:

3 problematic hallucinated references.

@article{jha2026reconstruction, author = {Saurav Jha and Artem Zholus and Sarath Chandar and others}, title = {Reconstruction or semantics? what makes a latent space useful for robotic world models}, journal = {arXiv preprint arXiv:2605.06388}, year = {2026} }

The citation leaves out the first author: Nilaksh. Also, "and others" suggests there are more authors after the third one, but the only missing author is the first one.

@article{lecun2022path, author = {Yann LeCun and others}, title = {A path towards autonomous machine intelligence version 0.9. 2, 2022-06-27}, journal = {Open Review}, volume = {62}, number = {1}, pages = {1--62}, year = {2022} }

The citation has a made-up co-author list. This position paper (OpenReview, id BZ5a1r-kVsf) has a single author, Yann LeCun. Also, "Open Review" is not a journal, and the volume 62, issue 1, pages 1–62 do not exist.

@article{allen2026physics, author = {Zeyuan Allen-Zhu}, title = {Physics of language models: Part 4.1, architecture design and the magic of canon layers}, journal = {Advances in Neural Information Processing Systems}, volume = {38}, pages = {42349--42369}, year = {2026} }

The publication year is wrong. Volume 38 of Advances in Neural Information Processing Systems is NeurIPS 2025.

Warning:

48 of the 107 references cut their author lists short with “et al.” / “and others”, often after only three to ten names, even when the full list is short and easy to find. Leaving author lists truncated like this is a typical pattern of AI-hallucinated references.

Area_Chair_MuLCOpenReview ↗

Discussion

Dear reviewers,

We have mixed signals from the review ratings

  • Y91C: 2: Weak rejection
  • VVgp: 3: Weak acceptance
  • kSmp: 4: Clear acceptance

Could you take a look at each other's reviews and the author responses and initiate a discussion to reach consensus? Were your concerns resolved and questions answered? How do you weigh the strengths and weaknesses?

Please note that ICLR 2027 has Policy on LLM use.

Best, Your AC

AuthorsOpenReview ↗

Response: the reference entries come from Google Scholar's citation export, not from AI (part1)

Thank you for checking our references so carefully.

We would like to clarify where these errors come from. The BibTeX entries in question, like the vast majority of our bibliography, were exported directly from Google Scholar using its "Cite → BibTeX" function. We did not use AI to generate or modify any reference. The truncated author lists, the missing author, and the incorrect bibliographic details you point out are all already present in the citation records that Google Scholar provides. Every cited work exists and is publicly available.

Below are the entries exactly as we obtained them from Google Scholar, together with an explanation of each error. Each entry is followed by a link to Google Scholar's "Cite" dialog for that paper; clicking "BibTeX" there reproduces the entry. Alternatively, search the paper title on Google Scholar and click "Cite" → "BibTeX".

1. jha2026reconstruction: missing first author

@article{jha2026reconstruction,
  title={Reconstruction or Semantics? What Makes a Latent Space Useful for Robotic World Models},
  author={Jha, Saurav and Zholus, Artem and Chandar, Sarath and others},
  journal={arXiv preprint arXiv:2605.06388},
  year={2026}
}

Google Scholar: Cite → BibTeX

Google Scholar's record of this arXiv paper omits the first author, Nilaksh (a single-name author), and appends "and others" after the other three. The arXiv listing has four authors: Nilaksh, Saurav Jha, Artem Zholus, and Sarath Chandar.

2. lecun2022path: journal, volume, issue, and pages

@article{lecun2022path,
  title={A path towards autonomous machine intelligence version 0.9. 2, 2022-06-27},
  author={LeCun, Yann and others},
  journal={Open Review},
  volume={62},
  number={1},
  pages={1--62},
  year={2022}
}

Google Scholar: Cite → BibTeX

For this position paper hosted on OpenReview, Google Scholar generates the journal "Open Review", volume 62, issue 1, and pages 1–62, and merges the version string ("version 0.9. 2, 2022-06-27") into the title. None of these correspond to a real journal publication. The document is a single-author position paper by Yann LeCun (OpenReview id BZ5a1r-kVsf).

3. allen2026physics: publication year

@article{allen2026physics,
  title={Physics of language models: Part 4.1, architecture design and the magic of canon layers},
  author={Allen-Zhu, Zeyuan},
  journal={Advances in Neural Information Processing Systems},
  volume={38},
  pages={42349--42369},
  year={2026}
}

Google Scholar: Cite → BibTeX

The volume (38) and page range (42349–42369) match the official NeurIPS proceedings; only the year is off. The proceedings of NeurIPS 2025 (Volume 38) were published online on 23 April 2026, which is presumably why Google Scholar records the year as 2026. The correct year is 2025.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.