acceptodds
Under review as a conference paper at ICLR 2027

WorldArena 2.0: Extending Embodied World Model Benchmarking on Modality, Functionality and Platform

Abstract

World models have emerged as a central paradigm for embodied intelligence, enabling agents to predict future observations and reason about environmental dynamics. However, existing embodied world model benchmarks are still largely confined to vision-only prediction, offline embodied applications, and simulator-based evaluation, making them insufficient for assessing increasingly comprehensive world models. In this work, we introduce WorldArena 2.0, an expanded benchmark that systematically broadens embodied world model evaluation along three dimensions: modality, functionality, and platform. Along the modality dimension, it extends evaluation from vision-only to visuotactile modalities, enabling assessment of multimodal prediction and manipulation. Along the functionality dimension, it extends beyond policy evaluation and action planning to assess world models as interactive RL environments for policy optimization. Along the platform dimension, it moves beyond simulator-only evaluation to a diverse suite of simulated and real-world robotic settings across multiple embodiments. Under a standardized protocol, WorldArena 2.0 comprehensively evaluates perceptual quality, interactive utility, and cross-platform performance, providing a comprehensive testbed for tracking progress toward embodied world models. The benchmark is available at: https://worldarena2.github.io/.

Primary area:
datasets and benchmarks
Keywords:
World Model, Embodied Intelligence, Benchmark

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_LGLJOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces WorldArena 2.0, extending embodied world-model evaluation along three dimensions: visuotactile modeling, world models as reinforcement learning environments, and evaluation across simulated and physical platforms. The benchmark combines perceptual metrics with downstream task performance, including policy optimization within learned environments, synthetic-data generation, and action planning. Its experiments highlight discrepancies between prediction quality and functional utility.

Critical strengths
  • The evaluation targets practically relevant capabilities beyond visual quality. Incorporating tactile sensing, policy improvement through learned environments, and physical execution makes the benchmark relevant to embodied applications. These extensions address meaningful limitations of evaluations centered primarily on video generation.

  • The experiments provide useful diagnostic observations. The discrepancies between tactile reconstruction quality and manipulation success, together with uneven functional performance across platforms, illustrate why perceptual metrics alone are insufficient. Evaluating seven world models under three reward designs also provides useful evidence about reward sensitivity.

Critical weaknesses
  • The world-model-based RL evaluation omits important reference methods. Although the paper evaluates multiple world models within a shared policy-optimization framework, it does not experimentally compare against WMPO [1] or VLA-MBPO [2]. WMPO is cited but not evaluated. Both methods directly address VLA policy improvement through learned world models, making them particularly relevant to the benchmark’s functionality extension. The standardized framework is useful for component comparisons, but it does not establish how the evaluated pipelines compare with existing systems designed specifically for this purpose. A complementary system-level comparison under comparable data and computational budgets would substantially strengthen this part of the benchmark.

  • The protocol does not sufficiently disentangle world-model quality from rollout configuration and computational budget. Appendix C.6 states that action chunk sizes, step limits, and batch sizes vary across models, while the main comparison fixes optimization iterations. Consequently, equal iteration counts do not imply equal simulated experience, recursive prediction depth, or computational expenditure. Appendix C.7 reports per-iteration times ranging from 7.8 to 28.8 minutes. Preserving native temporal interfaces is reasonable, but the resulting rankings reflect the complete configured training pipelines, with limited evidence about which differences arise from the world models themselves. Comparisons under matched compute and matched interaction budgets, together with rollout-horizon sensitivity analyses, are needed to make these rankings more interpretable.

  • The quantitative coverage and statistical evidence are insufficient to establish reliable benchmark rankings. The main RL evaluation covers only two RoboTwin tasks, uses a single training initialization seed, and disables domain randomization during checkpoint evaluation. Moreover, Appendix C.3 reports the highest success rate observed over repeated evaluations on the same held-out configurations. This measures peak validation performance and can introduce selection optimism; it does not establish performance on an independent final test set. Physical comparisons are also based on only 10–20 trials per condition. The reported Wilson intervals and explicit limitations are helpful, but they do not capture training variability or checkpoint-selection effects. Broader task coverage, multiple training seeds, and independent evaluation of selected checkpoints are needed to support stable comparative conclusions.

[1] Fangqi Zhu, Zhengyang Yan, Zicong Hong, Quanxin Shou, Xiao Ma, and Song Guo. WMPO: World Model-based Policy Optimization for Vision-Language-Action Models. arXiv:2511.09515, 2025.

[2] Zhilong Zhang, Haoxiang Ren, Yihao Sun, Yifei Sheng, Haonan Wang, Haoxin Lin, Zhichao Wu, Pierre-Luc Bacon, and Yang Yu. Towards Practical World Model-based Reinforcement Learning for Vision-Language-Action Models. arXiv:2603.20607, 2026.

Questions for authors
  • Can the authors include WMPO and VLA-MBPO as reference systems in the learned-environment RL evaluation, with clearly specified adaptation procedures and comparable resource budgets?

  • Do the relative rankings persist when comparing equal wall-clock budgets or equal amounts of generated interaction data?

  • How do the results change when checkpoints are selected on validation configurations and evaluated separately on unseen configurations across multiple training seeds?

6 comments
AuthorsOpenReview ↗

Response (1/4): System-level reference methods

W1. System-level comparisons with WMPO and VLA-MBPO

We thank the reviewer for this suggestion. We adapted the core optimization mechanisms of WMPO and VLA-MBPO to Ctrl-World, Wan, and OpenSora on Adjust Bottle, covering six configurations and 47 valid checkpoints.

All configurations share the same fine-tuned π0.5\pi_{0.5} initialization, single head-camera view, 14-dimensional absolute joint actions, frozen world models, and proxy reward. Native action chunk sizes are retained (Ctrl-World, 5; Wan and OpenSora, 8). Each checkpoint is evaluated over 1,024 episodes in the ground-truth RoboTwin simulator.

  • WMPO adaptation. Trajectory-level proxy returns, within-group GRPO (G = 4), and five denoising steps. Generated experience from skipped homogeneous groups is included in the budget.
  • VLA-MBPO adaptation. Two-chunk branches from real intermediate states, chunk-discounted GAE, bootstrapping at branch truncation, PPO, and three denoising steps. Ctrl-World retains updates to some visual layers; Wan and OpenSora freeze the vision-language backbone.
AdaptationWorld ModelFull-budget peak validation (%)Peak at ≤38,400 actions (%)
WMPO (proxy)Ctrl-World65.3365.14
WMPO (proxy)WoVR(Wan)61.4361.43
WMPO (proxy)OpenSora61.4361.43
VLA-MBPO (single-view)Ctrl-World64.9464.94
VLA-MBPO (single-view)WoVR(Wan)63.7763.77
VLA-MBPO (single-view)OpenSora64.9464.94

The results support three observations.

  1. Policy improvement extends across optimization methods. All six configurations achieve 61.43% to 65.33% success, exceeding SFT (55.08%). Training in learned world models improves policies under both long-trajectory on-policy GRPO and short-branch actor-critic PPO.
  2. The benchmark's default pipeline is competitive. With Ctrl-World and proxy reward, our RLinf-based GRPO pipeline achieves 70.70% (three-seed mean, 69.53%), compared with 65.33% for adapted WMPO and 64.94% for adapted VLA-MBPO. It also uses less training time and fewer training iterations. RLinf provides a unified interface that facilitates integrating additional world models and optimization methods.
PipelineTotal generated actionsRecorded wall timeRecorded GPU-hours
RLinf-GRPO / Proxy, Ctrl-World64,0004.94 h16.09
WMPO / Ctrl-World64,00019.26 h61.78
VLA-MBPO / Ctrl-World51,20016.99 h135.94
  1. Backend performance interacts with the optimization paradigm. OpenSora matches Ctrl-World under short-branch VLA-MBPO (64.94%), while its long-trajectory WMPO result is lower (61.43% versus 65.33%). This highlights long-horizon recursive prediction stability as a distinguishing factor among world models and shows that the benchmark accommodates different policy-optimization methods.

The updated manuscript cites WMPO [1] and VLA-MBPO [2] and identifies these implementations as adaptations under a unified interface.

[1] Fangqi Zhu, Zhengyang Yan, Zicong Hong, Quanxin Shou, Xiao Ma, and Song Guo. WMPO: World Model-based Policy Optimization for Vision-Language-Action Models. arXiv:2511.09515, 2025.

[2] Zhilong Zhang, Haoxiang Ren, Yihao Sun, Yifei Sheng, Haonan Wang, Haoxin Lin, Zhichao Wu, Pierre-Luc Bacon, and Yang Yu. Towards Practical World Model-based Reinforcement Learning for Vision-Language-Action Models. arXiv:2603.20607, 2026.

AuthorsOpenReview ↗

Response (2/4): Matched budgets and rollout-horizon sensitivity

W2. Rollout configurations, interaction volume, computational budgets, and horizon sensitivity

We agree that a fixed iteration count does not imply equal interaction or computational budgets. We therefore added comparisons along three dimensions: interaction volume, computation time, and recursive prediction depth.

  1. Matched interaction budget (≤38,400 generated low-level actions). We recorded cumulative generated actions, world-model queries, and optimization updates for each checkpoint. The peak success rates within this budget are:
Optimization methodCtrl-WorldWanOpenSora
WMPO adaptation65.14%61.43%61.43%
VLA-MBPO adaptation64.94%63.77%64.94%

Under both methods, backend rankings match those obtained from the peaks over the full training runs.

  1. Matched computational budget (wall-clock time / GPU-hours).
  • VLA-MBPO: All three backends use eight GPUs and the same timing convention. Within a 43–48 GPU-hour window, Ctrl-World achieves 64.94% (43.7 GPU-hours) = OpenSora 64.94% (47.6 GPU-hours) > Wan 63.77% (43.2 GPU-hours).
  • WMPO: Time per training step is 1.93 h for Ctrl-World, 0.50 h for OpenSora, and 0.32 h for Wan, differing by approximately sixfold. Within a wall-clock window of approximately 3.45 h, Ctrl-World achieves 64.75% > Wan 61.43% = OpenSora 61.43%.

The relative ordering of the three backends remains unchanged when interaction volume and computation time are matched separately. Although Ctrl-World is approximately six times slower per step, it still leads within the same time budget. The ranking is therefore not caused by budget differences.

  1. Recursive-depth (horizon) sensitivity. Action chunking bounds the number of recursive world-model generations by ⌈H/K⌉ (Appendix C.6). On Adjust Bottle with the proxy reward, we set the training horizon H of Ctrl-World and Wan to 0.5× (Short), 1× (Normal), and 2× (Long) their paper defaults. We matched the number of optimization updates and total interaction budget as closely as possible, keeping all other settings unchanged:
ModelConfigurationTraining HChunkRecursive depthCumulative trajectoriesAction chunksOptimization updatesPeak success
Ctrl-WorldShort1005201282,560869.63%
Ctrl-WorldNormal (paper)200540642,560870.70%
Ctrl-WorldLong400580322,560859.47%
Wan(WoVR)Short1208151281,920864.94%
Wan(WoVR)Normal (paper)240830641,920867.19%
Wan(WoVR)Long480860321,920855.18%

Each checkpoint is evaluated over 1,024 episodes. The three horizon settings yield three findings:

  1. The default horizon is optimal for both backends. Both attain their highest peaks under Normal (1×): 70.70% for Ctrl-World and 68.65% for Wan. The horizon choices in the main table do not favor a particular model.
  2. The relative ordering is preserved across all three horizons and is not driven by interaction volume. Ctrl-World's peaks exceed Wan's by 4.7, 2.1, and 4.3 percentage points under Short, Normal, and Long, respectively. Under Normal, Wan generates more actions than Ctrl-World because of their different native chunk lengths (15,360 versus 12,800), yet Ctrl-World still leads.
  3. Excessive recursive depth exposes differences in long-horizon stability. Doubling recursive depth reduces the peaks of both backends by 11–13 percentage points. With continued training under Long, Wan's success decreases from 55.18% to 35.84%, below SFT's 55.08%, while Ctrl-World decreases from 59.47% to 55.96%, a much smaller decline. This agrees with the divergence under long-trajectory WMPO in W1 and demonstrates the need to test world models' long-horizon stability through interactive RL.

Table 2 reports each world model's utility under its native temporal interface and a fixed iteration budget. The updated manuscript now uses this description and includes the three sets of comparisons above.

AuthorsOpenReview ↗

Response (3/4): Training variability and independent evaluation

W3. Training randomness, checkpoint selection, and evaluation on independent configurations

  1. Multiple seeds and checkpoint-selection effects. We added training seeds 2345 and 3456 for Ctrl-World and WoVR on Adjust Bottle, each with 20 checkpoints × 1,024 evaluation episodes. Together with the original seed 1234, this gives three seeds. Per-seed results are provided in our response to Reviewer prVG's W1.
  • Multiple seeds. The three-seed mean peaks are 69.53% for Ctrl-World and 67.38% for WoVR under the proxy reward, and 65.43% and 61.50%, respectively, under the similarity reward. The ordering is consistent and holds for every seed (6/6), with both models well above SFT (55.08%).
  • Selection effects. Each checkpoint is evaluated over 1,024 episodes, giving a standard error of approximately 1.5 percentage points per estimate. The estimated upper bound on the optimism from selecting the peak among 20 checkpoints is approximately 3 percentage points. Without selection, the last checkpoints (step 20) under the proxy reward achieve 66.31%/63.18% for Ctrl-World and 60.06%/66.50% for WoVR, remaining 5–11 percentage points above SFT. In the updated manuscript, we have renamed the metric peak validation performance and report peak and last-checkpoint results in separate tables.
  1. Independent evaluation on unseen configurations and under paired appearance randomization. To eliminate peak-selection bias and address the absence of domain randomization during evaluation, we evaluated the peak checkpoint selected on validation configurations for each seed on two sets of test configurations that had never been used for training or peak selection, with 1,024 episodes per set:
  • Test set A (unseen initial configurations): Newly sampled initial layouts outside the validation configurations.
  • Test set B (paired appearance robustness test): Physical initial states are paired scene by scene with the validation configurations. Only the background-wall and tabletop textures are independently sampled from RoboTwin's unseen texture library, and the colors and intensities of one directional light and two point lights are randomized. Appearance is determined by the scene ID, so all policies encounter exactly the same scenes, and changes in success rates arise solely from visual appearance.
ModelUnseen initial configurations (per seed)MeanAppearance randomization (per seed)Mean
SFT49.1249.1230.1830.18
Ctrl-World62.21 / 60.45 / 59.6760.7739.94 / 44.63 / 44.0442.87
WoVR52.64 / 53.22 / 53.8153.2240.72 / 45.21 / 42.6842.87

Success rates are percentages; the seed order is 1234 / 2345 / 3456. The results yield three findings:

  1. Policy gains persist on independent configurations. All 12 results across two models, three seeds, and two test sets exceed SFT. On unseen initial configurations, Ctrl-World's mean exceeds SFT by 11.7 percentage points and WoVR's by 4.1 percentage points (pooling three seeds, one-sided Fisher's exact test: p = 0.013).
  2. Gains are larger under visual distribution shift. With unseen textures and lighting, SFT drops to 30.18%, while Ctrl-World and WoVR both achieve a mean of 42.87%, each exceeding SFT by 12.7 percentage points (p < 10⁻¹²). Because physical states are paired scene by scene, this gap shows that training within world models does not cause policies to overfit the appearance of validation configurations and instead improves robustness to visual shifts.
  3. Peak-selection bias agrees with the estimate. The unseen initial configurations are themselves harder: even SFT, which involves no selection, drops from 55.08% to 49.12%. After accounting for this 6.0-percentage-point difficulty difference, the additional decrease in Ctrl-World's mean peak is approximately 2.8 percentage points (69.53% → 60.77%), consistent with the estimated upper bound of approximately 3 percentage points.

The leading-model ordering is clearer on unseen initial configurations: all three Ctrl-World seeds (minimum 59.67%) outperform all three WoVR seeds (maximum 53.81%). Under appearance randomization, the two models tie. Accordingly, we characterize Table 2 as peak validation performance and use the independent tests in this section as the basis for generalization conclusions.

  1. Real-robot validation. The tactile ablation uses identical training and evaluation settings, with n = 20 trials per task and condition. FastWAM improves from 3/20 to 9/20 on Chip and from 4/20 to 6/20 on Wiping. In this rebuttal, we additionally performed tactile ablations on Wiping and Tofu using three visuotactile models. Removing tactile input reduces success in all six comparisons, with a pooled one-sided Fisher's exact test giving p = 0.007 (see our response to Reviewer prVG's W3).
Reviewer_prVGOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper extends WorldArena in modality, functionality, and platform coverage. It adds visuotactile prediction to video world models and evaluates tactile quality and task success in simulation and on a real robot. It also uses seven action-conditioned world models as RL environments to fine-tune π0.5 with GRPO under three reward models, then evaluates the policies in the ground-truth simulator and a small real-robot pilot. Finally, it applies WorldArena’s evaluation protocols and 16 perceptual metrics to RoboTwin, LIBERO, and an AgileX ALOHA robot.

The results show that WM-based RL improves over SFT but remains below simulator-based RL. Perceptual quality does not reliably predict control performance, and gains in simulation transfer poorly to real robots.

Critical strengths
  1. Tactile extension. The paper provides a shared pipeline for adapting video WMs to visuotactile prediction and shows that tactile quality does not necessarily predict task success. On the real robot, tactile input improves FastWAM’s average success from 17.5% to 37.5%. This is the paper’s most novel contribution.
  2. Evaluation of WMs as RL environments. The paper compares seven WMs and three reward families under a shared GRPO pipeline, then tests the trained policies in the ground-truth simulator. Real-robot experiments provide an initial check on transfer. These shared protocols are my main reason for leaning toward acceptance.
  3. Broad evaluation and candid reporting. The paper covers simulation and real robots, reports uncertainty, and discusses failures, including poor real-world transfer and tasks where all WM-trained policies fail. These findings help reveal gaps between perceptual quality and usefulness for control.
Critical weaknesses
  1. RL results use a single seed and report the peak checkpoint. The peak is selected on the same evaluation episodes it is reported on (Table 7), which is upward-biased. With gaps of only a few points between WMs (e.g., 60.16 / 61.33 / 63.48), the fine-grained rankings may not be meaningful.
  2. The "robust ranking" claim is overstated. The cross-reward Spearman ρ (up to 0.957) is computed from the same single-seed peak values, pools two tasks, and is inflated by replicated baselines.
  3. Physical trials are too few to support the conclusions drawn from them. With n = 10–20 per cell and most results at 0–30%, many real-robot comparisons have overlapping CIs (e.g., tactile Wiping 20% → 30%).
  4. The visuotactile results may reflect the adapter rather than the WMs. All WM-based systems score 0% on Lift Bottle while ACT reaches 80%. The future-information ablation uses ground-truth futures, so it does not show that success tracks each WM's predicted dynamics.
  5. Task coverage per extension is thin. Each extension rests on 1–2 tasks (2 UniVTAC, 2 RoboTwin RL, 1 LIBERO, 2 real), which limits how general the benchmark's conclusions are.
Questions for authors
  1. How many evaluation episodes and checkpoints underlie each Table 2 entry? Can you report last-checkpoint numbers with ≥3 seeds, at least for the top WMs?
  2. Do the WM rankings stay concordant across reward designs when computed per task, from multi-seed results, and without the replicated baselines?
  3. Can you increase the physical trial counts, or report paired tests, for the key real-robot comparisons such as tactile vs. no tactile?
  4. Why do all WM-based visuotactile systems score 0% on Lift Bottle while ACT reaches 80%? Does a per-WM decoder fed ground-truth future visuotactile latents succeed, which would separate WM quality from the adapter?
  5. Do the main conclusions hold on additional tasks, especially in the tactile and RL extensions?
5 comments
AuthorsOpenReview ↗

Response (1/4): Training seeds, checkpoint selection, and reward rankings

W1. Training seeds, peak selection, and last-checkpoint performance

We thank the reviewer for the careful reading. We agree that selecting the peak on the same evaluation episodes introduces optimism. In the updated manuscript, we have renamed this metric peak validation performance and added three types of evidence.

Magnitude of the bias. Each checkpoint is evaluated over 1,024 episodes, giving a standard error of approximately 1.5 percentage points for an individual estimate. The upper bound on the optimism introduced by selecting the maximum among 20 checkpoints is approximately 3 percentage points. All world models follow the same peak-selection protocol, so this bias does not systematically favor any particular model or affect relative comparisons. In contrast, the leading models improve over SFT by 12–31 percentage points (Adjust Bottle: 55.08% → 67.19%/70.70%; Click Bell: 43.75% → 69.53%/75.00%), well above this bias. The 60.16/61.33/63.48 results highlighted by the reviewer are indeed within the noise range. We have therefore grouped the middle-performing models into the same tier and no longer rank them within that tier.

Multiple seeds. We added training seeds 2345 and 3456 for Ctrl-World and WoVR on Adjust Bottle, each with 20 checkpoints × 1,024 evaluation episodes. Together with the original seed 1234, this gives three seeds:

World modelRewardOriginal s1234New s2345New s34563-seed peak mean ± sample SDLast: two new seeds
Ctrl-Worldproxy70.769.4368.4669.53 ± 1.1266.31 / 63.18
Ctrl-Worldsimilarity66.0268.0762.2165.43 ± 2.9753.32 / 47.56
WoVR (Wan)proxy67.1968.2666.767.38 ± 0.8060.06 / 66.50
WoVR (Wan)similarity61.3561.6261.5261.50 ± 0.1457.91 / 55.76

The three-seed mean peak success rates are 69.53% for Ctrl-World and 67.38% for WoVR under the proxy reward, and 65.43% and 61.50%, respectively, under the similarity reward. The standard deviations of the peaks across seeds range from 0.14 to 2.97 percentage points. Across all three seeds and both rewards, Ctrl-World's peak exceeds WoVR's (6/6), and both remain well above the SFT baseline of 55.08%.

Last checkpoints without selection. Under the proxy reward, success rates at step 20 for the two additional seeds are 66.31%/63.18% for Ctrl-World and 60.06%/66.50% for WoVR, all 5–11 percentage points above SFT. These results show that policy improvement persists without peak selection. The last checkpoints also reveal a pattern obscured by the peak metric: under the similarity reward, Ctrl-World degrades late in training (53.32%/47.56%), while it remains stable under the proxy reward. This provides direct evidence that reward design affects training stability, motivating our reporting of peak and last-checkpoint results side by side.

All policies are evaluated using the ground-truth simulator's success criteria; learned rewards are not used for scoring.

W2. Reward-ranking consistency by task

Following the reviewer's suggestion, we recomputed the correlations separately for each task, retaining only the seven learned world models and excluding the SFT and simulator-RL baselines.

TaskProxy–VLM ρProxy–Similarity ρVLM–Similarity ρ
Click Bell0.96430.96430.9286

The correlations remain positive after separating the tasks and show task dependence. On Click Bell, rankings under the three rewards are nearly identical, with no model shifting by more than one position. Across all six task × reward combinations, WoVR and Ctrl-World consistently occupy the top two positions, and their order within each task remains unchanged. On Adjust Bottle, this order also agrees with the three-seed means (see the table in W1), showing that the ranking of the leading models also holds across multiple seeds.

Accordingly, we have revised the main-text statement to: “The leading models maintain consistent relative performance under the evaluated reward designs, while middle-performing models are more sensitive to the task and reward.”

AuthorsOpenReview ↗

Response (2/4): Real-robot evidence and extended tactile ablations

W3. Real-robot sample sizes and extended tactile ablations

The real-robot experiments support two types of conclusions, which have different sample-size requirements.

Tactile ablation (B.4). The two conditions use identical training and evaluation settings: 200 training episodes, an action chunk of 25, a maximum of 600 steps, no human intervention, and n = 20 trials per task and condition. On Chip, success increases from 3/20 to 9/20, a gain of 30 percentage points and a threefold increase in successful trials; the one-sided Fisher's exact test gives p = 0.041. We use Chip as the primary evidence for the benefit of tactile input. Wiping shows the same direction of improvement (4/20 → 6/20). We agree that the intervals overlap when this comparison is considered alone, so we added the multi-model experiments below. Success on Chip depends on controlling contact force on a fragile object, precisely a setting in which vision provides limited observability and tactile sensing is expected to help. The result is consistent with this expectation.

Extended tactile ablations. To test whether the benefit of tactile input is specific to FastWAM, we conducted the same ablation on three visuotactile models: N0-VTLA [a], N0-TWAM [b], and FTP-1 [c]. The tasks are Wiping and Tofu. Tofu requires transferring tofu from a plate to a bowl. Because tofu is fragile, excessive grasping force crushes it, while insufficient force causes it to slip; tactile information is needed to control contact force. The experimental settings are identical to B.4, with 20 trials per task and condition.

ModelWiping with tactileWiping without tactileWiping ΔTofu with tactileTofu without tactileTofu Δ
N0-VTLA15 (3/20)10 (2/20)−565 (13/20)30 (6/20)−35
N0-TWAM55 (11/20)30 (6/20)−2530 (6/20)25 (5/20)−5
FTP-145 (9/20)30 (6/20)−1545 (9/20)35 (7/20)−10
Average38.323.3−15.046.730−16.7

Success rates are reported as percentages. Removing tactile input reduces success in all six comparisons across three models and two tasks, with a mean decrease of 15.8 percentage points. Pooling successful trials across the six comparisons (51/120 versus 32/120), a one-sided Fisher's exact test gives p = 0.007. Including FastWAM on Chip and Wiping, tactile input improves success in all eight comparisons across four models and three tasks; a one-sided sign test gives p = 0.004.

The Wiping comparison highlighted by the reviewer is thus independently corroborated: removing tactile input from the three additional models reduces success by 5, 25, and 15 percentage points, respectively, consistent with the direction observed for FastWAM. On Tofu, N0-VTLA shows the largest decrease (65% → 30%). Without tactile input, failures mainly involve uncontrolled force, causing the tofu to be crushed or to slip during transfer.

[a] N0-VTLA: Scaling Vision-Tactile-Language-Action Model with Latent Tactile Tokens. arXiv:2607.23782, 2026.

[b] N0-TWAM: Scaling Tactile-Native World-Action Model for Contact-Rich Manipulation. arXiv:2607.23783, 2026.

[c] FTP-1: A Generalist Foundation Tactile Policy across Tactile Sensors for Contact-Rich Manipulation. arXiv:2606.13102, 2026.

Cross-platform diagnosis (Table 3, n = 10). The conclusion here is qualitative: functional utility in simulation does not transfer to real robots. A sample size of n = 10 is sufficient to support this assessment; for example, the upper bound of the 95% Wilson interval for 0/10 is 27.8% (Table 10). We do not derive fine-grained model rankings from these cells, as explicitly stated in Appendix D.5. Sample sizes and Wilson intervals for each condition are reported in Tables 5, 6, and 10.

AuthorsOpenReview ↗

Response (3/4): Decoder attribution, task coverage, and evaluation protocol

W4. World-model and decoder attribution

Downstream success measures the complete world-model–decoder system, as stated in Appendix B.2. We have aligned the corresponding statements in Section 3.2 with this interpretation. Within this scope, three pieces of evidence indicate that the results are not dominated by the adapter:

  1. The decoder is a controlled factor. Decoder architectures and training settings are identical across world models; only the latents supplied by the world model vary. Success rates on Insert HDMI (Wan2.2: 100%; Vidar: 70%; Genie Envisioner: 0%) follow the same order as the models' contact-timing coverage (97.2%, 95.3%, and 87.6%), while frame-level SSIM follows a different order.
  2. The decoder uses future information. On Adjust Bottle, supplying ground-truth future observations yields 66% success, compared with 32% when future inputs are removed and 33% when their temporal order is shuffled (Appendix C.9).
  3. The adapter does not fail universally. The same decoder pipeline achieves up to 100% success on Insert HDMI. On Lift Bottle, all three systems score 0%, while ACT reaches 80%, showing that the task is solvable and that failures are concentrated in the specific mechanism of sustained grasping over a long horizon.

The reviewer's proposed experiment—feeding ground-truth future visuotactile latents into each WM's decoder—would further localize the source of failure on Lift Bottle. The ground-truth-future ablation has so far been completed only on Adjust Bottle. We therefore describe the Lift Bottle result as a shared failure of the complete systems.

W5. Task coverage across benchmark extensions

The main tables select one or two tasks with complementary physical mechanisms for each dimension, allowing 12 models to be compared under controlled, matched conditions within a feasible computational budget. Table 2 alone involves 42 independent RL training runs. Beyond the two real-robot tasks in Table 3, the paper includes two tactile-ablation tasks in B.4, Lift Bottle in C.10, and the nine-task extended suite in D.3. The paper's main conclusions concern patterns that recur across models, platforms, and rewards, rather than rankings on a single task. The detailed argument is provided in our response to Reviewer 334i.

Q1. Evaluation episodes, checkpoints, and multi-seed last-checkpoint results

Each entry in Table 2 is obtained from 20 checkpoints (steps 1–20) of one training run. Each checkpoint is evaluated over 1,024 episodes in the ground-truth simulator, and the highest success rate is reported.

WoVR and Ctrl-World occupy the top two positions in all six task × reward combinations in Table 2. Their mean success rates across these combinations are 68.2% and 68.0%, respectively, approximately 7 percentage points above the third-ranked model, Cosmos-Predict-2.5 (61.3%). We therefore concentrated the computational budget for multiple seeds on these two leading models, whose relative order is also the ranking comparison most in need of a stability assessment.

Their last-checkpoint results on Adjust Bottle are as follows:

World ModelRewardSeed 2345Seed 3456
Ctrl-WorldProxy66.31%63.18%
WoVRProxy60.06%66.50%
Ctrl-WorldSimilarity53.32%47.56%
WoVRSimilarity57.91%55.76%

Under the proxy reward, all last-checkpoint results exceed SFT (55.08%). Under the similarity reward, Ctrl-World degrades late in training, while WoVR remains more stable. Per-seed peaks and their means are provided in the table in W1.

Reviewer_334iOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

WorldArena 2.0 extends the existing WorldArena on visuotactile prediction, policy optimization inside learned environments, and evaluation across simulated and physical robotic platforms. For visuotactile evaluation, it augments three visual world models with tactile encoding, joint prediction, and action decoding, then evaluates prediction quality and manipulation success on two UniVTAC tasks. For policy optimization, it compares seven learned environments under three reward designs on two RoboTwin tasks, evaluating the resulting policies using the ground-truth simulator’s success criterion. Additional experiments compare generated-data and action-planning utility across RoboTwin, LIBERO, and an AgileX robot, alongside separate physical tactile and policy-transfer studies.

Critical strengths

The benchmark adds a useful evaluation capability that compares world models as environments for policy improvement. The seven-model, three-reward comparison evaluates whether training inside a learned environment improves policy performance under ground-truth simulator dynamics. This is the strongest point because it provides a framework to studying an important use of world models.

The tactile and physical evaluations expose practical limitations that visual prediction scores alone would miss. All three visuotactile world-model systems fail bottle lifting while the tactile ACT baseline achieves 80% success, and higher tactile SSIM does not consistently correspond to better manipulation performance (Table 1). These observations identify concrete weaknesses and provide challenging evaluation settings for future methods. Making such failures measurable and comparable is itself a meaningful contribution.

Critical weaknesses

The main contribution is the framework. The tactile experiments use the existing UniVTAC dataset, and the RL experiments collect trajectories from existing RoboTwin tasks. No new datasets is provided. The RL and simulated tactile evaluations each contain only two tasks, while the LIBERO evaluation contains one. My feeling is that this is not sufficient to provide generalizable insights.

Questions for authors

I think the authors should address the concern of generazibility and provide clear reasoning why there aren't ways to build simulations that can support a broader set of tasks for dataset and benchmarking.

3 comments
AuthorsOpenReview ↗

Response: Benchmark contribution, task coverage, and generalizability

W1. Benchmark contribution and task coverage

We thank the reviewer for the comments. WorldArena 2.0 contributes an evaluation protocol that integrates visuotactile prediction, world models as RL training environments, and real-robot execution into a unified framework, evaluating 12 world models through their downstream functional utility. Using UniVTAC and RoboTwin 2.0 is a deliberate design choice: all models share the same data sources, physical dynamics, and success criteria, allowing differences between models to be attributed to the world models themselves. This also enables the community to reproduce the evaluation directly on familiar public benchmarks.

The paper’s conclusions are supported by three levels of evidence:

  1. Controlled comparisons. The WM-RL evaluation covers 7 world models × 3 reward designs × 2 tasks, yielding 42 independent training runs, with each checkpoint evaluated over 1,024 episodes in the ground-truth simulator. The cross-platform evaluation covers 6 models × 3 platforms × 2 functional protocols (Table 3).

  2. Real-robot evidence. Beyond the two real-robot tasks in Table 3, Appendix B.4 reports a tactile ablation under identical training and evaluation settings, with n = 20 trials per task and condition. Adding tactile input to FastWAM increases success on Chip from 15% to 45% and average success across the two tasks from 17.5% to 37.5%. In the real-robot Lift Bottle experiment in Appendix C.10, policies trained within WoVR and Ctrl-World achieve multi-stage task progress of 25% and 30%, respectively, compared with 10% for SFT. Appendix D.3 additionally provides an extended suite of 9 real-robot tasks—6 vision-only and 3 visuotactile—covering instruction-conditioned selection, deformable-object manipulation, precision insertion, and sustained tool contact. Detailed results are available in Track 3 of the leaderboard linked from the paper.

  3. Additional experiments in this rebuttal. We added two training seeds for Ctrl-World and WoVR on Adjust Bottle and adapted two optimization methods, WMPO and VLA-MBPO, to three world-model backends, yielding 6 configurations in total. Details are provided in our responses to Reviewers prVG and LGLJ.

Our conclusions concern patterns that recur across different settings, rather than fine-grained rankings on an individual task:

  • Frame-level metrics cannot predict control utility. On Insert HDMI, Genie Envisioner achieves higher SSIM than Vidar (0.456 versus 0.278), yet their success rates are 0% and 70%, respectively. On RoboTwin, Veo 3.1 and Wan 2.6 achieve the highest visual quality, but their trajectory accuracy is less than one-third that of Ctrl-World.

  • Training within learned world models consistently improves policies. All 42 configurations outperform the SFT baseline. This finding also holds under the additional training seeds and the two additional optimization methods evaluated in this rebuttal.

  • Simulation performance cannot be extrapolated to real robots. Task-success correlations are weak between RoboTwin and real-world evaluation and between LIBERO and real-world evaluation (ρ = 0.348 and 0.522, respectively). Meanwhile, model rankings on perceptual dimensions are consistent across platforms; for example, Physics Adherence has a RoboTwin–Real correlation of ρ = 0.811 (p = 0.001).

Each of these three patterns is corroborated across multiple models, platforms, or reward designs, providing the basis for their value to the community.

Q1. Extending task coverage under computational constraints

The simulators can support more tasks; the constraint comes from the computational cost of the evaluation itself. WorldArena 2.0 provides task-agnostic interfaces that already integrate RoboTwin 2.0, LIBERO, and UniVTAC. Adding a task requires providing trajectory data and a success criterion.

The cost arises from evaluating world models as RL environments through our RLinf-based implementation. Each additional task requires collecting 3,000 trajectories, adapting 7 world models and the reward models, and completing 21 RL training runs. Each run involves 20 checkpoints, with 1,024 evaluation episodes per checkpoint. A single RL iteration takes 7.8–28.8 minutes on 2×A800 GPUs (Appendix C.7).

Within this budget, we selected representative tasks with complementary physical mechanisms so that each probes a different capability: Adjust Bottle examines fine-grained pose adjustment; Click Bell examines contact and pressing; Insert HDMI examines short-horizon precision insertion; Lift Bottle examines long-horizon sustained force control; and LIBERO’s Turn on the Stove examines articulated-object manipulation. For this rebuttal, we prioritized computational resources for the multi-seed experiments and reference-method comparisons emphasized by the reviewers. The evaluation code and interfaces are also open-source, enabling the community to incorporate additional tasks under the same protocol.

I appreciate the authors' rebuttal. The authors provided clarification and additional experiments. The experiments on unseen configurations, tactile, and physical tasks help strengthen the paper. My concern about generalization is not fully address to the extend that I like, nevertheless, I think there is value in this paper. So I will maintain my original score of weak acceptance.

We sincerely thank you for your thoughtful review and for recognizing the value of our work. We truly appreciate your positive assessment and your acknowledgment that the additional experiments help strengthen the paper.

We also take your concern regarding generalization seriously. We understand that this is an important issue, and we will carefully consider your comments. We respect your decision to maintain the weak acceptance score, and we are very grateful for your time, effort, and constructive feedback.

Comments on the paper

2 comments

Dear Reviewers and Area Chair,

Thank you again for your thoughtful comments and constructive suggestions. We have uploaded a revised manuscript incorporating the additional experiments and clarifications discussed in our responses, with all revised content highlighted in blue. The main updates include:

(1) multi-seed and last-checkpoint results for the leading world models, the metric renamed to peak validation performance with an estimate of selection bias, and evaluation on independent unseen configurations and under appearance randomization;

(2) per-task reward-ranking correlations that exclude replicated baselines;

(3) adaptations of WMPO and VLA-MBPO under a unified interface, with comparisons under matched interaction and computation budgets and a rollout-horizon sensitivity analysis;

(4) an extended real-robot tactile ablation with three additional visuotactile models and a new Tofu task, together with statistical tests;

(5) clarified world-model–decoder attribution, the rationale for task selection and the cost of adding tasks, and qualified statements on sim-to-real transfer and limitations.

We sincerely appreciate the time and care you have devoted to reviewing our paper, and hope the revised manuscript addresses your concerns.

AuthorsOpenReview ↗

General Comments by Authors

Dear Area Chair and Reviewers,

Thank you for your thoughtful reviews. WorldArena 2.0 evaluates what embodied world models enable for robot control across three settings: visuotactile manipulation, policy optimization in learned environments, and execution on simulated and physical robots. Across 12 world models, these evaluations reveal when perceptual quality tracks task success, when learned dynamics provide useful training experience, and where simulation results diverge from real-robot performance. The revised manuscript incorporates the experiments and clarifications below, with all revisions highlighted in blue.

1. The benchmark reveals where perceptual scores and task success diverge

On Insert HDMI, Genie Envisioner achieves higher tactile SSIM than Vidar (0.456 vs. 0.278), yet the two systems attain 0% and 70% task success, respectively. Across platforms, Physics Adherence rankings agree between RoboTwin and the real robot (ρ = 0.811), while the association for task success is weaker (ρ = 0.348). Measuring prediction quality and task success together exposes gaps that either measure alone would miss. As Reviewer 334i noted, “Making such failures measurable and comparable is itself a meaningful contribution.”

2. Policy gains persist at the last checkpoint and on independent tests

The submitted paper compared seven learned environments across three reward designs and two RoboTwin tasks. Policies trained in all 42 configurations exceeded the supervised fine-tuning (SFT) baseline when evaluated in ground-truth RoboTwin. In response to Reviewers prVG and LGLJ, we added two training seeds for Ctrl-World and WoVR on Adjust Bottle and report peak validation, last-checkpoint, and independent-test performance separately. Under the proxy reward, the last checkpoint from each new run remained 5–11 percentage points above SFT. On unseen initial layouts, Ctrl-World and WoVR exceeded SFT by 11.7 and 4.1 points on average. With unseen textures and lighting, both reached 42.87% success, compared with 30.18% for SFT. All 12 results across two models, three seeds, and two independent test conditions exceeded SFT.

We also recomputed reward correlations separately for each task after excluding the replicated baselines. Ctrl-World and WoVR retained the top two positions across all six task and reward combinations, while the order of the other models was more sensitive to task and reward. The revision focuses the ranking claim on this consistent result for the leading models.

3. The same protocol shows how world models and optimizers interact

The submitted comparison used a shared GRPO pipeline with a fixed iteration budget. At Reviewer LGLJ’s suggestion, we added WMPO and VLA-MBPO adaptations for Ctrl-World, Wan, and OpenSora under the same benchmark interface. All six configurations exceeded SFT on Adjust Bottle. Within each adaptation, the order of the three backends held when we matched interaction volume and computation budgets separately. Across adaptations, OpenSora tied Ctrl-World under short-branch VLA-MBPO (64.94% each), while Ctrl-World led OpenSora under long-trajectory WMPO (65.33% vs. 61.43%). The new rollout study also found that doubling the horizon reduced peak success for both tested backends. These results show why evaluating a world model across optimizers and rollout lengths can reveal differences that a single training setup leaves unresolved.

4. The real-robot benefit of tactile input extends beyond FastWAM

In the submitted FastWAM ablation, adding tactile input increased mean success on Chip and Wiping from 17.5% to 37.5%. We extended the ablation to N0-VTLA, N0-TWAM, and FTP-1 on Wiping and the new Tofu transfer task, with 20 physical trials per model, task, and condition. Tactile input raised pooled success from 32/120 to 51/120 and improved success in all six new comparisons. Across the original and new experiments, tactile input helped in all eight comparisons across four models and three tasks.

The revision also explains how the selected tasks probe distinct skills, including precision insertion, sustained force control, and manipulation of fragile objects. It clarifies that the Lift Bottle result measures the combined world model and decoder system. Appendix D.3 documents the separate nine-task real-robot suite, and the released interfaces allow additional tasks to be evaluated given trajectory data and a success criterion.

Reviewer 334i noted that the new experiments on unseen configurations, tactile sensing, and physical tasks strengthened the paper. Reviewer LGLJ had no further questions following the response and asked us to include the additions in the manuscript. We appreciate the reviewers’ and Area Chair’s time and feedback.

Best regards,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.